[Statepoints] FixupStatepoint: Clear isKill flag if COPY is not deleted.

When spilling CSRs, FixupStatepoint pass does simple copy propagation,
trying to find COPY instruction which defines register

[Statepoints] FixupStatepoint: Clear isKill flag if COPY is not deleted.

When spilling CSRs, FixupStatepoint pass does simple copy propagation,
trying to find COPY instruction which defines register being spilled
and spill COPY source instead. I.e., if we have CSR $x and found
$x = COPY $y
we will spill $y instead.
But we may be unable to delete COPY instruction for some reason.
Then, spill will be inserted after it, adding another use of $y.
If COPY instruction was last use of $y (killed it), after insertion of
the spill it is not, so `isKill` flag must be cleared. We failed to do
so and this patch fixes this issue.

Reviewed By: skatkov

Differential Revision: https://reviews.llvm.org/D127308

show more ...

Cleanup codegen includes

This is a (fixed) recommit of https://reviews.llvm.org/D121169

after: 1061034926
before: 1063332844

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-in

Cleanup codegen includes

This is a (fixed) recommit of https://reviews.llvm.org/D121169

after: 1061034926
before: 1063332844

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D121681

show more ...

Revert "Cleanup codegen includes"

This reverts commit 7f230feeeac8a67b335f52bd2e900a05c6098f20.
Breaks CodeGenCUDA/link-device-bitcode.cu in check-clang,
and many LLVM tests, see comments on https:/

Revert "Cleanup codegen includes"

This reverts commit 7f230feeeac8a67b335f52bd2e900a05c6098f20.
Breaks CodeGenCUDA/link-device-bitcode.cu in check-clang,
and many LLVM tests, see comments on https://reviews.llvm.org/D121169

show more ...

Cleanup codegen includes

after: 1061034926
before: 1063332844

Differential Revision: https://reviews.llvm.org/D121169

[llvm][clang][NFC] updates inline licence info

Some files still contained the old University of Illinois Open Source
Licence header. This patch replaces that with the Apache 2 with LLVM
Exception li

[llvm][clang][NFC] updates inline licence info

Some files still contained the old University of Illinois Open Source
Licence header. This patch replaces that with the Apache 2 with LLVM
Exception licence.

Differential Revision: https://reviews.llvm.org/D107528

show more ...

[Statepoint] Handle 'undef' operand tied to def

FixupStatepoints pass does not take into account the undef use
it skips may have a tied def. So when defs are handled pass
considers that tied-use sho

[Statepoint] Handle 'undef' operand tied to def

FixupStatepoints pass does not take into account the undef use
it skips may have a tied def. So when defs are handled pass
considers that tied-use should be spilled and triggers an assert.

FixupStatepoints should skip undef def as well.

Reviewers: reames, dantrushin
Reviewed By: dantrushin
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D95858

show more ...

[Statepoint] Handle `undef` operands in statepoint.

Currently when spilling statepoint register operands in FixupStatepoints
we do not pay attention that it might be `undef`. We just generate a
spil

[Statepoint] Handle `undef` operands in statepoint.

Currently when spilling statepoint register operands in FixupStatepoints
we do not pay attention that it might be `undef`. We just generate a
spill, which may lead to verifier error because we have a use without def.

To handle it, let FixupStateponts ignore `undef` register operands
completely and change them to some constant value when generating
stack map. Use same value as used by ISel for this purpose (0xFEFEFEFE).

Reviewed By: reames

Differential Revision: https://reviews.llvm.org/D94703

show more ...

[llvm] Use *::empty (NFC)

[CodeGen, Transforms] Use llvm::sort (NFC)

[Statepoints] Change statepoint machine instr format to better suit VReg lowering.

Current Statepoint MI format is this:

STATEPOINT
<id>, <num patch bytes >, <num call arguments>, <call targe

[Statepoints] Change statepoint machine instr format to better suit VReg lowering.

Current Statepoint MI format is this:

STATEPOINT
<id>, <num patch bytes >, <num call arguments>, <call target>,
[call arguments...],
<StackMaps::ConstantOp>, <calling convention>,
<StackMaps::ConstantOp>, <statepoint flags>,
<StackMaps::ConstantOp>, <num deopt args>, [deopt args...],
<gc base/derived pairs...> <gc allocas...>

Note that GC pointers are listed in pairs <base,derived>.
This causes base pointers to appear many times (at least twice) in
instruction, which is bad for us when VReg lowering is ON.
The problem is that machine operand tiedness is 1-1 relation, so
it might look like this:

%vr2 = STATEPOINT ... %vr1, %vr1(tied-def0)

Since only one instance of %vr1 is tied, that may lead to incorrect
codegen (see PR46917 for more details), so we have to always spill
base pointers. This mostly defeats new VReg lowering scheme.

This patch changes statepoint instruction format so that every
gc pointer appears only once in operand list. That way they all can
be tied. Additional set of operands is added to preserve base-derived
relation required to build stackmap.
New statepoint has following format:

STATEPOINT
<id>, <num patch bytes>, <num call arguments>, <call target>,
[call arguments...],
<StackMaps::ConstantOp>, <calling convention>,
<StackMaps::ConstantOp>, <statepoint flags>,
<StackMaps::ConstantOp>, <num deopt args>, [deopt args...],
<StackMaps::ConstantOp>, <num gc pointers>, [gc pointers...],
<StackMaps::ConstantOp>, <num gc allocas>, [gc allocas...]
<StackMaps::ConstantOp>, <num entries in gc map>, [base/derived indices...]

Changes are:
- every gc pointer is listed only once in a flat length-prefixed list;
- alloca list is prefixed with its length too;
- following alloca list is length-prefixed list of base-derived
indices of pointers from gc pointer list. Note that indices are
logical (number of pointer), not absolute (index of machine operand).

Differential Revision: https://reviews.llvm.org/D87154

show more ...

[NFC] Silence variables unused in release builds

[Statepoints] FixupStatepoint: properly set isKill on spilled register.

When spilling statepoint meta arg register it is incorrect to blindly
mark it as killed - it may be used in non-meta args (e.g

[Statepoints] FixupStatepoint: properly set isKill on spilled register.

When spilling statepoint meta arg register it is incorrect to blindly
mark it as killed - it may be used in non-meta args (e.g., as call
parameter).

show more ...

[Statepoints] Spill GC Ptr regs in FixupStatepoints.

Extend FixupStatepointCallerSaved pass with ability to spill
statepoint GC pointer arguments (optionally allowing them on CSRs).
Special handling

[Statepoints] Spill GC Ptr regs in FixupStatepoints.

Extend FixupStatepointCallerSaved pass with ability to spill
statepoint GC pointer arguments (optionally allowing them on CSRs).
Special handling is required for invoke statepoints, because at MI
level single landing pad may be shared by multiple statepoints, so
we must ensure we spill landing pad's live-ins into the same stack
slots.

Full statepoint refactoring change set is available at D81603.

Reviewed By: skatkov

Differential Revision: https://reviews.llvm.org/D81647

show more ...

[Alignment][NFC] Migrate MachineFrameInfo::CreateSpillStackObject to Align

iThis patch is part of a series to introduce an Alignment type.
See this thread for context: http://lists.llvm.org/pipermai

[Alignment][NFC] Migrate MachineFrameInfo::CreateSpillStackObject to Align

iThis patch is part of a series to introduce an Alignment type.
See this thread for context: http://lists.llvm.org/pipermail/llvm-dev/2019-July/133851.html
See this patch for the introduction of the type: https://reviews.llvm.org/D64790

Differential Revision: https://reviews.llvm.org/D82934

show more ...

[Statepoint] Mark FixupStatepointCallerSaved as preserving the CFG

I'm hoping this will restore some compile time lost by D75936 and D75937.

Differential Revision: https://reviews.llvm.org/D79813

Give helpers internal linkage. NFC.

[Statepoint] Add getters to StatepointOpers.

To simplify future work on statepoint representation, hide
direct access to statepoint field indices and provide getters
for them. Add getters for couple

[Statepoint] Add getters to StatepointOpers.

To simplify future work on statepoint representation, hide
direct access to statepoint field indices and provide getters
for them. Add getters for couple more statepoint fields.

This also fixes two bugs in MachineVerifier for statepoint:
First, the `break` statement was falling out of `if` statement
scope, thus disabling following checks.
Second, it was incorrectly accessing some fields like CallingConv -
StatepointOpers gives index to their value directly, not to
preceeding field type encoding.

Reviewed By: skatkov
Differential Revision: https://reviews.llvm.org/D78119

show more ...

Re-land [Codegen/Statepoint] Allow usage of registers for non gc deopt values.

The change introduces the usage of physical registers for non-gc deopt values.
This require runtime support to know how

Re-land [Codegen/Statepoint] Allow usage of registers for non gc deopt values.

The change introduces the usage of physical registers for non-gc deopt values.
This require runtime support to know how to take a value from register.
By default usage is off and can be switched on by option.

The change also introduces additional fix-up patch which forces the spilling
of caller saved registers (clobbered after the call) and re-writes statepoint
to use spill slots instead of caller saved registers.

Reviewers: reames, danstrushin
Reviewed By: dantrushin
Subscribers: mgorny, hiraditya, mgrang, llvm-commits
Differential Revision: https://reviews.llvm.org/D77797

show more ...

[Codegen/Statepoint] Allow usage of registers for non gc deopt values.

The change introduces the usage of physical registers for non-gc deopt values.
This require runtime support to know how to take

[Codegen/Statepoint] Allow usage of registers for non gc deopt values.

The change introduces the usage of physical registers for non-gc deopt values.
This require runtime support to know how to take a value from register.
By default usage is off and can be switched on by option.

The change also introduces additional fix-up patch which forces the spilling
of caller saved registers (clobbered after the call) and re-writes statepoint
to use spill slots instead of caller saved registers.

Reviewers: reames, dantrushin
Reviewed By: reames, dantrushin
Subscribers: mgorny, hiraditya, mgrang, llvm-commits
Differential Revision: https://reviews.llvm.org/D77371

show more ...