[CodeGen] Qualify auto variables in for loops (NFC)

[MachineSink] Clear kill flags on operands outside loop

If an instruction is sunk into a loop then any kill flags on
operands declared outside the loop must be cleared as these
will be live for all

[MachineSink] Clear kill flags on operands outside loop

If an instruction is sunk into a loop then any kill flags on
operands declared outside the loop must be cleared as these
will be live for all loop iterations.

Fixes #46827

Reviewed By: MatzeB

Differential Revision: https://reviews.llvm.org/D126754

show more ...

[machinesink] fix debug invariance issue

Do not include debug instructions when comparing block sizes with
thresholds.

Differential Revision: https://reviews.llvm.org/D127208

[CodeGen] Fix the bug of machine sink

The use operand may be undefined. In that case we can just continue to
check the next operand since it won't increase register pressure.

Differential Revision:

[CodeGen] Fix the bug of machine sink

The use operand may be undefined. In that case we can just continue to
check the next operand since it won't increase register pressure.

Differential Revision: https://reviews.llvm.org/D127848

show more ...

[MachineSink] replace MachineLoop with MachineCycle

reapply 62a9b36fcf728b104ea87e6eb84c0be69b779df7 and fix module build
failue:
1: remove MachineCycleInfoWrapperPass in MachinePassRegistry.def

[MachineSink] replace MachineLoop with MachineCycle

reapply 62a9b36fcf728b104ea87e6eb84c0be69b779df7 and fix module build
failue:
1: remove MachineCycleInfoWrapperPass in MachinePassRegistry.def
MachineCycleInfoWrapperPass is a anylysis pass, should not be there.
2: move the definition for MachineCycleInfoPrinterPass to cpp file.

Otherwise, there are module conflicit for MachineCycleInfoWrapperPass
in MachinePassRegistry.def and MachineCycleAnalysis.h after
62a9b36fcf728b104ea87e6eb84c0be69b779df7.

MachineCycle can handle irreducible loop. Natural loop
analysis (MachineLoop) can not return correct loop depth if
the loop is irreducible loop. And MachineSink is sensitive
to the loop depth, see MachineSinking::isProfitableToSinkTo().

This patch tries to use MachineCycle so that we can handle
irreducible loop better.

Reviewed By: sameerds, MatzeB

Differential Revision: https://reviews.llvm.org/D123995

show more ...

Revert "[MachineSink] replace MachineLoop with MachineCycle"

This reverts commit 62a9b36fcf728b104ea87e6eb84c0be69b779df7.
Cause build failure on lldb incremental buildbot:
https://green.lab.llvm.or

Revert "[MachineSink] replace MachineLoop with MachineCycle"

This reverts commit 62a9b36fcf728b104ea87e6eb84c0be69b779df7.
Cause build failure on lldb incremental buildbot:
https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/43994/changes

show more ...

[MachineSink] replace MachineLoop with MachineCycle

MachineCycle can handle irreducible loop. Natural loop
analysis (MachineLoop) can not return correct loop depth if
the loop is irreducible loop. A

[MachineSink] replace MachineLoop with MachineCycle

MachineCycle can handle irreducible loop. Natural loop
analysis (MachineLoop) can not return correct loop depth if
the loop is irreducible loop. And MachineSink is sensitive
to the loop depth, see MachineSinking::isProfitableToSinkTo().

This patch tries to use MachineCycle so that we can handle
irreducible loop better.

Reviewed By: sameerds, MatzeB

Differential Revision: https://reviews.llvm.org/D123995

show more ...

[MachineSink] Check block prologue interference

Sinking must check for interference between the block prologue
and the instruction being sunk.
Specifically check for clobbering of uses by the prolog

[MachineSink] Check block prologue interference

Sinking must check for interference between the block prologue
and the instruction being sunk.
Specifically check for clobbering of uses by the prologue, and
overwrites to prologue defined registers by the sunk instruction.

Reviewed By: rampitec, ruiling

Differential Revision: https://reviews.llvm.org/D121277

show more ...

Cleanup codegen includes

This is a (fixed) recommit of https://reviews.llvm.org/D121169

after: 1061034926
before: 1063332844

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-in

Cleanup codegen includes

This is a (fixed) recommit of https://reviews.llvm.org/D121169

after: 1061034926
before: 1063332844

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D121681

show more ...

Revert "Cleanup codegen includes"

This reverts commit 7f230feeeac8a67b335f52bd2e900a05c6098f20.
Breaks CodeGenCUDA/link-device-bitcode.cu in check-clang,
and many LLVM tests, see comments on https:/

Revert "Cleanup codegen includes"

This reverts commit 7f230feeeac8a67b335f52bd2e900a05c6098f20.
Breaks CodeGenCUDA/link-device-bitcode.cu in check-clang,
and many LLVM tests, see comments on https://reviews.llvm.org/D121169

show more ...

Cleanup codegen includes

after: 1061034926
before: 1063332844

Differential Revision: https://reviews.llvm.org/D121169

[MachineSink] Disable if there are any irreducible cycles

This is an alternative to D120330, which disables MachineSink for
functions with irreducible cycles entirely. This avoids both the
correctne

[MachineSink] Disable if there are any irreducible cycles

This is an alternative to D120330, which disables MachineSink for
functions with irreducible cycles entirely. This avoids both the
correctness problem, and ensures we don't perform non-profitable
sinks into cycles. At the same time, it may also disable
profitable sinks in the same function. This can be made more
precise by using MachineCycleInfo in the future.

Fixes https://github.com/llvm/llvm-project/issues/53990.

Differential Revision: https://reviews.llvm.org/D120800

show more ...

[MachineSink] Use SkipPHIsAndLabels for sink insertion points

For AMDGPU the insertion point for a block may not be the first
non-PHI instruction. This happens when a block contains EXEC
mask manip

[MachineSink] Use SkipPHIsAndLabels for sink insertion points

For AMDGPU the insertion point for a block may not be the first
non-PHI instruction. This happens when a block contains EXEC
mask manipulation related to control flow (converging lanes).

Use SkipPHIsAndLabels to determine the block insertion point
so that the target can skip any block prologue instructions.

Reviewed By: rampitec, ruiling

Differential Revision: https://reviews.llvm.org/D119399

show more ...

[MachineSink] Inline getRegUnits

Reg unit sets are uniqued, so no need to wrap it in a set.

[MachineSink] Allow sinking of constant or ignorable physreg uses

For AMDGPU, any use of the physical register EXEC prevents sinking even if it is not a real physical register read. Add check to see

[MachineSink] Allow sinking of constant or ignorable physreg uses

For AMDGPU, any use of the physical register EXEC prevents sinking even if it is not a real physical register read. Add check to see if a physical
register use can be ignored for sinking.

Also perform same constant and ignorable physical register check when considering sinking in loops.

https://reviews.llvm.org/D116053

show more ...

[llvm] Use range-based for loops (NFC)

Fix minor deficiency in machine-sink.

Register uses that are MRI->isConstantPhysReg() should not inhibit
sinking transformation.

Reviewed By: StephenTozer

Differential Revision: https://reviews.ll

Fix minor deficiency in machine-sink.

Register uses that are MRI->isConstantPhysReg() should not inhibit
sinking transformation.

Reviewed By: StephenTozer

Differential Revision: https://reviews.llvm.org/D111531

show more ...

[llvm] Use MachineBasicBlock::{successors,predecessors} (NFC)

[llvm] Use llvm::reverse (NFC)

[CodeGen] Use make_early_inc_range (NFC)

[MachineSink] Compile time improvement for large testcases which has many kill flags

We did a experiment and observed dramatic decrease on compilation time which spent on clearing kill flags.
Before

[MachineSink] Compile time improvement for large testcases which has many kill flags

We did a experiment and observed dramatic decrease on compilation time which spent on clearing kill flags.
Before:
Number of BasicBlocks:33357
Number of Instructions:162067
Number of Cleared Kill Flags:32869
Time of handling kill flags(ms):1.607509e+05

After:
Number of BasicBlocks:33357
Number of Instructions:162067
Number of Cleared Kill Flags:32869
Time of handling kill flags:3.987371e+03

Reviewed By: MatzeB

Differential Revision: https://reviews.llvm.org/D111688

show more ...

[DebugInfo][InstrRef][4/4] Support DBG_INSTR_REF through all backend passes

This is a cleanup patch -- we're now able to support all flavours of
variable location in instruction referencing mode. Th

[DebugInfo][InstrRef][4/4] Support DBG_INSTR_REF through all backend passes

This is a cleanup patch -- we're now able to support all flavours of
variable location in instruction referencing mode. This patch updates
various tests for debug instructions to be broader: numerous code paths
try to ignore debug isntructions, and they now have to ignore the
additional DBG_PHI and DBG_INSTR_REFs that we can generate.

A small amount of rework happens for LiveDebugVariables: as we don't need
to track live intervals through regalloc any more, we can get away with
unlinking debug instructions before regalloc, then re-inserting them after.
Note that this isn't (yet) true of DBG_VALUE_LISTs, they still have to go
through live interval tracking.

In SelectionDAG, add a helper lambda that emits half-formed DBG_INSTR_REFs
for arguments in instr-ref mode, DBG_VALUE otherwise. This is one of the
final locations where DBG_VALUEs are emitted for vreg arguments.

X86InstrInfo now un-sets the debug instr number on SUB instructions that
get mutated into CMP instructions. As the instruction no longer computes a
subtraction, we can't use it for variable locations.

Differential Revision: https://reviews.llvm.org/D88898

show more ...

[CSSPGO] Exclude pseudo probes from slot index

Pseudo probe are currently given a slot index like other regular instructions. This affects register pressure and lifetime weight computation because o

[CSSPGO] Exclude pseudo probes from slot index

Pseudo probe are currently given a slot index like other regular instructions. This affects register pressure and lifetime weight computation because of enlarged lifetime length with pseudo probe instructions. As a consequence, program could get different code generated w/ and w/o pseudo probes. I'm closing the gap by excluding pseudo probes from stack index and downstream register allocation related passes.

Reviewed By: wmi

Differential Revision: https://reviews.llvm.org/D100334

show more ...

[DebugInfo] Handle DBG_VALUES with multiple variable location operands in MIR

This patch adds handling for DBG_VALUE_LIST in the MIR-passes (after
finalize-isel), excluding the debug liveness passes

[DebugInfo] Handle DBG_VALUES with multiple variable location operands in MIR

This patch adds handling for DBG_VALUE_LIST in the MIR-passes (after
finalize-isel), excluding the debug liveness passes and DWARF emission. This
most significantly affects MachineSink, which now needs to consider all used
registers of a debug value when sinking, but for most passes this change is
simply replacing getDebugOperand(0) with an iteration over all debug operands.

Differential Revision: https://reviews.llvm.org/D92578

show more ...

[CodeGen] Use range-based for loops (NFC)