Fix regalloc assignment of overlapping registers

SplitEditor::defFromParent() can create a register copy.
If register is a tuple of other registers and not all lanes are used
a copy will be done on

Fix regalloc assignment of overlapping registers

SplitEditor::defFromParent() can create a register copy.
If register is a tuple of other registers and not all lanes are used
a copy will be done on a full tuple regardless. Later register unit
for an unused lane will be considered free and another overlapping
register tuple can be assigned to a different value even though first
register is live at that point. That is because interference only look at
liveness info, while full register copy clobbers all lanes, even unused.

This patch fixes copy to only cover used lanes.

Differential Revision: https://reviews.llvm.org/D29105

llvm-svn: 293728

show more ...

Implement LaneBitmask::any(), use it to replace !none(), NFCI

llvm-svn: 289974

Extract LaneBitmask into a separate type

Specifically avoid implicit conversions from/to integral types to
avoid potential errors when changing the underlying type. For example,
a typical initializa

Extract LaneBitmask into a separate type

Specifically avoid implicit conversions from/to integral types to
avoid potential errors when changing the underlying type. For example,
a typical initialization of a "full" mask was "LaneMask = ~0u", which
would result in a value of 0x00000000FFFFFFFF if the type was extended
to uint64_t.

Differential Revision: https://reviews.llvm.org/D27454

llvm-svn: 289820

show more ...

Check proper live range in extendPHIRanges

The function extendPHIRanges checks the main range of the original live
interval, even when dealing with a subrange. This could also lead to an
assert when

Check proper live range in extendPHIRanges

The function extendPHIRanges checks the main range of the original live
interval, even when dealing with a subrange. This could also lead to an
assert when the subrange is not live at the extension point, but the
main range is. To avoid this, check the corresponding subrange of the
original live range, instead of always checking the main range.

Review (as a part of a bigger set of changes):
https://reviews.llvm.org/D26359

llvm-svn: 287571

show more ...

Place the lowered phi instruction(s) before the DEBUG_VALUE entry

When a phi node is finally lowered to a machine instruction it is
important that the lowered "load" instruction is placed before the

Place the lowered phi instruction(s) before the DEBUG_VALUE entry

When a phi node is finally lowered to a machine instruction it is
important that the lowered "load" instruction is placed before the
associated DEBUG_VALUE entry describing the value loaded.

Renamed the existing SkipPHIsAndLabels to SkipPHIsLabelsAndDebug to
more fully describe that it also skips debug entries. Then used the
"new" function SkipPHIsAndLabels when the debug information should not
be skipped when placing the lowered "load" instructions so that it is
placed before the debug entries.

Differential Revision: https://reviews.llvm.org/D23760

llvm-svn: 281727

show more ...

Do not consider subreg defs as reads when computing subrange liveness

Subregister definitions are considered uses for the purpose of tracking
liveness of the whole register. At the same time, when c

Do not consider subreg defs as reads when computing subrange liveness

Subregister definitions are considered uses for the purpose of tracking
liveness of the whole register. At the same time, when calculating live
interval subranges, subregister defs should not be treated as uses.

Differential Revision: https://reviews.llvm.org/D24190

llvm-svn: 280532

show more ...

Make buildbots happy.

"warning: extra ‘;’ [-Wpedantic]"

llvm-svn: 279703

Create subranges for new intervals resulting from live interval splitting

The register allocator can split a live interval of a register into a set
of smaller intervals. After the allocation of regi

Create subranges for new intervals resulting from live interval splitting

The register allocator can split a live interval of a register into a set
of smaller intervals. After the allocation of registers is complete, the
rewriter will modify the IR to replace virtual registers with the corres-
ponding physical registers. At this stage, if a register corresponding
to a subregister of a virtual register is used, the rewriter will check
if that subregister is undefined, and if so, it will add the <undef> flag
to the machine operand. The function verifying liveness of the subregis-
ter would assume that it is undefined, unless any of the subranges of the
live interval proves otherwise.
The problem is that the live intervals created during splitting do not
have any subranges, even if the original parent interval did. This could
result in the <undef> flag placed on a register that is actually defined.

Differential Revision: http://reviews.llvm.org/D21189

llvm-svn: 279625

show more ...

Allow dead insts to be kept in DeadRemat only when they are rematerializable.

Because isReallyTriviallyReMaterializableGeneric puts many limits on
rematerializable instructions, this fix can prevent

Allow dead insts to be kept in DeadRemat only when they are rematerializable.

Because isReallyTriviallyReMaterializableGeneric puts many limits on
rematerializable instructions, this fix can prevent instructions with
tied virtual operands and instructions with virtual register uses from
being kept in DeadRemat, so as to workaround the live interval consistency
problem for the dummy instructions kept in DeadRemat.

But we still need to fix the live interval consistency problem. This patch
is just a short time relieve. PR28464 has been filed as a reminder.

Differential Revision: http://reviews.llvm.org/D19486

llvm-svn: 274928

show more ...

InsertPointAnalysis: Move current live interval from being a class member
to query interfaces argument; NFC

Differential Revision: http://reviews.llvm.org/D20532

llvm-svn: 270481

[NFC] Extract LastSplitPoint computation from SplitAnalysis to a new class
InsertPointAnalysis.

Because both split and spill hoisting want to use LastSplitPoint computation
result, extract the LastS

[NFC] Extract LastSplitPoint computation from SplitAnalysis to a new class
InsertPointAnalysis.

Because both split and spill hoisting want to use LastSplitPoint computation
result, extract the LastSplitPoint computation from SplitAnalysis class which
also contains a bunch of other analysises only related to split.

Differential Revision: http://reviews.llvm.org/D20027.

llvm-svn: 269248

show more ...

[WinEH] Update SplitAnalysis::computeLastSplitPoint to cope with multiple EH successors

We didn't have logic to correctly handle CFGs where there was more than
one EH-pad successor (these are novel

[WinEH] Update SplitAnalysis::computeLastSplitPoint to cope with multiple EH successors

We didn't have logic to correctly handle CFGs where there was more than
one EH-pad successor (these are novel with WinEH).
There were situations where a register was live in one exceptional
successor but not another but the code as written would only consider
the first exceptional successor it found.

This resulted in split points which were insufficiently early if an
invoke was present.

This fixes PR27501.

N.B. This removes getLandingPadSuccessor.

llvm-svn: 267412

show more ...

Recommit r265547, and r265610,r265639,r265657 on top of it, plus
two fixes with one about error verify-regalloc reported, and
another about live range update of phi after rematerialization.

r265547:

Recommit r265547, and r265610,r265639,r265657 on top of it, plus
two fixes with one about error verify-regalloc reported, and
another about live range update of phi after rematerialization.

r265547:
Replace analyzeSiblingValues with new algorithm to fix its compile
time issue. The patch is to solve PR17409 and its duplicates.

analyzeSiblingValues is a N x N complexity algorithm where N is
the number of siblings generated by reg splitting. Although it
causes siginificant compile time issue when N is large, it is also
important for performance since it removes redundent spills and
enables rematerialization.

To solve the compile time issue, the patch removes analyzeSiblingValues
and replaces it with lower cost alternatives containing two parts. The
first part creates a new spill hoisting method in postOptimization of
register allocation. It does spill hoisting at once after all the spills
are generated instead of inside every instance of selectOrSplit. The
second part queries the define expr of the original register for
rematerializaiton and keep it always available during register allocation
even if it is already dead. It deletes those dead instructions only in
postOptimization. With the two parts in the patch, it can remove
analyzeSiblingValues without sacrificing performance.

Patches on top of r265547:
r265610 "Fix the compare-clang diff error introduced by r265547."
r265639 "Fix the sanitizer bootstrap error in r265547."
r265657 "InlineSpiller.cpp: Escap \@ in r265547. [-Wdocumentation]"

Differential Revision: http://reviews.llvm.org/D15302
Differential Revision: http://reviews.llvm.org/D18934
Differential Revision: http://reviews.llvm.org/D18935
Differential Revision: http://reviews.llvm.org/D18936

llvm-svn: 266162

show more ...

Revert r265547 "Recommit r265309 after fixed an invalid memory reference bug happened"

It caused PR27275: "ARM: Bad machine code: Using an undefined physical register"

Also reverting the following

Revert r265547 "Recommit r265309 after fixed an invalid memory reference bug happened"

It caused PR27275: "ARM: Bad machine code: Using an undefined physical register"

Also reverting the following commits that were landed on top:
r265610 "Fix the compare-clang diff error introduced by r265547."
r265639 "Fix the sanitizer bootstrap error in r265547."
r265657 "InlineSpiller.cpp: Escap \@ in r265547. [-Wdocumentation]"

llvm-svn: 265790

show more ...

Recommit r265309 after fixed an invalid memory reference bug happened
when DenseMap growed and moved memory. I verified it fixed the bootstrap
problem on x86_64-linux-gnu but I cannot verify whether

Recommit r265309 after fixed an invalid memory reference bug happened
when DenseMap growed and moved memory. I verified it fixed the bootstrap
problem on x86_64-linux-gnu but I cannot verify whether it fixes
the bootstrap error on clang-ppc64be-linux. I will watch the build-bot
result closely.

Replace analyzeSiblingValues with new algorithm to fix its compile
time issue. The patch is to solve PR17409 and its duplicates.

analyzeSiblingValues is a N x N complexity algorithm where N is
the number of siblings generated by reg splitting. Although it
causes siginificant compile time issue when N is large, it is also
important for performance since it removes redundent spills and
enables rematerialization.

To solve the compile time issue, the patch removes analyzeSiblingValues
and replaces it with lower cost alternatives containing two parts. The
first part creates a new spill hoisting method in postOptimization of
register allocation. It does spill hoisting at once after all the spills
are generated instead of inside every instance of selectOrSplit. The
second part queries the define expr of the original register for
rematerializaiton and keep it always available during register allocation
even if it is already dead. It deletes those dead instructions only in
postOptimization. With the two parts in the patch, it can remove
analyzeSiblingValues without sacrificing performance.

Differential Revision: http://reviews.llvm.org/D15302

llvm-svn: 265547

show more ...

Revert r265309 and r265312 because they caused some errors I need to investigate.

llvm-svn: 265317

Replace analyzeSiblingValues with new algorithm to fix its compile
time issue. The patch is to solve PR17409 and its duplicates.

analyzeSiblingValues is a N x N complexity algorithm where N is
the n

Replace analyzeSiblingValues with new algorithm to fix its compile
time issue. The patch is to solve PR17409 and its duplicates.

analyzeSiblingValues is a N x N complexity algorithm where N is
the number of siblings generated by reg splitting. Although it
causes siginificant compile time issue when N is large, it is also
important for performance since it removes redundent spills and
enables rematerialization.

To solve the compile time issue, the patch removes analyzeSiblingValues
and replaces it with lower cost alternatives containing two parts. The
first part creates a new spill hoisting method in postOptimization of
register allocation. It does spill hoisting at once after all the spills
are generated instead of inside every instance of selectOrSplit. The
second part queries the define expr of the original register for
rematerializaiton and keep it always available during register allocation
even if it is already dead. It deletes those dead instructions only in
postOptimization. With the two parts in the patch, it can remove
analyzeSiblingValues without sacrificing performance.

Differential Revision: http://reviews.llvm.org/D15302

llvm-svn: 265309

show more ...

CodeGen: Take MachineInstr& in SlotIndexes and LiveIntervals, NFC

Take MachineInstr by reference instead of by pointer in SlotIndexes and
the SlotIndex wrappers in LiveIntervals. The MachineInstrs

CodeGen: Take MachineInstr& in SlotIndexes and LiveIntervals, NFC

Take MachineInstr by reference instead of by pointer in SlotIndexes and
the SlotIndex wrappers in LiveIntervals. The MachineInstrs here are
never null, so this cleans up the API a bit. It also incidentally
removes a few implicit conversions from MachineInstrBundleIterator to
MachineInstr* (see PR26753).

At a couple of call sites it was convenient to convert to a range-based
for loop over MachineBasicBlock::instr_begin/instr_end, so I added
MachineBasicBlock::instrs.

llvm-svn: 262115

show more ...

Annotate dump() methods with LLVM_DUMP_METHOD, addressing Richard Smith r259192 post commit comment.
clang part in r259232, this is the LLVM part of the patch.

llvm-svn: 259240

CodeGen: Remove implicit ilist iterator conversions, NFC

Finish removing implicit ilist iterator conversions from LLVMCodeGen.
I'm sure there are lots more of these in lib/CodeGen/*/.

llvm-svn: 249

CodeGen: Remove implicit ilist iterator conversions, NFC

Finish removing implicit ilist iterator conversions from LLVMCodeGen.
I'm sure there are lots more of these in lib/CodeGen/*/.

llvm-svn: 249915

show more ...

LiveIntervalAnalysis: Factor common code into splitSeparateComponents; NFC

llvm-svn: 248241

[WinEH] Add and use hasEHPadSuccessor instead of getLandingPadSuccessor

getLandingPadSuccessor assumes that each invoke can have at most one EH
pad successor, but WinEH invokes can have more than on

[WinEH] Add and use hasEHPadSuccessor instead of getLandingPadSuccessor

getLandingPadSuccessor assumes that each invoke can have at most one EH
pad successor, but WinEH invokes can have more than one. Two out of
three callers of getLandingPadSuccessor don't use the returned
landingpad, so we can make them use this simple predicate instead.

Eventually we'll have to circle back and fix SplitKit.cpp so that
register allocation works. Baby steps.

llvm-svn: 247904

show more ...

LiveIntervalAnalysis: Factor out code to update liveness on vreg def removal

This cleans up code and is more in line with the general philosophy of
modifying LiveIntervals through LiveIntervalAnalys

LiveIntervalAnalysis: Factor out code to update liveness on vreg def removal

This cleans up code and is more in line with the general philosophy of
modifying LiveIntervals through LiveIntervalAnalysis instead of changing
them directly.

This also fixes a case where SplitEditor::removeBackCopies() would miss
the subregister ranges.

llvm-svn: 226690

show more ...

LiveInterval: Use more range based for loops for value numbers and segments.

llvm-svn: 223978

Grab the subtarget and subtarget dependent variables off of
MachineFunction rather than TargetMachine.

llvm-svn: 219671