[CodeGen] Increase the size of a SmallVector

The SmallVector reserve() call in
MachineInstrExpressionTrait::getHashValue accounted for over 3% of all
calls to malloc() when I compiled a bunch of gra

[CodeGen] Increase the size of a SmallVector

The SmallVector reserve() call in
MachineInstrExpressionTrait::getHashValue accounted for over 3% of all
calls to malloc() when I compiled a bunch of graphics shaders for the
AMDGPU target. Its initial size was only enough for machine instructions
with up to 7 operands, but for AMDGPU 8 and 10 operands are very common.
Here's a histogram of number of operands for each call to getHashValue,
gathered from the same collection of shaders:

1 13503
2 254273
3 135781
4 422508
5 614997
6 194953
7 287248
8 1517255
9 31218
10 1191269
11 70731
12 24
13 77
15 84
17 4692
27 16
33 705
49 6

Typical instructions with 8 and 10 operands are floating point
arithmetic and multiply-accumulate instructions like:

%83:vgpr_32 = V_MUL_F32_e64 0, killed %82:vgpr_32, 0, killed %81:vgpr_32, 0, 0, implicit $exec
%330:vgpr_32 = V_MAC_F32_e64 0, killed %327:vgpr_32, 0, killed %329:sgpr_32, 0, %328:vgpr_32(tied-def 0), 0, 0, implicit $exec

Differential Revision: https://reviews.llvm.org/D70301

show more ...

[MIR] Add MIR parsing for heap alloc site instruction markers

Summary:
This patch adds MIR parsing and printing for heap alloc markers, which were
added in D69136. They are printed as an operand sim

[MIR] Add MIR parsing for heap alloc site instruction markers

Summary:
This patch adds MIR parsing and printing for heap alloc markers, which were
added in D69136. They are printed as an operand similar to pre-/post-instr
symbols, with a heap-alloc-marker token and a metadata node.

Reviewers: rnk

Subscribers: hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D69864

show more ...

Fix unused variable warning. NFCI.

Recommit "Add a heap alloc site marker field to the ExtraInfo in MachineInstrs"

Summary:
Fixes some things from original commit at https://reviews.llvm.org/D69136. The main
change is that the heap a

Recommit "Add a heap alloc site marker field to the ExtraInfo in MachineInstrs"

Summary:
Fixes some things from original commit at https://reviews.llvm.org/D69136. The main
change is that the heap alloc marker is always stored as ExtraInfo in the machine
instruction instead of in the PointerSumType because it cannot hold more than
4 pointer types.

Add instruction marker to MachineInstr ExtraInfo. This does almost the
same thing as Pre/PostInstrSymbols, except that it doesn't create a label until
printing instructions. This allows for labels to be put around instructions that
are deleted/duplicated somewhere.
Use this marker to track heap alloc site call instructions.

Reviewers: rnk

Subscribers: MatzeB, hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D69536

show more ...

Revert "Add an instruction marker field to the ExtraInfo in MachineInstrs."

Reverting commit b85b4e5a6f8579c137fecb59a4d75d7bfb111f79 due to some
buildbot failures/ out of memory errors.

Add an instruction marker field to the ExtraInfo in MachineInstrs.

Summary:
Add instruction marker to MachineInstr ExtraInfo. This does almost the
same thing as Pre/PostInstrSymbols, except that it

Add an instruction marker field to the ExtraInfo in MachineInstrs.

Summary:
Add instruction marker to MachineInstr ExtraInfo. This does almost the
same thing as Pre/PostInstrSymbols, except that it doesn't create a label until
printing instructions. This allows for labels to be put around instructions that
are deleted/duplicated somewhere.

Also undo the workaround in r375137.

Reviewers: rnk

Subscribers: MatzeB, hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D69136

show more ...

Prune two MachineInstr.h includes, fix up deps

MachineInstr.h included AliasAnalysis.h, which includes a world of IR
constructs mostly unneeded in CodeGen. Prune it. Same for
DebugInfoMetadata.h.

N

Prune two MachineInstr.h includes, fix up deps

MachineInstr.h included AliasAnalysis.h, which includes a world of IR
constructs mostly unneeded in CodeGen. Prune it. Same for
DebugInfoMetadata.h.

Noticed with -ftime-trace.

llvm-svn: 375311

show more ...

Remove the AliasAnalysis argument in function areMemAccessesTriviallyDisjoint

Reviewers:
arsenm

Differential Revision:
https://reviews.llvm.org/D58360

llvm-svn: 373024

[DebugInfo] LiveDebugValues: correctly discriminate kinds of variable locations

The missing line added by this patch ensures that only spilt variable
locations are candidates for being restored from

[DebugInfo] LiveDebugValues: correctly discriminate kinds of variable locations

The missing line added by this patch ensures that only spilt variable
locations are candidates for being restored from the stack. Otherwise,
register or constant-value information can be interpreted as a spill
location, through a union.

The added regression test replicates a scenario where this occurs: the
stack load from [rsp] causes the register-location DBG_VALUE to be
"restored" to rsi, when it should be left alone. See PR43058 for details.

Un x-fail a test that was suffering from this from a previous patch.

Differential Revision: https://reviews.llvm.org/D66895

llvm-svn: 370648

show more ...

[llvm] Migrate llvm::make_unique to std::make_unique

Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of

[llvm] Migrate llvm::make_unique to std::make_unique

Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.

llvm-svn: 369013

show more ...

CodeGen: Migration to using Register

llvm-svn: 367974

Finish moving TargetRegisterInfo::isVirtualRegister() and friends to llvm::Register as started by r367614. NFC

llvm-svn: 367633

Allow target to handle STRICT floating-point nodes

The ISD::STRICT_ nodes used to implement the constrained floating-point
intrinsics are currently never passed to the target back-end, which makes
i

Allow target to handle STRICT floating-point nodes

The ISD::STRICT_ nodes used to implement the constrained floating-point
intrinsics are currently never passed to the target back-end, which makes
it impossible to handle them correctly (e.g. mark instructions are depending
on a floating-point status and control register, or mark instructions as
possibly trapping).

This patch allows the target to use setOperationAction to switch the action
on ISD::STRICT_ nodes to Legal. If this is done, the SelectionDAG common code
will stop converting the STRICT nodes to regular floating-point nodes, but
instead pass the STRICT nodes to the target using normal SelectionDAG
matching rules.

To avoid having the back-end duplicate all the floating-point instruction
patterns to handle both strict and non-strict variants, we make the MI
codegen explicitly aware of the floating-point exceptions by introducing
two new concepts:

- A new MCID flag "mayRaiseFPException" that the target should set on any
instruction that possibly can raise FP exception according to the
architecture definition.
- A new MI flag FPExcept that CodeGen/SelectionDAG will set on any MI
instruction resulting from expansion of any constrained FP intrinsic.

Any MI instruction that is *both* marked as mayRaiseFPException *and*
FPExcept then needs to be considered as raising exceptions by MI-level
codegen (e.g. scheduling).

Setting those two new flags is straightforward. The mayRaiseFPException
flag is simply set via TableGen by marking all relevant instruction
patterns in the .td files.

The FPExcept flag is set in SDNodeFlags when creating the STRICT_ nodes
in the SelectionDAG, and gets inherited in the MachineSDNode nodes created
from it during instruction selection. The flag is then transfered to an
MIFlag when creating the MI from the MachineSDNode. This is handled just
like fast-math flags like no-nans are handled today.

This patch includes both common code changes required to implement the
new features, and the SystemZ implementation.

Reviewed By: andrew.w.kaylor

Differential Revision: https://reviews.llvm.org/D55506

llvm-svn: 362663

show more ...

[X86] Fix several places that weren't passing what they though they were to MachineInstr::print

Over a year ago, MachineInstr gained a fourth boolean parameter that occurs
before the TII pointer. Wh

[X86] Fix several places that weren't passing what they though they were to MachineInstr::print

Over a year ago, MachineInstr gained a fourth boolean parameter that occurs
before the TII pointer. When this happened, several places started accidentally
passing TII into this boolean parameter instead of the TII parameter.

llvm-svn: 362312

show more ...

[DebugInfoMetadata] Refactor DIExpression::prepend constants (NFC)

Refactor DIExpression::With* into a flag enum in order to be less
error-prone to use (as discussed on D60866).

Patch by Djordje To

[DebugInfoMetadata] Refactor DIExpression::prepend constants (NFC)

Refactor DIExpression::With* into a flag enum in order to be less
error-prone to use (as discussed on D60866).

Patch by Djordje Todorovic.

Differential Revision: https://reviews.llvm.org/D61943

llvm-svn: 361137

show more ...

Recommitting r358783 and r358786 "[MS] Emit S_HEAPALLOCSITE debug info" with fixes for buildbot error (undefined assembler label).

Summary:
This emits labels around heapallocsite calls and S_HEAPALL

Recommitting r358783 and r358786 "[MS] Emit S_HEAPALLOCSITE debug info" with fixes for buildbot error (undefined assembler label).

Summary:
This emits labels around heapallocsite calls and S_HEAPALLOCSITE debug
info in codeview. Currently only changes FastISel, so emitting labels still
needs to be implemented in SelectionDAG.

Reviewers: rnk

Subscribers: aprantl, hiraditya, cfe-commits, llvm-commits

Tags: #clang, #llvm

Differential Revision: https://reviews.llvm.org/D61083

llvm-svn: 359149

show more ...

[CodeGen] Add "const" to MachineInstr::mayAlias

Summary:
The basic idea here is to make it possible to use
MachineInstr::mayAlias also when the MachineInstr
is const (or the "Other" MachineInstr is

[CodeGen] Add "const" to MachineInstr::mayAlias

Summary:
The basic idea here is to make it possible to use
MachineInstr::mayAlias also when the MachineInstr
is const (or the "Other" MachineInstr is const).

The addition of const in MachineInstr::mayAlias
then rippled down to the need for adding const
in several other places, such as
TargetTransformInfo::getMemOperandWithOffset.

Reviewers: hfinkel

Reviewed By: hfinkel

Subscribers: hfinkel, MatzeB, arsenm, jvesely, nhaehnle, hiraditya, javed.absar, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60856

llvm-svn: 358744

show more ...

Allow unordered loads to be considered invariant in CodeGen

The actual code change is fairly straight forward, but exercising it isn't. First, it turned out we weren't adding the appropriate flags i

Allow unordered loads to be considered invariant in CodeGen

The actual code change is fairly straight forward, but exercising it isn't. First, it turned out we weren't adding the appropriate flags in SelectionDAG. Second, it turned out that we've got some optimization gaps, so obvious test cases don't work.

My first attempt (in atomic-unordered.ll) points out a deficiency in our peephole-opt folding logic which I plan to fix separately. Instead, I'm exercising this through MachineLICM.

Differential Revision: https://reviews.llvm.org/D59375

llvm-svn: 356494

show more ...

Allow code motion (and thus folding) for atomic (but unordered) memory operands

Building on the work done in D57601, now that we can distinguish between atomic and volatile memory accesses, go ahead

Allow code motion (and thus folding) for atomic (but unordered) memory operands

Building on the work done in D57601, now that we can distinguish between atomic and volatile memory accesses, go ahead and allow code motion of unordered atomics. As seen in the diffs, this allows much better folding of memory operations into using instructions. (Mostly done by the PeepholeOpt pass.)

Note: I have not reviewed all callers of hasOrderedMemoryRef since one of them - isSafeToMove - is very widely used. I'm relying on the documented semantics of each method to judge correctness.

Differential Revision: https://reviews.llvm.org/D59345

llvm-svn: 356170

show more ...

[NFC] add/modify wrapper function for findRegisterDefOperand().

llvm-svn: 354438

Be conservative about unordered accesses for the moment

Background: As described in https://reviews.llvm.org/D57601, I'm working towards separating volatile and atomic in the MMO uses for atomic ins

Be conservative about unordered accesses for the moment

Background: As described in https://reviews.llvm.org/D57601, I'm working towards separating volatile and atomic in the MMO uses for atomic instructions.

In https://reviews.llvm.org/D57593, I fixed a bug where isUnordered was returning the wrong result, but didn't account for the fact I was getting slightly ahead of myself. While both uses of isUnordered are correct (as far as I can tell), we don't have tests to demonstrate this and being aggressive gets in the way of having the removal of volatile truly be non-functional. Once D57601 lands, I will return to these call sites, revert this patch, and add the appropriate tests to show the expected behaviour.

Differential Revision: https://reviews.llvm.org/D57802

llvm-svn: 353766

show more ...

Move IR flag handling directly into builder calls for cases translated from Instructions in GlobalIsel

Reviewers: aditya_nandakumar, volkan

Reviewed By: aditya_nandakumar

Subscribers: rovka, krist

Move IR flag handling directly into builder calls for cases translated from Instructions in GlobalIsel

Reviewers: aditya_nandakumar, volkan

Reviewed By: aditya_nandakumar

Subscribers: rovka, kristof.beyls, volkan, Petar.Avramovic

Differential Revision: https://reviews.llvm.org/D57630

llvm-svn: 353336

show more ...

[DEBUGINFO] Reposting r352642: Handle restore instructions in LiveDebugValues

The LiveDebugValues pass recognizes spills but not restores, which can
cause large gaps in location information for some

[DEBUGINFO] Reposting r352642: Handle restore instructions in LiveDebugValues

The LiveDebugValues pass recognizes spills but not restores, which can
cause large gaps in location information for some variables, depending
on control flow. This patch make LiveDebugValues recognize restores and
generate appropriate DBG_VALUE instructions.

This patch was posted previously with r352642 and reverted in r352666 due
to buildbot errors. A missing return statement was the cause for the
failures.

Reviewers: aprantl, NicolaPrica

Differential Revision: https://reviews.llvm.org/D57271

llvm-svn: 353089

show more ...

[CodeGen] Be as conservative about atomic accesses as for volatile

Background: At the moment, we record the AtomicOrdering of an access in the MMO, but also mark any atomic access as volatile in Sel

[CodeGen] Be as conservative about atomic accesses as for volatile

Background: At the moment, we record the AtomicOrdering of an access in the MMO, but also mark any atomic access as volatile in SelectionDAG. I'm working towards separating that. See https://reviews.llvm.org/D57601 for context.

Update all usages of isVolatile in lib/CodeGen to preserve behaviour once atomic MMOs stop being also volatile. This is NFC in it's current form, but is essential for correctness once we make that final change.

It useful to keep in mind that AtomicSDNode is not a parent of LoadSDNode, StoreSDNode, or LSBaseSDNode. As a result, any call to isVolatile on one of those static types doesn't need a companion isAtomic check. We should probably adjust that class hierarchy long term, but for now, that seperation is useful.

I'm deliberately being conservative about handling. I want the change to stop adding volatile to be NFC itself, and then will work through places where we can be less conservative for atomics one by one in separate changes w/tests.

Differential Revision: https://reviews.llvm.org/D57596

llvm-svn: 352937

show more ...

Reverting r352642 - Handle restore instructions in LiveDebugValues - as it's causing
assertions on some buildbots.

llvm-svn: 352666