[SROA] Don't speculate phis with different load user types

Fixes an SROA crash.

Fallout from opaque pointers since with typed pointers we'd bail out at the bitcast.

Reviewed By: nikic

Differentia

[SROA] Don't speculate phis with different load user types

Fixes an SROA crash.

Fallout from opaque pointers since with typed pointers we'd bail out at the bitcast.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D136119

(cherry picked from commit 6219ec07c6f8d1ead51beca7cf21fbf2323c51d7)

show more ...

[SROA] Don't create constant expressions (NFC)

Use IRBuilder instead, which will fold these. Just to clarify
that this does not actually create any udiv expression.

[NFC] Use proper getSliceAlign type in SROA

[NFC] format InstructionSimplify & lowerCaseFunctionNames

Clang-format InstructionSimplify and convert all "FunctionName"s to
"functionName". This patch does touch a lot of files but gets done with

[NFC] format InstructionSimplify & lowerCaseFunctionNames

Clang-format InstructionSimplify and convert all "FunctionName"s to
"functionName". This patch does touch a lot of files but gets done with
the cleanup of InstructionSimplify in one commit.

This is the alternative to the less invasive clang-format only patch: D126783

Reviewed By: spatel, rengolin

Differential Revision: https://reviews.llvm.org/D126889

show more ...

[SROA] Avoid postponing rewriting load/store by ignoring lifetime intrinsics in partition's promotability checking

This patch fixes a bug that generates unnecessary packing/unpacking structure code

[SROA] Avoid postponing rewriting load/store by ignoring lifetime intrinsics in partition's promotability checking

This patch fixes a bug that generates unnecessary packing/unpacking structure code because of incorrectly handling lifetime intrinsic.
For example, a partition of an alloca may contain many slices:
```
Partition [0, 4):
Slice0: [0, 4) used by: load i32 addr;
Slice1: [0, 4) used by: store i32 v, addr;
Slice2: [0, 16) used by lifetime.start(16, addr);
```
When SROA determines if the partition can be promoted, lifetime.start is currently treated as a whole alloca load/store, so Slice0 and Slice1 cannot be promoted at this attempt,
but the packing/unpacking code for Slice0 and Slice1 has been generated.
After rewrite lifetime.start/end intrinsic, SROA tries again with Slice0 and Slice1 and finally promotes them, but redundant packing/unpacking code remaining in the IRs.
This patch changes promotability checking to ignore lifetime intrinsic (they will be rewritten to correct sizes later), so we can promote the real users (load/store) at the first attempt with optimal code.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D124967

show more ...

[IRBuilder] Add IsInBounds parameter to CreateGEP()

We commonly want to create either an inbounds or non-inbounds GEP
based on a boolean value, e.g. when preserving inbounds from
existing GEPs. Dire

[IRBuilder] Add IsInBounds parameter to CreateGEP()

We commonly want to create either an inbounds or non-inbounds GEP
based on a boolean value, e.g. when preserving inbounds from
existing GEPs. Directly accept such a boolean in the API, rather
than requiring a ternary between CreateGEP and CreateInBoundsGEP.

This change is not entirely NFC, because we now preserve an
inbounds flag in a constant expression edge-case in InstCombine.

show more ...

[SROA] Handle over-large loads during presplitting

When a load extends past the extent of the alloca, SROA will
restrict the slice size to extend to the end of the alloca only.
However, presplitting

[SROA] Handle over-large loads during presplitting

When a load extends past the extent of the alloca, SROA will
restrict the slice size to extend to the end of the alloca only.
However, presplitting was asserting that the load size and the
slice size match exactly, which does not hold in this case.
Relax the assertion to only require that the load size is greater
or equal than the slice size.

show more ...

Revert "Reland [SROA] Maintain shadow/backing alloca when some slices are noncapturnig read-only calls to allow alloca partitioning/promotion"

There seems to be one more uncaught problem, SROA may n

Revert "Reland [SROA] Maintain shadow/backing alloca when some slices are noncapturnig read-only calls to allow alloca partitioning/promotion"

There seems to be one more uncaught problem, SROA may now end up trying
to re-re-repromote the just-promoted shadow alloca, and do that endlessly.

This reverts commit adc0984d81f570ecc38ea23e7f556b95c7831e4c.

show more ...

Reland [SROA] Maintain shadow/backing alloca when some slices are noncapturnig read-only calls to allow alloca partitioning/promotion

This is inspired by the original variant of D109749 by Graham Hu

Reland [SROA] Maintain shadow/backing alloca when some slices are noncapturnig read-only calls to allow alloca partitioning/promotion

This is inspired by the original variant of D109749 by Graham Hunter,
but is a more general version.

Roughly, instead of promoting the alloca, we call it
a shadow/backing alloca, go through all it's slices,
clone(!) instructions that operated on it,
but make them operate on the cloned alloca,
and promote cloned alloca instead.

This keeps the shadow/backing alloca, and all the original instructions
around, which results in said shadow/backing alloca being
a perfect mirror/representation of the promoted alloca's content,
so calls that take the alloca as arguments (non-capturingly!)
can be supported.

For now, we require that the calls also don't modify the alloca's content,
but that is only to simplify the initial implementation,
and that will be supported in a follow-up.

Overall, this leads to *smaller* codesize:
https://llvm-compile-time-tracker.com/compare.php?from=a8b4f5bbab62091835205f3d648902432a4a5b58&to=aeae054055b125b011c1122f82c86457e159436f&stat=size-total
and is roughly neutral compile-time wise:
https://llvm-compile-time-tracker.com/compare.php?from=a8b4f5bbab62091835205f3d648902432a4a5b58&to=aeae054055b125b011c1122f82c86457e159436f&stat=instructions

This relands commit 703240c71fd640af7490069e8149d32d78d14da1,
that was reverted by commit 7405581f7ca3ba54be8a04c394a96fe6d980f073,
because the assertion `isa<LoadInst>(OrigInstr)` didn't hold in practice,
as the newly added test `@select_of_ptrs` shows:
If the pointers into alloca are used by select's/PHI's, then even if
we manage to fracture the alloca, some sub-alloca's will likely remain.
And if there are any non-capturing calls, then we will also decide to
keep the original backing alloca around, and we suddenly ~doubled
the alloca size, and the amount of memory traffic.
I'm not sure if this is a problem or we could live with it,
but let's leave that for later...

Reviewed By: djtodoro

Differential Revision: https://reviews.llvm.org/D113520

show more ...

Revert "[SROA] Maintain shadow/backing alloca when some slices are noncapturnig read-only calls to allow alloca partitioning/promotion"

Bots are reporting that the assertion about only expecting loa

Revert "[SROA] Maintain shadow/backing alloca when some slices are noncapturnig read-only calls to allow alloca partitioning/promotion"

Bots are reporting that the assertion about only expecting loads is wrong.

This reverts commit 703240c71fd640af7490069e8149d32d78d14da1.

show more ...

[SROA] Maintain shadow/backing alloca when some slices are noncapturnig read-only calls to allow alloca partitioning/promotion

This is inspired by the original variant of D109749 by Graham Hunter,
b

[SROA] Maintain shadow/backing alloca when some slices are noncapturnig read-only calls to allow alloca partitioning/promotion

This is inspired by the original variant of D109749 by Graham Hunter,
but is a more general version.

Roughly, instead of promoting the alloca, we call it
a shadow/backing alloca, go through all it's slices,
clone(!) instructions that operated on it,
but make them operate on the cloned alloca,
and promote cloned alloca instead.

This keeps the shadow/backing alloca, and all the original instructions
around, which results in said shadow/backing alloca being
a perfect mirror/representation of the promoted alloca's content,
so calls that take the alloca as arguments (non-capturingly!)
can be supported.

For now, we require that the calls also don't modify the alloca's content,
but that is only to simplify the initial implementation,
and that will be supported in a follow-up.

Overall, this leads to *smaller* codesize:
https://llvm-compile-time-tracker.com/compare.php?from=a8b4f5bbab62091835205f3d648902432a4a5b58&to=aeae054055b125b011c1122f82c86457e159436f&stat=size-total
and is roughly neutral compile-time wise:
https://llvm-compile-time-tracker.com/compare.php?from=a8b4f5bbab62091835205f3d648902432a4a5b58&to=aeae054055b125b011c1122f82c86457e159436f&stat=instructions

Reviewed By: djtodoro

Differential Revision: https://reviews.llvm.org/D113520

show more ...

Cleanup includes: Transform/Scalar

Estimated impact on preprocessor output line:
before: 1062981579
after: 1062494547

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cl

Cleanup includes: Transform/Scalar

Estimated impact on preprocessor output line:
before: 1062981579
after: 1062494547

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D120817

show more ...

Simplify mask creation with llvm::seq. NFCI.

Cleanup header dependencies in LLVMCore

Based on the output of include-what-you-use.

This is a big chunk of changes. It is very likely to break downstream code
unless they took a lot of care in avo

Cleanup header dependencies in LLVMCore

Based on the output of include-what-you-use.

This is a big chunk of changes. It is very likely to break downstream code
unless they took a lot of care in avoiding hidden ehader dependencies, something
the LLVM codebase doesn't do that well :-/

I've tried to summarize the biggest change below:

- llvm/include/llvm-c/Core.h: no longer includes llvm-c/ErrorHandling.h
- llvm/IR/DIBuilder.h no longer includes llvm/IR/DebugInfo.h
- llvm/IR/IRBuilder.h no longer includes llvm/IR/IntrinsicInst.h
- llvm/IR/LLVMRemarkStreamer.h no longer includes llvm/Support/ToolOutputFile.h
- llvm/IR/LegacyPassManager.h no longer include llvm/Pass.h
- llvm/IR/Type.h no longer includes llvm/ADT/SmallPtrSet.h
- llvm/IR/PassManager.h no longer includes llvm/Pass.h nor llvm/Support/Debug.h

And the usual count of preprocessed lines:
$ clang++ -E -Iinclude -I../llvm/include ../llvm/lib/IR/*.cpp -std=c++14 -fno-rtti -fno-exceptions | wc -l
before: 6400831
after: 6189948

200k lines less to process is no that bad ;-)

Discourse thread on the topic: https://llvm.discourse.group/t/include-what-you-use-include-cleanup

Differential Revision: https://reviews.llvm.org/D118652

show more ...

[NFC] Remove uses of PointerType::getElementType()

Instead use either Type::getPointerElementType() or
Type::getNonOpaquePointerElementType().

This is part of D117885, in preparation for deprecatin

[NFC] Remove uses of PointerType::getElementType()

Instead use either Type::getPointerElementType() or
Type::getNonOpaquePointerElementType().

This is part of D117885, in preparation for deprecating the API.

show more ...

[OpaquePtrs] Add getNonOpaquePointerElementType() method (NFC)

This method is intended for use in places that cannot be reached
with opaque pointers, or part of deprecated methods. This makes
it eas

[OpaquePtrs] Add getNonOpaquePointerElementType() method (NFC)

This method is intended for use in places that cannot be reached
with opaque pointers, or part of deprecated methods. This makes
it easier to see that some uses of getPointerElementType() don't
need further action.

Differential Revision: https://reviews.llvm.org/D117870

show more ...

[SROA] Bail out on PHIs in catchswitch BBs

In the process of rewriting `alloca`s and `phi`s that use them, the SROA
pass can try to insert a non-PHI instruction by calling
`getFirstInsertionPt()`, w

[SROA] Bail out on PHIs in catchswitch BBs

In the process of rewriting `alloca`s and `phi`s that use them, the SROA
pass can try to insert a non-PHI instruction by calling
`getFirstInsertionPt()`, which is not possible in a catchswitch BB. This
CL makes we bail out on these cases.

Reviewed By: dschuff

Differential Revision: https://reviews.llvm.org/D117168

show more ...

[SROA] Switch replacement of dead/UB/unreachable ops from undef to poison

SROA has 3 data-structures where it stores sets of instructions that should
be deleted:
- DeadUsers -> instructions that ar

[SROA] Switch replacement of dead/UB/unreachable ops from undef to poison

SROA has 3 data-structures where it stores sets of instructions that should
be deleted:
- DeadUsers -> instructions that are UB or have no users
- DeadOperands -> instructions that are UB or operands of useless phis
- DeadInsts -> "dead" instructions, including loads of uninitialized memory
with users

The first 2 sets can be RAUW with poison instead of undef. No brainer as UB
can be replaced with poison, and for instructions with no users RAUW is a
NOP.

The 3rd case cannot be currently replaced with poison because the set mixes
the loads of uninit memory. I leave that alone for now.

Another case where we can use poison is in the construction of vectors from
multiple loads. The base vector for the first insertelement is now poison as
it doesn't matter as it is fully overwritten by inserts.

Differential Revision: https://reviews.llvm.org/D116887

show more ...

[SROA] Reduce the number of times a IRBuilder is constructed (NFC).

This patch reduces the number of times IRBuilders need to be constructed
in SROA.cpp by passing existing ones by reference to the

[SROA] Reduce the number of times a IRBuilder is constructed (NFC).

This patch reduces the number of times IRBuilders need to be constructed
in SROA.cpp by passing existing ones by reference to the appropriate
places.

show more ...

[NFC] Rename GVN -> GVNPass and SROA -> SROAPass

To be more consistent with other pass struct names.

There are still more passes that don't end with "Pass", but these are the important ones.

Revie

[NFC] Rename GVN -> GVNPass and SROA -> SROAPass

To be more consistent with other pass struct names.

There are still more passes that don't end with "Pass", but these are the important ones.

Reviewed By: asbirlea

Differential Revision: https://reviews.llvm.org/D112935

show more ...

[IR] Add helper to convert offset to GEP indices

We implement logic to convert a byte offset into a sequence of GEP
indices for that offset in a number of places. This patch adds a
DataLayout::getGE

[IR] Add helper to convert offset to GEP indices

We implement logic to convert a byte offset into a sequence of GEP
indices for that offset in a number of places. This patch adds a
DataLayout::getGEPIndicesForOffset() method, which implements the
core logic. I've updated SROA, ConstantFolding and InstCombine to
use it, and there's a few more places where it looks relevant.

Differential Revision: https://reviews.llvm.org/D110043

show more ...

[IR] Return AAMDNodes from Instruction::getMetadata() (NFC)

getMetadata() currently uses a weird API where it populates a
structure passed to it, and optionally merges into it. Instead,
we can retur

[IR] Return AAMDNodes from Instruction::getMetadata() (NFC)

getMetadata() currently uses a weird API where it populates a
structure passed to it, and optionally merges into it. Instead,
we can return the AAMDNodes and provide a separate merge() API.
This makes usages more compact.

Differential Revision: https://reviews.llvm.org/D109852

show more ...

[SROA] Support opaque pointers

Make the following changes in order to support opaque pointers in SROA:

* Generate i8 GEPs for opaque pointers.
* Explicitly enforce that promotable allocas only ha

[SROA] Support opaque pointers

Make the following changes in order to support opaque pointers in SROA:

* Generate i8 GEPs for opaque pointers.
* Explicitly enforce that promotable allocas only have stores of
the alloca type -- previously this was implicitly enforced.
* Replace a check for pointer element type with load/store type.

Differential Revision: https://reviews.llvm.org/D109259

show more ...

Reapply "SROA: Enhance speculateSelectInstLoads"

Originally committed as ffc3fb665d0a0dccd64cc8c803ad8cc1a0d5dfa1
Reverted in fcf2d5f40296be4e0f0e954001beb7814f97a212 due to an
assertion failure.

O

Reapply "SROA: Enhance speculateSelectInstLoads"

Originally committed as ffc3fb665d0a0dccd64cc8c803ad8cc1a0d5dfa1
Reverted in fcf2d5f40296be4e0f0e954001beb7814f97a212 due to an
assertion failure.

Original commit message:

Allow the folding even if there is an
intervening bitcast.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D106667

show more ...

Revert "SROA: Enhance speculateSelectInstLoads"

This reverts commit ffc3fb665d0a0dccd64cc8c803ad8cc1a0d5dfa1.