[APInt] Remove all uses of zextOrSelf, sextOrSelf and truncOrSelf

Most clients only used these methods because they wanted to be able to
extend or truncate to the same bit width (which is a no-op).

[APInt] Remove all uses of zextOrSelf, sextOrSelf and truncOrSelf

Most clients only used these methods because they wanted to be able to
extend or truncate to the same bit width (which is a no-op). Now that
the standard zext, sext and trunc allow this, there is no reason to use
the OrSelf versions.

The OrSelf versions additionally have the strange behaviour of allowing
extending to a *smaller* width, or truncating to a *larger* width, which
are also treated as no-ops. A small amount of client code relied on this
(ConstantRange::castOp and MicrosoftCXXNameMangler::mangleNumber) and
needed rewriting.

Differential Revision: https://reviews.llvm.org/D125557

show more ...

[InstCombine] Remove memset of undef value

This removes memset with undef char. We already do this for stores
of undef value.

This comes with the caveat that this optimization is not, strictly
spea

[InstCombine] Remove memset of undef value

This removes memset with undef char. We already do this for stores
of undef value.

This comes with the caveat that this optimization is not, strictly
speaking, legal for undef values, because we might be overwriting
a poison value. However, our entire load/store model currently still
operates on undef values, so we need to support undef here as well
for internal consistency.

Once https://github.com/llvm/llvm-project/issues/52930 is resolved,
these and related folds can be limited to poison -- I've added
FIXMEs to that effect.

Differential Revision: https://reviews.llvm.org/D124173

show more ...

Cleanup includes: Transform/Scalar

Estimated impact on preprocessor output line:
before: 1062981579
after: 1062494547

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cl

Cleanup includes: Transform/Scalar

Estimated impact on preprocessor output line:
before: 1062981579
after: 1062494547

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D120817

show more ...

[InstCombine][OpaquePtr] Check store type in DSE implementation

[InstCombine] Avoid pointer element type access in PointerReplacer

This code replaces the address space of the pointers while keeping
the element type. Use the appropriate helpers to make this work

[InstCombine] Avoid pointer element type access in PointerReplacer

This code replaces the address space of the pointers while keeping
the element type. Use the appropriate helpers to make this work
with opaque pointers.

show more ...

[NFC] Remove uses of PointerType::getElementType()

Instead use either Type::getPointerElementType() or
Type::getNonOpaquePointerElementType().

This is part of D117885, in preparation for deprecatin

[NFC] Remove uses of PointerType::getElementType()

Instead use either Type::getPointerElementType() or
Type::getNonOpaquePointerElementType().

This is part of D117885, in preparation for deprecating the API.

show more ...

InstCombine: Gracefully handle more allocas in the wrong address space

Officially this is currently required to always use the datalayout's
alloca address space. This may change in the future, and i

InstCombine: Gracefully handle more allocas in the wrong address space

Officially this is currently required to always use the datalayout's
alloca address space. This may change in the future, and it's cleaner
to propagate the existing alloca's addrspace anyway.

This is a triple fix. Initially the change in simplifyAllocaArraySize
would drop the address space, but produce output. Fixing this hit an
assertion in the cast combine.

This patch also makes the changes to handle this situation from
a33e12801279a947c74fdee2655b24480941fb39 dead, so eliminate
it. InstCombine should not take it upon itself to introduce
addrspacecasts, and preserve the original address space instead.

show more ...

[NFC] Remove more calls to getAlignment()

These are deprecated and should be replaced with getAlign().

Some of these asserts don't do anything because Load/Store/AllocaInst never have a 0 align val

[NFC] Remove more calls to getAlignment()

These are deprecated and should be replaced with getAlign().

Some of these asserts don't do anything because Load/Store/AllocaInst never have a 0 align value.

show more ...

[NFC] Replace some deprecated getAlignment() calls with getAlign()

Reviewed By: gchatelet

Differential Revision: https://reviews.llvm.org/D115370

[CSSPGO] Unblock optimizations with pseudo probe instrumentation part 3.

This patch continues unblocking optimizations that are blocked by pseudo probe instrumentation.

Not exactly like DbgIntrinsi

[CSSPGO] Unblock optimizations with pseudo probe instrumentation part 3.

This patch continues unblocking optimizations that are blocked by pseudo probe instrumentation.

Not exactly like DbgIntrinsics, PseudoProbe intrinsic has other attributes (such as mayread, maywrite, mayhaveSideEffect) that can block optimizations. The issues fixed are:
- Flipped default param of getFirstNonPHIOrDbg API to skip pseudo probes
- Unblocked CSE by avoiding pseudo probe from clobbering memory SSA
- Unblocked induction variable simpliciation
- Allow empty loop deletion by treating probe intrinsic isDroppable
- Some refactoring.

Reviewed By: wenlei

Differential Revision: https://reviews.llvm.org/D110847

show more ...

[IR] Return AAMDNodes from Instruction::getMetadata() (NFC)

getMetadata() currently uses a weird API where it populates a
structure passed to it, and optionally merges into it. Instead,
we can retur

[IR] Return AAMDNodes from Instruction::getMetadata() (NFC)

getMetadata() currently uses a weird API where it populates a
structure passed to it, and optionally merges into it. Instead,
we can return the AAMDNodes and provide a separate merge() API.
This makes usages more compact.

Differential Revision: https://reviews.llvm.org/D109852

show more ...

Fixing an infinite loop problem in InstCombine

Patch by Mohammad Fawaz

This issues started happening after
https://github.com/llvm/llvm-project/commit/b373b5990d5991a920c421b21a352e4ccf4c4993
Basic

Fixing an infinite loop problem in InstCombine

Patch by Mohammad Fawaz

This issues started happening after
https://github.com/llvm/llvm-project/commit/b373b5990d5991a920c421b21a352e4ccf4c4993
Basically, if the memcpy is volatile, the collectUsers() function should
return false, just like we do for volatile loads.

Differential Revision: https://reviews.llvm.org/D106950

show more ...

Enabling the copy-constant-to-alloca optimization in more instances

Patch by Mohammad Fawaz

This patch allows lifetime calls to be ignored (and later erased) if we
know that the copy-constant-to-al

Enabling the copy-constant-to-alloca optimization in more instances

Patch by Mohammad Fawaz

This patch allows lifetime calls to be ignored (and later erased) if we
know that the copy-constant-to-alloca optimization is going to happen.
The case that is missed is when the global variable is in a different address
space than the alloca (as shown in the example added to the lit test.)

This used to work before https://github.com/llvm/llvm-project/commit/6da31fa4a61d68af21dfa1e144e726ed6d77903e

Differential Revision: https://reviews.llvm.org/D106573

show more ...

[InstCombine] Gracefully handle an alloca outside the alloca-AS

While we might eventually want to disallow allocas that do not have the
alloca-AS set, it seems undesirable to crash on them. Add a ca

[InstCombine] Gracefully handle an alloca outside the alloca-AS

While we might eventually want to disallow allocas that do not have the
alloca-AS set, it seems undesirable to crash on them. Add a cast when
required so that we can support such allocas (at least here).

Differential Revision: https://reviews.llvm.org/D104866

show more ...

[OpaquePtr] Support changing load type in InstCombine

When the load type is changed to ptr, we need the load pointer type
to also be ptr, because it's not allowed to create a pointer to an
opaque po

[OpaquePtr] Support changing load type in InstCombine

When the load type is changed to ptr, we need the load pointer type
to also be ptr, because it's not allowed to create a pointer to an
opaque pointer. This is achieved by adjusting the getPointerTo() API
to return an opaque pointer for an opaque pointer base type.

Differential Revision: https://reviews.llvm.org/D104718

show more ...

[InstCombine] Use poison constant to represent the result of unreachable instrs

This patch updates InstCombine to use poison constant to represent the resulting value of (either semantically or synt

[InstCombine] Use poison constant to represent the result of unreachable instrs

This patch updates InstCombine to use poison constant to represent the resulting value of (either semantically or syntactically) unreachable instrs, or a don't-care value of an unreachable store instruction.

This allows more aggressive folding of unused results, as shown in llvm/test/Transforms/InstCombine/getelementptr.ll .

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D104602

show more ...

[CSSPGO] Unblock optimizations with pseudo probe instrumentation part 2.

As a follow-up to D95982, this patch continues unblocking optimizations that are blocked by pseudu probe instrumention.

The

[CSSPGO] Unblock optimizations with pseudo probe instrumentation part 2.

As a follow-up to D95982, this patch continues unblocking optimizations that are blocked by pseudu probe instrumention.

The optimizations unblocked are:
- In-block load propagation.
- In-block dead store elimination
- Memory copy optimization that turns stores to consecutive memories into a memset.

These optimizations are local to a block, so they shouldn't affect the profile quality.

Reviewed By: wmi

Differential Revision: https://reviews.llvm.org/D100075

show more ...

Update InstCombine to use undef matcher instead

This is a patch to use m_Undef() matcher instead of isa<UndefValue>().

As suggested in D100122, this update is separately committed.

[Instcombine] Disable memcpy of alloca bypass for instruction sources

This transformation is fundamentally broken when it comes to dominance,
it just happened to work when the source of the memcpy c

[Instcombine] Disable memcpy of alloca bypass for instruction sources

This transformation is fundamentally broken when it comes to dominance,
it just happened to work when the source of the memcpy can be moved into
the place of the alloca. The bug shows up a lot more often since
077bff39d46364035a5dcfa32fc69910ad0975d0 allows the source to be a
switch.

It would be possible to check dominance of the source and all its
operands, but that seems very heavy for instcombine.

show more ...

[InstCombine] tmp alloca bypass: ensure that the replacement dominates all alloca uses

After 077bff39d46364035a5dcfa32fc69910ad0975d0,
isDereferenceableForAllocaSize() can recurse into selects,
whic

[InstCombine] tmp alloca bypass: ensure that the replacement dominates all alloca uses

After 077bff39d46364035a5dcfa32fc69910ad0975d0,
isDereferenceableForAllocaSize() can recurse into selects,
which is causing a problem for the new test case,
reduced from https://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20210412/904154.html
because the replacement (the select) is defined after the first use
of an alloca, so we'd end up with a verifier error.

Now, this new check is too restrictive.
We likely can handle *some* cases, by trying to sink all uses of an alloca
to after the the def.

show more ...

[Loads] Add optimized FindAvailableLoadedValue() overload (NFCI)

FindAvailableLoadedValue() accepts an iterator by reference. If no
available value is found, then the iterator will either be left
at

[Loads] Add optimized FindAvailableLoadedValue() overload (NFCI)

FindAvailableLoadedValue() accepts an iterator by reference. If no
available value is found, then the iterator will either be left
at a clobbering instruction or the beginning of the basic block.
This allows using FindAvailableLoadedValue() across multiple blocks.

If this functionality is not needed, as is the case in InstCombine,
then we can use a much more efficient implementation: First try
to find an available value, and only perform clobber checks if
we actually found one. As this function only looks at a very small
number of instructions (6 by default) and usually doesn't find an
available value, this saves many expensive alias analysis queries.

show more ...

[Transforms] Use llvm::append_range (NFC)

[X86][AMX] Prohibit pointer cast on load.

The load/store instruction will be transformed to amx intrinsics in the
pass of AMX type lowering. Prohibiting the pointer cast make that pass
happy.

Diffe

[X86][AMX] Prohibit pointer cast on load.

The load/store instruction will be transformed to amx intrinsics in the
pass of AMX type lowering. Prohibiting the pointer cast make that pass
happy.

Differential Revision: https://reviews.llvm.org/D94372

show more ...

[X86] Add x86_amx type for intel AMX.

The x86_amx is used for AMX intrisics. <256 x i32> is bitcast to x86_amx when
it is used by AMX intrinsics, and x86_amx is bitcast to <256 x i32> when it
is use

[X86] Add x86_amx type for intel AMX.

The x86_amx is used for AMX intrisics. <256 x i32> is bitcast to x86_amx when
it is used by AMX intrinsics, and x86_amx is bitcast to <256 x i32> when it
is used by load/store instruction. So amx intrinsics only operate on type x86_amx.
It can help to separate amx intrinsics from llvm IR instructions (+-*/).
Thank Craig for the idea. This patch depend on https://reviews.llvm.org/D87981.

Differential Revision: https://reviews.llvm.org/D91927

show more ...

[CodeGen, Transforms] Use *Map::lookup (NFC)