[llvm] Use value instead of getValue (NFC)

Revert "Don't use Optional::hasValue (NFC)"

This reverts commit aa8feeefd3ac6c78ee8f67bf033976fc7d68bc6d.

Don't use Optional::hasValue (NFC)

[ARM] Fix MVE gather/scatter merged gep offsets

This fixes the combining of constant vector GEP operands in the
optimization of MVE gather/scatter addresses, when opaque pointers are
enabled. As opa

[ARM] Fix MVE gather/scatter merged gep offsets

This fixes the combining of constant vector GEP operands in the
optimization of MVE gather/scatter addresses, when opaque pointers are
enabled. As opaque pointers reduce the number of bitcasts between geps,
more can be folded than before. This can cause problems if the index
types are now different between the two geps.

This fixes that by making sure each constant is scaled appropriately,
which has the effect of transforming the geps to have a scale of 1,
changing [r0, q0, uxtw #1] gathers to [r0, q0] with a larger q0. This
helps use a simpler instruction that doesn't need the extra uxtw.

Differential Revision: https://reviews.llvm.org/D127733

show more ...

[llvm] Don't use Optional::getValue (NFC)

[ARM] Add more opaque pointer gather/scatter tests. NFC

Some of the newly added tests are incorrect, fixed in D127733.

[ARM] Fix GatherScatter AddLikeOr condition

[llvm] Use isa instead of dyn_cast (NFC)

[ARM] Treat MVE gather add-like-or's like adds

LLVM has the habit of turning adds with no common bits set into ors,
which means we need to detect them and treat them like adds again in the
MVE gathe

[ARM] Treat MVE gather add-like-or's like adds

LLVM has the habit of turning adds with no common bits set into ors,
which means we need to detect them and treat them like adds again in the
MVE gather/scatter lowering pass.

Differential Revision: https://reviews.llvm.org/D112922

show more ...

[ARM] Push gather/scatter shl index updates out of loops

This teaches the MVE gather scatter lowering pass that SHL is
essentially the same as Mul, where we are able to optimize the
induction of a g

[ARM] Push gather/scatter shl index updates out of loops

This teaches the MVE gather scatter lowering pass that SHL is
essentially the same as Mul, where we are able to optimize the
induction of a gather/scatter address by pushing them out of loops.
https://alive2.llvm.org/ce/z/wG4VyT

Differential Revision: https://reviews.llvm.org/D112920

show more ...

[ARM] Lower non-extended small gathers via truncated gathers.

Corollary to 1113e06821e6baffc84b8caf96a28bf62e6d28dc this allows us to
match gather that dont produce a full vector width results. They

[ARM] Lower non-extended small gathers via truncated gathers.

Corollary to 1113e06821e6baffc84b8caf96a28bf62e6d28dc this allows us to
match gather that dont produce a full vector width results. They use an
extended gather which is truncated back to the original type.

show more ...

[ARM] Pass 2 instead of 0 to PHINode::Create in MVEGatherScatterLowering. NFC

This parameter controls how much space is reserved for incoming
values. There are always going to be 2 incoming values i

[ARM] Pass 2 instead of 0 to PHINode::Create in MVEGatherScatterLowering. NFC

This parameter controls how much space is reserved for incoming
values. There are always going to be 2 incoming values in this case.

While there remove the unused std::vector right below.

Found while looking at porting this code to RISCV.

show more ...

[ARM] Use matchSimpleRecurrence to simplify some code in MVEGatherScatterLowering. NFCI

Reviewed By: dmgreen

Differential Revision: https://reviews.llvm.org/D105262

[ARM] Fix incorrect assignment of Changed variable in MVEGatherScatterLowering::optimiseOffsets.

I believe this Changed flag should be initialized to false,
otherwise the if (!Changed) is always dea

[ARM] Fix incorrect assignment of Changed variable in MVEGatherScatterLowering::optimiseOffsets.

I believe this Changed flag should be initialized to false,
otherwise the if (!Changed) is always dead. This doesn't
manifest in a functional issue because the PHINode checks will
fail if nothing changed. They are identical to the earlier
checks that must have already failed to get into this else block.

While there remove an else after return to reduce indentation.

Differential Revision: https://reviews.llvm.org/D105159

show more ...

[ARM] Extend narrow values to allow using truncating scatters

As a minor adjustment to the existing lowering of offset scatters, this
extends any smaller-than-legal vectors into full vectors using a

[ARM] Extend narrow values to allow using truncating scatters

As a minor adjustment to the existing lowering of offset scatters, this
extends any smaller-than-legal vectors into full vectors using a zext,
so that the truncating scatters can be used. Due to the way MVE
legalizes the vectors this should be cheap in most situations, and will
prevent the vector from being scalarized.

Differential Revision: https://reviews.llvm.org/D103704

show more ...

[ARM] Change some Gather/Scatter interface types to Instructions. NFC

These returned Values are cast to an Instruction already, this just
cleans up the interface a little to match the expected types.

Revert "[ARM] Extend narrow values to allow using truncating scatters"

This commit adds nodes that might not always be used, which the
expensive checks builder does not like. Reverting for now to th

Revert "[ARM] Extend narrow values to allow using truncating scatters"

This commit adds nodes that might not always be used, which the
expensive checks builder does not like. Reverting for now to think up a
better way of handling it.

show more ...

[ARM] Extend narrow values to allow using truncating scatters

As a minor adjustment to the existing lowering of offset scatters, this
extends any smaller-than-legal vectors into full vectors using a

[ARM] Extend narrow values to allow using truncating scatters

As a minor adjustment to the existing lowering of offset scatters, this
extends any smaller-than-legal vectors into full vectors using a zext,
so that the truncating scatters can be used. Due to the way MVE
legalizes the vectors this should be cheap in most situations, and will
prevent the vector from being scalarized.

Differential Revision: https://reviews.llvm.org/D103704

show more ...

[ARM] Use rq gather/scatters for smaller v4 vectors

A pointer will always fit into an i32, so a rq offset gather/scatter can
be used with v4i8 and v4i16 gathers, using a base of 0 and the Ptr as
the

[ARM] Use rq gather/scatters for smaller v4 vectors

A pointer will always fit into an i32, so a rq offset gather/scatter can
be used with v4i8 and v4i16 gathers, using a base of 0 and the Ptr as
the offsets. The rq gather can then correctly extend the type, allowing
us to use the gathers without falling back to scalarizing.

This patch rejigs tryCreateMaskedGatherOffset in the
MVEGatherScatterLowering pass to decompose the Ptr into Base:0 +
Offset:Ptr (with a scale of 1), if the Ptr could not be decomposed from
a GEP. v4i32 gathers will already use qi gathers, this extends that to
v4i8 and v4i16 gathers using the extending rq variants.

Differential Revision: https://reviews.llvm.org/D103674

show more ...

[ARM] Rejig some of the MVE gather/scatter lowering pass. NFC

This adjusts some of how the gather/scatter lowering pass passes around
data and where certain gathers/scatters are created from. It sho

[ARM] Rejig some of the MVE gather/scatter lowering pass. NFC

This adjusts some of how the gather/scatter lowering pass passes around
data and where certain gathers/scatters are created from. It should not
effect code generation on its own, but allows other patches to more
clearly reason about the code.

A number of extra test cases were also added for smaller gathers/
scatters that can be extended, and some of the test comments were
updated.

show more ...

[ARM] Fix Changed status in MVEGatherScatterLoweringPass.

Now that we are calling SimplifyInstructionsInBlock, make sure we update
Changed when it reports alterations.

[ARM] Ensure instructions are simplified prior to GatherScatter lowering.

Surprisingly, not all instructions are always simplified after unrolling
and before MVE gather/scatter lowering. Notably dea

[ARM] Ensure instructions are simplified prior to GatherScatter lowering.

Surprisingly, not all instructions are always simplified after unrolling
and before MVE gather/scatter lowering. Notably dead gather operations
can be left around which cause the gather/scatter lowering pass to crash
if there are multiple gathers, some of which are dead.

This patch ensures they are simplified before we modify anything, which
can change some of the existing tests, including making them no-longer
test what they originally tested. This uses a combination of disabling
the gather/scatter lowering pass and adjusting the test to keep them as
before.

Differential Revision: https://reviews.llvm.org/D103150

show more ...

[ARM] Prettify gather/scatter debug comments. NFC

[OpaquePtr] Clean up some uses of Type::getPointerElementType()

These depend on pointee types.

[ARM] Guard against loop variant gather ptr operands

This ensures that the operands of any gather/scatter instructions that
we attempt to push out of the loop are invariant, preventing invalid IR
fr

[ARM] Guard against loop variant gather ptr operands

This ensures that the operands of any gather/scatter instructions that
we attempt to push out of the loop are invariant, preventing invalid IR
from being generated.

show more ...