[NFC][LoopVectorize] Explicitly disable tail-folding on some SVE tests

This patch is in preparation for enabling vectorisation with tail-folding
by default for SVE targets. Once we do that many exis

[NFC][LoopVectorize] Explicitly disable tail-folding on some SVE tests

This patch is in preparation for enabling vectorisation with tail-folding
by default for SVE targets. Once we do that many existing tests will
break that depend upon having normal unpredicated vector loops. For
all such tests I have added the flag:

-prefer-predicate-over-epilogue=scalar-epilogue

Differential Revision: https://reviews.llvm.org/D129137

show more ...

Revert "[NFCI] Regenerate SROA/LoopVectorize test checks"

This reverts commit 14e3450fb57305aa9ff3e9e60687b458e43835c9.

[NFCI] Regenerate LoopVectorize test checks

[LV] Enable scalable vectorization by default for SVE cores.

The availability of SVE should be sufficient to enable scalable
auto-vectorization.

This patch adds a new TTI interface to query the tar

[LV] Enable scalable vectorization by default for SVE cores.

The availability of SVE should be sufficient to enable scalable
auto-vectorization.

This patch adds a new TTI interface to query the target what style of
vectorization it wants when scalable vectors are available. For other
targets than AArch64, this currently defaults to 'FixedWidthOnly'.

Differential Revision: https://reviews.llvm.org/D115651

show more ...

[IR] In ConstantFoldShuffleVectorInstruction use zeroinitializer for splats of 0

When creating a splat of 0 for scalable vectors we tend to create them
with using a combination of shufflevector and

[IR] In ConstantFoldShuffleVectorInstruction use zeroinitializer for splats of 0

When creating a splat of 0 for scalable vectors we tend to create them
with using a combination of shufflevector and insertelement, i.e.

shufflevector (<vscale x 4 x i32> insertelement (<vscale x 4 x i32> poison, i32 0, i32 0),
<vscale x 4 x i32> poison, <vscale x 4 x i32> zeroinitializer)

However, for the case of a zero splat we can actually just replace the
above with zeroinitializer instead. This makes the IR a lot simpler and
easier to read. I have changed ConstantFoldShuffleVectorInstruction to
use zeroinitializer when creating a splat of integer 0 or FP +0.0 values.

Differential Revision: https://reviews.llvm.org/D113394

show more ...

[LV] Add -scalable-vectorization=<option> flag.

This patch adds a new option to the LoopVectorizer to control how
scalable vectors can be used.

Initially, this suggests three levels to control scal

[LV] Add -scalable-vectorization=<option> flag.

This patch adds a new option to the LoopVectorizer to control how
scalable vectors can be used.

Initially, this suggests three levels to control scalable
vectorization, although other more aggressive options can be added in
the future.

The possible options are:
- Disabled: Disables vectorization with scalable vectors.
- Enabled: Vectorize loops using scalable vectors or fixed-width
vectors, but favors fixed-width vectors when the cost
is a tie.
- Preferred: Like 'Enabled', but favoring scalable vectors when the
cost-model is inconclusive.

Reviewed By: paulwalker-arm, vkmr

Differential Revision: https://reviews.llvm.org/D101945

show more ...

[LoopVectorize][SVE] Fix crash when vectorising FP negation

This patch fixes a crash encountered when vectorising the following loop:

void foo(float *dst, float *src, long long n) {
for (long l

[LoopVectorize][SVE] Fix crash when vectorising FP negation

This patch fixes a crash encountered when vectorising the following loop:

void foo(float *dst, float *src, long long n) {
for (long long i = 0; i < n; i++)
dst[i] = -src[i];
}

using scalable vectors. I've added a test to

Transforms/LoopVectorize/AArch64/sve-basic-vec.ll

as well as cleaned up the other tests in the same file.

Differential Revision: https://reviews.llvm.org/D98054

show more ...

[SVE] Remove checks for warnings in scalable-vector tests.

After D98856 these tests will by default break (fatal_error) if any of
the wrong interfaces are used, so there's no longer a need to have a

[SVE] Remove checks for warnings in scalable-vector tests.

After D98856 these tests will by default break (fatal_error) if any of
the wrong interfaces are used, so there's no longer a need to have a
RUN line that checks for a warning message emitted by the compiler.

show more ...

[SVE] Add support for scalable vectorization of loops with selects and cmps

I have removed an unnecessary assert in LoopVectorizationCostModel::getInstructionCost
that prevented a cost being calcula

[SVE] Add support for scalable vectorization of loops with selects and cmps

I have removed an unnecessary assert in LoopVectorizationCostModel::getInstructionCost
that prevented a cost being calculated for select instructions when using
scalable vectors. In addition, I have changed AArch64TTIImpl::getCmpSelInstrCost
to only do special cost calculations for fixed width vectors and fall
back to the base version for scalable vectors.

I have added a simple cost model test for cmps and selects:

test/Analysis/CostModel/sve-cmpsel.ll

and some simple tests that show we vectorize loops with cmp and select:

test/Transforms/LoopVectorize/AArch64/sve-basic-vec.ll

Differential Revision: https://reviews.llvm.org/D95039

show more ...