[RISCV] Disable subregister liveness by default

We previously enabled subregister liveness by default when compiling
with RVV. This has been shown to cause miscompilations where RVV
register operand

[RISCV] Disable subregister liveness by default

We previously enabled subregister liveness by default when compiling
with RVV. This has been shown to cause miscompilations where RVV
register operand constraints are not met. A test was added for this in
D129639 which explains the issue in more detail.

Until this issue is fixed in some way, we should not be enabling
subregister liveness unless the user asks for it.

Reviewed By: craig.topper, rogfer01, kito-cheng

Differential Revision: https://reviews.llvm.org/D129646

show more ...

[IR] Move vector.insert/vector.extract out of experimental namespace

These intrinsics are now fundemental for SVE code generation and have been
present for a year and a half, hence move them out of

[IR] Move vector.insert/vector.extract out of experimental namespace

These intrinsics are now fundemental for SVE code generation and have been
present for a year and a half, hence move them out of the experimental
namespace.

Differential Revision: https://reviews.llvm.org/D127976

show more ...

Recommit "[RISCV] Enable subregister liveness tracking for RVV."

The failure that caused the previous revert has been fixed
by https://reviews.llvm.org/D126048

Original commit message:

RVV makes h

Recommit "[RISCV] Enable subregister liveness tracking for RVV."

The failure that caused the previous revert has been fixed
by https://reviews.llvm.org/D126048

Original commit message:

RVV makes heavy use of subregisters due to LMUL>1 and segment
load/store tuples. Enabling subregister liveness tracking improves the quality
of the register allocation.

I've added a command line that can be used to turn it off if it causes compile
time or functional issues. I used the command line to keep the old behavior
for one interesting test case that was testing register allocation.

Reviewed By: kito-cheng

Differential Revision: https://reviews.llvm.org/D128016

show more ...

[DAG] Fold (shl (srl x, c), c) -> and(x, m) even if srl has other uses

If we're using shift pairs to mask, then relax the one use limit if the shift amounts are equal - we'll only be generating a si

[DAG] Fold (shl (srl x, c), c) -> and(x, m) even if srl has other uses

If we're using shift pairs to mask, then relax the one use limit if the shift amounts are equal - we'll only be generating a single AND node.

AArch64 has a couple of regressions due to this, so I've enforced the existing one use limit inside a AArch64TargetLowering::shouldFoldConstantShiftPairToMask callback.

Part of the work to fix the regressions in D77804

Differential Revision: https://reviews.llvm.org/D125607

show more ...

[RISCV] Fix incorrect use of tail agnostic vslideup.

We need to use tail undisturbed for vslideup to implement
vector insert operation correctly.

Ideally, we cound use the tail agnostic when insert

[RISCV] Fix incorrect use of tail agnostic vslideup.

We need to use tail undisturbed for vslideup to implement
vector insert operation correctly.

Ideally, we cound use the tail agnostic when insert subvector
or element at the end of the vector. This will be in follow-up
patch.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D125545

show more ...

Revert "[RISCV] Enable subregister liveness tracking for RVV."

This reverts most of ed242b54c9c2aa84a47f66af5b8497d93646b68d

I'm seeing failures in our intrinsic testing on qemu that seem
related t

Revert "[RISCV] Enable subregister liveness tracking for RVV."

This reverts most of ed242b54c9c2aa84a47f66af5b8497d93646b68d

I'm seeing failures in our intrinsic testing on qemu that seem
related to this. Reverting while I investigate.

I've left the command line option in place for directed testing.
It defaults to off.

show more ...

[RISCV] Enable subregister liveness tracking for RVV.

RVV makes heavy use of subregisters due to LMUL>1 and segment
load/store tuples. Enabling subregister liveness tracking improves the quality
of

[RISCV] Enable subregister liveness tracking for RVV.

RVV makes heavy use of subregisters due to LMUL>1 and segment
load/store tuples. Enabling subregister liveness tracking improves the quality
of the register allocation.

I've added a command line that can be used to turn it off if it causes compile
time or functional issues. I used the command line to keep the old behavior
for one interesting test case that was testing register allocation.

Reviewed By: frasercrmck

Differential Revision: https://reviews.llvm.org/D125108

show more ...

[RISCV] Fix incorrect policy implement for unmasked vslidedown and vslideup.

vslideup works by leaving elements 0<i<OFFSET undisturbed.
so it need the destination operand as input for correctness
re

[RISCV] Fix incorrect policy implement for unmasked vslidedown and vslideup.

vslideup works by leaving elements 0<i<OFFSET undisturbed.
so it need the destination operand as input for correctness
regardless of policy. Add a operand to indicate policy.

We also add policy operand for unmaksed vslidedown to keep the interface consistent with vslideup
because vslidedown have only undisturbed at 0<i<vstart but user have no way to control of vstart.

Reviewed By: rogfer01, craig.topper

Differential Revision: https://reviews.llvm.org/D124186

show more ...

[RISCV] Add +experimental-zvfh extension to cover half types in vectors.

Currently we allow half types in vectors if the scalar Zfh extension
is enabled. This behavior is not inline with the vector

[RISCV] Add +experimental-zvfh extension to cover half types in vectors.

Currently we allow half types in vectors if the scalar Zfh extension
is enabled. This behavior is not inline with the vector spec. For f32
and f64 types, the Zve32f, Zve64f, Zve64d, and V explicitly control
the availablity of floating point types in vectors.

In order to make our compiler compliant, we either need to remove all support
for half in vectors or we need an extension to control it.

Draft spec here https://github.com/riscv/riscv-v-spec/pull/780

Reviewed By: kito-cheng

Differential Revision: https://reviews.llvm.org/D121345

show more ...

[RISCV] Remove experimental prefix from rvv-related extensions.

Extensions affected: +v, +zve*, +zvl*

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D117860

[RISCV] update zfh and zfhmin extention to v1.0

`zfh` and `zfhmin` have been ratified, with version 1.0.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D117098

[RISCV] Reorder the vector register allocation order.

GPR uses argument registers as the first group of registers to allocate.
This patch uses vector argument registers, v8 to v23, as the first grou

[RISCV] Reorder the vector register allocation order.

GPR uses argument registers as the first group of registers to allocate.
This patch uses vector argument registers, v8 to v23, as the first group
to allocate.

Differential Revision: https://reviews.llvm.org/D111304

show more ...

[RISCV] Manually emit the best shift for VSCALE lowering to improve codegen.

We assume VLENB is a multiple of 8 and previously relied on shift
pairs being optimized to an AND+SHL/SHR and computeKnow

[RISCV] Manually emit the best shift for VSCALE lowering to improve codegen.

We assume VLENB is a multiple of 8 and previously relied on shift
pairs being optimized to an AND+SHL/SHR and computeKnownBits
removing the AND. This doesn't happen if (vlenb >> 3) gets CSEd
to have multiple uses. This patch manually emits the best shift
to workaround this.

show more ...

[RISCV][NFC] Add a single space after comma for VType

In most of cases, it has a single space after comma in assembly operands.

Reviewed By: craig.topper

Differential Revision: https://reviews.llv

[RISCV][NFC] Add a single space after comma for VType

In most of cases, it has a single space after comma in assembly operands.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D103790

show more ...

[RISCV] Use X0 as destination of inserted vsetvli when possible.

We aren't going to connect the result to anything so we might
as well avoid allocating a register.

Reviewed By: frasercrmck, HsiangK

[RISCV] Use X0 as destination of inserted vsetvli when possible.

We aren't going to connect the result to anything so we might
as well avoid allocating a register.

Reviewed By: frasercrmck, HsiangKai

Differential Revision: https://reviews.llvm.org/D102031

show more ...

[RISCV] Test llvm.experimental.vector.insert intrinsics on RV32

RV32 is able to use the llvm.experimental.vector.insert intrinsics too.
This patch ensures they're tested.

Reviewed By: khchen, asb

[RISCV] Test llvm.experimental.vector.insert intrinsics on RV32

RV32 is able to use the llvm.experimental.vector.insert intrinsics too.
This patch ensures they're tested.

Reviewed By: khchen, asb

Differential Revision: https://reviews.llvm.org/D99655

show more ...

[CodeGen] Fix issues with scalable-vector INSERT/EXTRACT_SUBVECTORs

This patch addresses a few issues when dealing with scalable-vector
INSERT_SUBVECTOR and EXTRACT_SUBVECTOR nodes.

When legalizing

[CodeGen] Fix issues with scalable-vector INSERT/EXTRACT_SUBVECTORs

This patch addresses a few issues when dealing with scalable-vector
INSERT_SUBVECTOR and EXTRACT_SUBVECTOR nodes.

When legalizing in DAGTypeLegalizer::SplitVecRes_INSERT_SUBVECTOR, we
store the low and high halves to the stack separately. The offset for
the high half was calculated incorrectly.

Additionally, we can optimize this process when we can detect that the
subvector is contained entirely within the low/high split vector type.
While this optimization is valid on scalable vectors, when performing
the 'high' optimization, the subvector must also be a scalable vector.
Note that the 'low' optimization is still conservative: it may be
possible to insert v2i32 into the low half of a split nxv1i32/nxv1i32,
but we can't guarantee it. It is always possible to insert v2i32 into
nxv2i32 or v2i32 into nxv4i32+2 as we know vscale is at least 1.

Lastly, in SelectionDAG::isSplatValue, we early-exit on the extracted subvector value
type being a scalable vector, forgetting that we can also extract a
fixed-length vector from a scalable one.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D98495

show more ...

[RISCV] Support INSERT_SUBVECTOR on vector masks

Like with EXTRACT_SUBVECTOR, INSERT_SUBVECTOR poses a problem
for vector masks as RVV isn't able to slide mask types around. We choose
instead to bit

[RISCV] Support INSERT_SUBVECTOR on vector masks

Like with EXTRACT_SUBVECTOR, INSERT_SUBVECTOR poses a problem
for vector masks as RVV isn't able to slide mask types around. We choose
instead to bitcast to equivalently-sized i8 types where we can, else we
zero-extend, perform the operation, and truncate back down.

One test was left disabled due to a crash in the legalizer.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D97559

show more ...

[RISCV] Fix INSERT/EXTRACT_SUBVECTOR on fractional LMUL types

This patch fixes a bug where the lowering for INSERT_SUBVECTOR and
EXTRACT_SUBVECTOR would insist on first extracting a register-aligned

[RISCV] Fix INSERT/EXTRACT_SUBVECTOR on fractional LMUL types

This patch fixes a bug where the lowering for INSERT_SUBVECTOR and
EXTRACT_SUBVECTOR would insist on first extracting a register-aligned
LMUL1 vector type before perfoming the slide up/down. This was even if
the vector was a fractional LMUL type, in which case the aligned
EXTRACT_SUBVECTOR was invalid.

This issue only occurred for scalable vector types, but a variety of
tests for both scalable and fixed-length vectors have been added to
ensure this does not regress in the future.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D97556

show more ...

[RISCV] Support insertion of misaligned subvectors

This patch extends the support for RVV INSERT_SUBVECTOR to cover those
which don't align to a vector register boundary. Like the support for
EXTRAC

[RISCV] Support insertion of misaligned subvectors

This patch extends the support for RVV INSERT_SUBVECTOR to cover those
which don't align to a vector register boundary. Like the support for
EXTRACT_SUBVECTOR in D96959, it accomplishes this by extracting the
nearest register-sized subvector (a subregister operation), then sliding
the vector down with VSLIDEDOWN, inserting the subvector to the first
position, and sliding the vector back up again afterwards.

Unlike subvector extraction, for vectors that occupy less than a full
vector register we must preserve the untouched elements. We do this by
lowering to an LMUL=1 INSERT_SUBVECTOR using the above method and
lowering that to a VSLIDEUP with a zero offset. This uses a
tail-undisturbed policy and so has the effect of "sliding in" the
subvector elements while preserving the surrounding ones.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D96972

show more ...

[RISCV] Begin to support more subvector inserts/extracts

This patch adds support for INSERT_SUBVECTOR and EXTRACT_SUBVECTOR
(nominally where both operands are scalable vector types) where the
vector

[RISCV] Begin to support more subvector inserts/extracts

This patch adds support for INSERT_SUBVECTOR and EXTRACT_SUBVECTOR
(nominally where both operands are scalable vector types) where the
vector, subvector, and index align sufficiently to allow decomposition
to subregister manipulation:

* For extracts, the extracted subvector must correctly align with the
lower elements of a vector register.
* For inserts, the inserted subvector must be at least one full vector
register, and correctly align as above.

This approach should work for fixed-length vector insertion/extraction
too, but that will come later.

Reviewed By: craig.topper, khchen, arcbbb

Differential Revision: https://reviews.llvm.org/D96873

show more ...