[mlir][vector][nfc] Rename index optimizations option

We are using "enable-index-optimizations" and "indexOptimizations" as
names for an optimization that consists of using i32 for indices within
a

[mlir][vector][nfc] Rename index optimizations option

We are using "enable-index-optimizations" and "indexOptimizations" as
names for an optimization that consists of using i32 for indices within
a vector. For instance, when building a vector comparison for mask
generation. The name is confusing and suggests a scope beyond these
vector indices. This change makes the function of the option explicit
in its name.

Differential Revision: https://reviews.llvm.org/D122415

show more ...

[mlir][Vector] Enable create_mask for scalable vectors

The way vector.create_mask is currently lowered is
vector-length-dependent, and therefore incompatible with scalable vector
types. This patch a

[mlir][Vector] Enable create_mask for scalable vectors

The way vector.create_mask is currently lowered is
vector-length-dependent, and therefore incompatible with scalable vector
types. This patch adds an alternative lowering path for create_mask
operations that return a scalable vector mask.

Differential Revision: https://reviews.llvm.org/D118248

show more ...

Apply clang-tidy fixes for modernize-use-default-member-init to MLIR (NFC)

[mlir][Linalg] NFC - Make more option names consistent.

Differential Revision: https://reviews.llvm.org/D112640

[mlir] Rename AVX512 dialect to X86Vector

We will soon be adding non-AVX512 operations to MLIR, such as AVX's rsqrt. In https://reviews.llvm.org/D99818 several possibilities were discussed, namely t

[mlir] Rename AVX512 dialect to X86Vector

We will soon be adding non-AVX512 operations to MLIR, such as AVX's rsqrt. In https://reviews.llvm.org/D99818 several possibilities were discussed, namely to (1) add non-AVX512 ops to the AVX512 dialect, (2) add more dialects (e.g. AVX dialect for AVX rsqrt), and (3) expand the scope of the AVX512 to include these SIMD x86 ops, thereby renaming the dialect to something more accurate such as X86Vector.

Consensus was reached on option (3), which this patch implements.

Reviewed By: aartbik, ftynse, nicolasvasilache

Differential Revision: https://reviews.llvm.org/D100119

show more ...

[mlir] Add "mask" operand to vector.transfer_read/write.

Also factors out out-of-bounds mask generation from vector.transfer_read/write into a new MaterializeTransferMask pattern.

Differential Revi

[mlir] Add "mask" operand to vector.transfer_read/write.

Also factors out out-of-bounds mask generation from vector.transfer_read/write into a new MaterializeTransferMask pattern.

Differential Revision: https://reviews.llvm.org/D100001

show more ...

[PatternMatch] Big mechanical rename OwningRewritePatternList -> RewritePatternSet and insert -> add. NFC

This doesn't change APIs, this just cleans up the many in-tree uses of these
names to use t

[PatternMatch] Big mechanical rename OwningRewritePatternList -> RewritePatternSet and insert -> add. NFC

This doesn't change APIs, this just cleans up the many in-tree uses of these
names to use the new preferred names. We'll keep the old names around for a
couple weeks to help transitions.

Differential Revision: https://reviews.llvm.org/D99127

show more ...

[mlir][amx] Add Intel AMX dialect (architectural-specific vector dialect)

The Intel Advanced Matrix Extensions (AMX) provides a tile matrix
multiply unit (TMUL), a tile control register (TILECFG), a

[mlir][amx] Add Intel AMX dialect (architectural-specific vector dialect)

The Intel Advanced Matrix Extensions (AMX) provides a tile matrix
multiply unit (TMUL), a tile control register (TILECFG), and eight
tile registers TMM0 through TMM7 (TILEDATA). This new MLIR dialect
provides a bridge between MLIR concepts like vectors and memrefs
and the lower level LLVM IR details of AMX.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D98470

show more ...

[mlir][ArmSVE][RFC] Add an ArmSVE dialect

This revision starts an Arm-specific ArmSVE dialect discussed in the discourse RFC thread:

https://llvm.discourse.group/t/rfc-vector-dialects-neon-and-sve/

[mlir][ArmSVE][RFC] Add an ArmSVE dialect

This revision starts an Arm-specific ArmSVE dialect discussed in the discourse RFC thread:

https://llvm.discourse.group/t/rfc-vector-dialects-neon-and-sve/2284

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D92172

show more ...

[mlir][ArmNeon][RFC] Add a Neon dialect

This revision starts an Arm-specific ArmNeon dialect discussed in the [discourse RFC thread](https://llvm.discourse.group/t/rfc-vector-dialects-neon-and-sve/2

[mlir][ArmNeon][RFC] Add a Neon dialect

This revision starts an Arm-specific ArmNeon dialect discussed in the [discourse RFC thread](https://llvm.discourse.group/t/rfc-vector-dialects-neon-and-sve/2284).

Differential Revision: https://reviews.llvm.org/D92171

show more ...

[mlir][vector] rephrased description

More carefully worded description. Added constructor to options.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D92664

[mlir][vector][avx512] move avx512 lowering pass into general vector lowering

A separate AVX512 lowering pass does not compose well with the regular
vector lowering pass. As such, it is at risk of c

[mlir][vector][avx512] move avx512 lowering pass into general vector lowering

A separate AVX512 lowering pass does not compose well with the regular
vector lowering pass. As such, it is at risk of code duplication and
lowering inconsistencies. This change removes the separate AVX512 lowering
pass and makes it an "option" in the regular vector lowering pass
(viz. vector dialect "augmented" with AVX512 dialect).

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D92614

show more ...

[mlir] [VectorOps] Enable 32-bit index optimizations

Rationale:
After some discussion we decided that it is safe to assume 32-bit
indices for all subscripting in the vector dialect (it is unlikely
t

[mlir] [VectorOps] Enable 32-bit index optimizations

Rationale:
After some discussion we decided that it is safe to assume 32-bit
indices for all subscripting in the vector dialect (it is unlikely
the dialect will be used; or even work; for such long vectors).
So rather than detecting specific situations that can exploit
32-bit indices with higher parallel SIMD, we just optimize it
by default, and let users that don't want it opt-out.

Reviewed By: nicolasvasilache, bkramer

Differential Revision: https://reviews.llvm.org/D87404

show more ...

[mlir] [VectorOps] Improve SIMD compares with narrower indices

When allowed, use 32-bit indices rather than 64-bit indices in the
SIMD computation of masks. This runs up to 2x and 4x faster on
a num

[mlir] [VectorOps] Improve SIMD compares with narrower indices

When allowed, use 32-bit indices rather than 64-bit indices in the
SIMD computation of masks. This runs up to 2x and 4x faster on
a number of AVX2 and AVX512 microbenchmarks.

Reviewed By: bkramer

Differential Revision: https://reviews.llvm.org/D87116

show more ...

[mlir] [VectorOps] Expose lowering pass options programmatically

The ConvertVectorToLLVM pass defines options that can be passed
on the command line (currently only reassociation of FP reductions
th

[mlir] [VectorOps] Expose lowering pass options programmatically

The ConvertVectorToLLVM pass defines options that can be passed
on the command line (currently only reassociation of FP reductions
through -convert-vector-to-llvm='reassociate-fp-reductions). This
CL enables setting these options programmatically (forward looking
to more options than just reassociation, as well as setting the
values from code rather than command line).

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D83420

show more ...

[mlir] [VectorOps] Add the ability to mark FP reductions with "reassociate" attribute

Rationale:
In general, passing "fastmath" from MLIR to LLVM backend is not supported, and even just providing su

[mlir] [VectorOps] Add the ability to mark FP reductions with "reassociate" attribute

Rationale:
In general, passing "fastmath" from MLIR to LLVM backend is not supported, and even just providing such a feature for experimentation is under debate. However, passing fine-grained fastmath related attributes on individual operations is generally accepted. This CL introduces an option to instruct the vector-to-llvm lowering phase to annotate floating-point reductions with the "reassociate" fastmath attribute, which allows the LLVM backend to use SIMD implementations for such constructs. Oher lowering passes can start using this mechanism right away in cases where reassociation is allowed.

Benefit:
For some microbenchmarks on x86-avx2, speedups over 20 were observed for longer vector (due to cleaner, spill-free and SIMD exploiting code).

Usage:
mlir-opt --convert-vector-to-llvm="reassociate-fp-reductions"

Reviewed By: ftynse, mehdi_amini

Differential Revision: https://reviews.llvm.org/D82624

show more ...

[MLIR][SPIRVToLLVM] Add skeleton for SPIR-V to LLVM dialect conversion

These commits set up the skeleton for SPIR-V to LLVM dialect conversion.
I created SPIR-V to LLVM pass, registered it in Passes

[MLIR][SPIRVToLLVM] Add skeleton for SPIR-V to LLVM dialect conversion

These commits set up the skeleton for SPIR-V to LLVM dialect conversion.
I created SPIR-V to LLVM pass, registered it in Passes.td, InitAllPasses.h.
Added a pattern for `spv.BitwiseAndOp` and tests for it. Integer, float
and vector types are converted through LLVMTypeConverter.

Differential Revision: https://reviews.llvm.org/D81100

show more ...

[mlir][Pass] Remove the use of CRTP from the Pass classes

This revision removes all of the CRTP from the pass hierarchy in preparation for using the tablegen backend instead. This creates a much cle

[mlir][Pass] Remove the use of CRTP from the Pass classes

This revision removes all of the CRTP from the pass hierarchy in preparation for using the tablegen backend instead. This creates a much cleaner interface in the C++ code, and naturally fits with the rest of the infrastructure. A new utility class, PassWrapper, is added to replicate the existing behavior for passes not suitable for using the tablegen backend.

Differential Revision: https://reviews.llvm.org/D77350

show more ...

[mlir][Vector] Mostly-NFC - Restructure options for lowering to LLVM Matrix Intrinsics

Summary:
This revision restructures the calling of vector transforms to make it more flexible to ask for loweri

[mlir][Vector] Mostly-NFC - Restructure options for lowering to LLVM Matrix Intrinsics

Summary:
This revision restructures the calling of vector transforms to make it more flexible to ask for lowering through LLVM matrix intrinsics.
This also makes sure we bail out in degenerate cases (i.e. 1) in which LLVM complains about not being able to scalarize.

Differential Revision: https://reviews.llvm.org/D76266

show more ...

[mlir][Vector] Add a vector.matrix_multiply op on 1-D vectors

Summary: This op mirrors the llvm.intr counterpart and allows lowering + type conversions in a progressive fashion.

Differential Revisi

[mlir][Vector] Add a vector.matrix_multiply op on 1-D vectors

Summary: This op mirrors the llvm.intr counterpart and allows lowering + type conversions in a progressive fashion.

Differential Revision: https://reviews.llvm.org/D75775

show more ...

Mass update the MLIR license header to mention "Part of the LLVM project"

This is an artifact from merging MLIR into LLVM, the file headers are
now aligned with the rest of the project.

Adjust License.txt file to use the LLVM license

PiperOrigin-RevId: 286906740

Refactor dependencies to expose Vector transformations as patterns - NFC

This CL refactors some of the MLIR vector dependencies to allow decoupling VectorOps, vector analysis, vector transformations

Refactor dependencies to expose Vector transformations as patterns - NFC

This CL refactors some of the MLIR vector dependencies to allow decoupling VectorOps, vector analysis, vector transformations and vector conversions from each other.
This makes the system more modular and allows extracting VectorToVector into VectorTransforms that do not depend on vector conversions.

This refactoring exhibited a bunch of cyclic library dependencies that have been cleaned up.

PiperOrigin-RevId: 283660308

show more ...