[mlir] [VectorOps] Expose lowering pass options programmatically

The ConvertVectorToLLVM pass defines options that can be passed
on the command line (currently only reassociation of FP reductions
th

[mlir] [VectorOps] Expose lowering pass options programmatically

The ConvertVectorToLLVM pass defines options that can be passed
on the command line (currently only reassociation of FP reductions
through -convert-vector-to-llvm='reassociate-fp-reductions). This
CL enables setting these options programmatically (forward looking
to more options than just reassociation, as well as setting the
values from code rather than command line).

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D83420

show more ...

[mlir][NFC] Remove usernames and google bug numbers from TODO comments.

These were largely leftover from when MLIR was a google project, and don't really follow LLVM guidelines.

[mlir] [VectorOps] Add the ability to mark FP reductions with "reassociate" attribute

Rationale:
In general, passing "fastmath" from MLIR to LLVM backend is not supported, and even just providing su

[mlir] [VectorOps] Add the ability to mark FP reductions with "reassociate" attribute

Rationale:
In general, passing "fastmath" from MLIR to LLVM backend is not supported, and even just providing such a feature for experimentation is under debate. However, passing fine-grained fastmath related attributes on individual operations is generally accepted. This CL introduces an option to instruct the vector-to-llvm lowering phase to annotate floating-point reductions with the "reassociate" fastmath attribute, which allows the LLVM backend to use SIMD implementations for such constructs. Oher lowering passes can start using this mechanism right away in cases where reassociation is allowed.

Benefit:
For some microbenchmarks on x86-avx2, speedups over 20 were observed for longer vector (due to cleaner, spill-free and SIMD exploiting code).

Usage:
mlir-opt --convert-vector-to-llvm="reassociate-fp-reductions"

Reviewed By: ftynse, mehdi_amini

Differential Revision: https://reviews.llvm.org/D82624

show more ...

[mlir][DialectConversion] Refactor how block argument types get converted

This revision removes the TypeConverter parameter passed to the apply* methods, and instead moves the responsibility of regi

[mlir][DialectConversion] Refactor how block argument types get converted

This revision removes the TypeConverter parameter passed to the apply* methods, and instead moves the responsibility of region type conversion to patterns. The types of a region can be converted using the 'convertRegionTypes' method, which acts similarly to the existing 'applySignatureConversion'. This method ensures that all blocks within, and including those moved into, a region will have the block argument types converted using the provided converter.

This has the benefit of making more of the legalization logic controlled by patterns, instead of being handled explicitly by the driver. It also opens up the possibility to support multiple type conversions at some point in the future.

This revision also adds a new utility class `FailureOr<T>` that provides a LogicalResult friendly facility for returning a failure or a valid result value.

Differential Revision: https://reviews.llvm.org/D81681

show more ...

[mlir] [VectorOps] remove print_i1 from runtime support library

Summary:
The "i1" (viz. bool) type does not have a proper equivalent on the "C"
size. So, to avoid any ABIs issues, we simply use prin

[mlir] [VectorOps] remove print_i1 from runtime support library

Summary:
The "i1" (viz. bool) type does not have a proper equivalent on the "C"
size. So, to avoid any ABIs issues, we simply use print_i32 on an i32
value of one or zero for true and false. This has the added advantage
that one less function needs to be implemented when porting the runtime
support library.

Reviewers: ftynse, bkramer, nicolasvasilache

Reviewed By: ftynse

Subscribers: mehdi_amini, rriddle, jpienaar, shauheen, antiagainst, nicolasvasilache, arpith-jacob, mgester, lucyrfox, liufengdb, stephenneuendorffer, Joonsoo, grosul1, frgossen, Kayjukh, jurahul, msifontes

Tags: #mlir

Differential Revision: https://reviews.llvm.org/D82048

show more ...

[mlir] Remove OperandAdaptor

Use ::Adaptor alias instead uniformly. Makes the naming more consistent as
adaptor can refer to attributes now too.

Differential Revision: https://reviews.llvm.org/D817

[mlir] Remove OperandAdaptor

Use ::Adaptor alias instead uniformly. Makes the naming more consistent as
adaptor can refer to attributes now too.

Differential Revision: https://reviews.llvm.org/D81789

show more ...

[mlir][Vector] Fix vector.transfer alignment calculation

https://reviews.llvm.org/D79246 introduces alignment propagation for vector transfer operations. Unfortunately, the alignment calculation is

[mlir][Vector] Fix vector.transfer alignment calculation

https://reviews.llvm.org/D79246 introduces alignment propagation for vector transfer operations. Unfortunately, the alignment calculation is incorrect and can result in crashes.

This revision fixes the calculation by using the natural alignment of the memref elemental type, instead of the resulting vector type.

If more alignment is desired, it can be done in 2 ways:
1. use a proper vector.type_cast to transform a memref<axbxcxdxf32> into a memref<axbxvector<cxdxf32>> giving a natural alignment of vector<cxdxf32>
2. add an alignment attribute to vector transfer operations and propagate it.

With this change the alignment in the relevant tests goes down from 128 to 4.

Lastly, a few minor cleanups are performed and the custom `isMinorIdentityMap` is deprecated.

Differential Revision: https://reviews.llvm.org/D80734

show more ...

[mlir] [VectorOps] Add 'vector.flat_transpose' operation

Summary:
Provides a representation of the linearized LLVM instrinsic.
With tests and lowering implementation to LLVM IR dialect.
Prepares bet

[mlir] [VectorOps] Add 'vector.flat_transpose' operation

Summary:
Provides a representation of the linearized LLVM instrinsic.
With tests and lowering implementation to LLVM IR dialect.
Prepares better lowering for 2-D vector.transpose.

Reviewers: nicolasvasilache, ftynse, reidtatge, bkramer, dcaballe

Reviewed By: ftynse, dcaballe

Subscribers: mehdi_amini, rriddle, jpienaar, shauheen, antiagainst, nicolasvasilache, arpith-jacob, mgester, lucyrfox, liufengdb, stephenneuendorffer, Joonsoo, grosul1, frgossen, Kayjukh, jurahul, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80419

show more ...

[mlir][Vector] Add an optional "masked" boolean array attribute to vector transfer operations

Summary:
Vector transfer ops semantic is extended to allow specifying a per-dimension `masked`
attribute

[mlir][Vector] Add an optional "masked" boolean array attribute to vector transfer operations

Summary:
Vector transfer ops semantic is extended to allow specifying a per-dimension `masked`
attribute. When the attribute is false on a particular dimension, lowering to LLVM emits
unmasked load and store operations.

Differential Revision: https://reviews.llvm.org/D80098

show more ...

[mlir] [VectorOps] Implement vector.create_mask lowering to LLVM IR

Summary:
First, compact implementation of lowering to LLVM IR. A bit more
challenging than the constant mask due to the dynamic in

[mlir] [VectorOps] Implement vector.create_mask lowering to LLVM IR

Summary:
First, compact implementation of lowering to LLVM IR. A bit more
challenging than the constant mask due to the dynamic indices, of course.
I like to hear if there are more efficient ways of doing this in LLVM,
but this for now at least gives us a functional reference implementation.

Reviewers: nicolasvasilache, ftynse, bkramer, reidtatge, andydavis1, mehdi_amini

Reviewed By: nicolasvasilache

Subscribers: mehdi_amini, rriddle, jpienaar, shauheen, antiagainst, nicolasvasilache, arpith-jacob, mgester, lucyrfox, liufengdb, stephenneuendorffer, Joonsoo, grosul1, frgossen, Kayjukh, jurahul, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D79954

show more ...

[mlir][Vector] NFC - Rename vector.strided_slice into vector.extract_strided_slice

Differential Revision: https://reviews.llvm.org/D79734

[mlir] VectorToLLVM: propagate up from replaceTransferOp

In the Vector to LLVM conversion, the `replaceTransferOp` function calls
into a type converter that may fail and suppresses the status. Chang

[mlir] VectorToLLVM: propagate up from replaceTransferOp

In the Vector to LLVM conversion, the `replaceTransferOp` function calls
into a type converter that may fail and suppresses the status. Change
the function to return the failure status instead, Since it is called
from a pattern, the failure can be readily propagated to the rest of
infrastructure.

show more ...

[mlir][vector] set alignment when lowering transfer_read and transfer_write.

When emitting masked load / store, set alignment from data layout.

Differential Revision: https://reviews.llvm.org/D79246

[mlir] [VectorOps] Included i1 support for vector.print

Summary:
Added boolean support to vector.print.
Useful for upcoming "mask" tests.

Reviewers: ftynse, nicolasvasilache, andydavis1

Reviewed B

[mlir] [VectorOps] Included i1 support for vector.print

Summary:
Added boolean support to vector.print.
Useful for upcoming "mask" tests.

Reviewers: ftynse, nicolasvasilache, andydavis1

Reviewed By: andydavis1

Subscribers: mehdi_amini, rriddle, jpienaar, shauheen, antiagainst, nicolasvasilache, arpith-jacob, mgester, lucyrfox, liufengdb, Joonsoo, grosul1, frgossen, Kayjukh, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D79198

show more ...

[mlir][vector] let transfer_read and transfer_write take non-zero addrspace.

Enhance lowering logic and tests so vector.transfer_read and
vector.transfer_write take memrefs on non-zero addrspaces.

[mlir][vector] let transfer_read and transfer_write take non-zero addrspace.

Enhance lowering logic and tests so vector.transfer_read and
vector.transfer_write take memrefs on non-zero addrspaces.

Differential Revision: https://reviews.llvm.org/D79023

show more ...

[mlir][VectorOps] Extend VectorTransfer lowering to n-D memref with minor identity map

Summary: This revision extends the lowering of vector transfers to work with n-D memref and 1-D vector where th

[mlir][VectorOps] Extend VectorTransfer lowering to n-D memref with minor identity map

Summary: This revision extends the lowering of vector transfers to work with n-D memref and 1-D vector where the permutation map is an identity on the most minor dimensions (1 for now).

Differential Revision: https://reviews.llvm.org/D78925

show more ...

[mlir] [VectorOps] Progressive lowering of vector.broadcast

Summary:
Rather than having a full, recursive, lowering of vector.broadcast
to LLVM IR, it is much more elegant to have a progressive lowe

[mlir] [VectorOps] Progressive lowering of vector.broadcast

Summary:
Rather than having a full, recursive, lowering of vector.broadcast
to LLVM IR, it is much more elegant to have a progressive lowering
of each vector.broadcast into a lower dimensional vector.broadcast,
until only elementary vector operations remain. This results
in more elegant, step-wise code, that is easier to understand.
Also makes some optimizations in the generated code.

Reviewers: nicolasvasilache, mehdi_amini, andydavis1, grosul1

Reviewed By: nicolasvasilache

Subscribers: mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, nicolasvasilache, arpith-jacob, mgester, lucyrfox, liufengdb, Joonsoo, grosul1, frgossen, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D78071

show more ...

[MLIR][NFC] applyPatternsGreedily -> applyPatternsAndFoldGreedily

Rename mlir::applyPatternsGreedily -> applyPatternsAndFoldGreedily. The
new name is a more accurate description of the method - it p

[MLIR][NFC] applyPatternsGreedily -> applyPatternsAndFoldGreedily

Rename mlir::applyPatternsGreedily -> applyPatternsAndFoldGreedily. The
new name is a more accurate description of the method - it performs
both, application of the specified patterns and folding of all ops in
the op's region irrespective of whether any patterns have been supplied.

Differential Revision: https://reviews.llvm.org/D77478

show more ...

[mlir][Vector] Add lowering of 1-D vector transfer_read/write to masked load/store

Summary:
This revision adds support to lower 1-D vector transfers to LLVM.
A mask of the vector length is created t

[mlir][Vector] Add lowering of 1-D vector transfer_read/write to masked load/store

Summary:
This revision adds support to lower 1-D vector transfers to LLVM.
A mask of the vector length is created that compares the base offset + linear index to the dim of the vector.
In each position where this does not overflow (i.e. offset + vector index < dim), the mask is set to 1.

A notable fact is that the lowering uses llvm.dialect_cast to allow writing code in the simplest form by targeting the simplest mix of vector and LLVM dialects and
letting other conversions kick in.

Differential Revision: https://reviews.llvm.org/D77703

show more ...

[mlir] Add a new RewritePattern::hasBoundedRewriteRecursion hook.

Summary: Some pattern rewriters, like dialect conversion, prohibit the unbounded recursion(or reapplication) of patterns on generate

[mlir] Add a new RewritePattern::hasBoundedRewriteRecursion hook.

Summary: Some pattern rewriters, like dialect conversion, prohibit the unbounded recursion(or reapplication) of patterns on generated IR. Most patterns are not written with recursive application in mind, so will generally explode the stack if uncaught. This revision adds a hook to RewritePattern, `hasBoundedRewriteRecursion`, to signal that the pattern can safely be applied to the generated IR of a previous application of the same pattern. This allows for establishing a contract between the pattern and rewriter that the pattern knows and can handle the potential recursive application.

Differential Revision: https://reviews.llvm.org/D77782

show more ...

[mlir][Pass] Update the PassGen to generate base classes instead of utilities

Summary:
This is much cleaner, and fits the same structure as many other tablegen backends. This was not done originally

[mlir][Pass] Update the PassGen to generate base classes instead of utilities

Summary:
This is much cleaner, and fits the same structure as many other tablegen backends. This was not done originally as the CRTP in the pass classes made it overly verbose/complex.

Differential Revision: https://reviews.llvm.org/D77367

show more ...

[mlir][Pass] Remove the use of CRTP from the Pass classes

This revision removes all of the CRTP from the pass hierarchy in preparation for using the tablegen backend instead. This creates a much cle

[mlir][Pass] Remove the use of CRTP from the Pass classes

This revision removes all of the CRTP from the pass hierarchy in preparation for using the tablegen backend instead. This creates a much cleaner interface in the C++ code, and naturally fits with the rest of the infrastructure. A new utility class, PassWrapper, is added to replicate the existing behavior for passes not suitable for using the tablegen backend.

Differential Revision: https://reviews.llvm.org/D77350

show more ...

[mlir][Pass][NFC] Replace usages of ModulePass with OperationPass<ModuleOp>

ModulePass doesn't provide any special utilities and thus doesn't give enough benefit to warrant a special pass class. Thi

[mlir][Pass][NFC] Replace usages of ModulePass with OperationPass<ModuleOp>

ModulePass doesn't provide any special utilities and thus doesn't give enough benefit to warrant a special pass class. This revision replaces all usages with the more general OperationPass.

Differential Revision: https://reviews.llvm.org/D77339

show more ...

[mlir][Pass] Add support for generating pass utilities via tablegen

This revision adds support for generating utilities for passes such as options/statistics/etc. that can be inferred from the table

[mlir][Pass] Add support for generating pass utilities via tablegen

This revision adds support for generating utilities for passes such as options/statistics/etc. that can be inferred from the tablegen definition. This removes additional boilerplate from the pass, and also makes it easier to remove the reliance on the pass registry to provide certain things(e.g. the pass argument).

Differential Revision: https://reviews.llvm.org/D76659

show more ...

[mlir][Pass] Move the registration of conversion passes to tablegen

This removes the need to statically register conversion passes, and also puts all of the conversions within one centralized file.

[mlir][Pass] Move the registration of conversion passes to tablegen

This removes the need to statically register conversion passes, and also puts all of the conversions within one centralized file.

Differential Revision: https://reviews.llvm.org/D76658

show more ...