[mlir] Remove unneeded cl::ZeroOrMore for ListOption variables. NFC

[mlir] Don't use Optional::hasValue (NFC)

[mlir] move SCF headers to SCF/{IR,Transforms} respectively

This aligns the SCF dialect file layout with the majority of the dialects.

Reviewed By: jpienaar

Differential Revision: https://reviews.

[mlir] move SCF headers to SCF/{IR,Transforms} respectively

This aligns the SCF dialect file layout with the majority of the dialects.

Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D128049

show more ...

[mlir][vector] NFC remove dependency of VectorTransform to GPU dialect

Make the reduction distribution pattern more generic and remove layering
problem. The new pattern to distribute reduction is no

[mlir][vector] NFC remove dependency of VectorTransform to GPU dialect

Make the reduction distribution pattern more generic and remove layering
problem. The new pattern to distribute reduction is now independent of
GPU and takes a lamdba to decide how the distributed reduction should be
generated.

Differential Revision: https://reviews.llvm.org/D127867

show more ...

[mlir][vector] Add pattern to distribute vector reduction to GPU shuffles

Add a pattern to do ad hoc lowering of vector.reduction to a sequence of
warp shuffles. This allow distributing reduction on

[mlir][vector] Add pattern to distribute vector reduction to GPU shuffles

Add a pattern to do ad hoc lowering of vector.reduction to a sequence of
warp shuffles. This allow distributing reduction on a warp for GPU targets.
Also add an execution test for warp reduction.

co-authored with @springerm

Differential Revision: https://reviews.llvm.org/D127176

show more ...

[mlir][vector] Add patterns to ppropagate vector distribution

Add patterns to propagate vector distribution and remove dead
arguments. This handles propagation for several vector operations.

recomm

[mlir][vector] Add patterns to ppropagate vector distribution

Add patterns to propagate vector distribution and remove dead
arguments. This handles propagation for several vector operations.

recommit after minor bug fix.

Differential Revision: https://reviews.llvm.org/D127167

show more ...

Revert "[mlir][vector] Add patterns to ppropagate vector distribution"

This reverts commit 1c84800c42d2183a29392c175c8d5f20a4be65d2.

This was causing asan crash.

[mlir][vector] Add patterns to ppropagate vector distribution

Add patterns to propagate vector distribution and remove dead
arguments. This handles propagation for several vector operations.

Differ

[mlir][vector] Add patterns to ppropagate vector distribution

Add patterns to propagate vector distribution and remove dead
arguments. This handles propagation for several vector operations.

Differential Revision: https://reviews.llvm.org/D127167

show more ...

[mlir][vector] Add patterns for vector distribution

Add pattern to hoist scalar code outside of warp distribute region as
those cannot be distributed and we would want to execute them on all
the lan

[mlir][vector] Add patterns for vector distribution

Add pattern to hoist scalar code outside of warp distribute region as
those cannot be distributed and we would want to execute them on all
the lanes.
Add patterns to distribute transfer_write ops. Those operations can be
distributed in different ways and it is control by user.

Differential Revision: https://reviews.llvm.org/D127152

show more ...

[mlir][gpu] Move GPU headers into IR/ and Transforms/

Depends on D127350

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D127352

Recommit "[mlir][vector] Allow unroll of contraction in arbitrary order"

Fixed issue with vector.contract default unroll permutation.

Adds support for vector unroll transformations to unroll in dif

Recommit "[mlir][vector] Allow unroll of contraction in arbitrary order"

Fixed issue with vector.contract default unroll permutation.

Adds support for vector unroll transformations to unroll in different
orders. For example, the vector.contract can be unrolled into a
smaller set of contractions. There is a choice of how to unroll the
decomposition based on the traversal order of (dim0, dim1, dim2).
The choice of traversal order can now be specified by a callback which
given by the caller of the transform. For now, only the
vector.contract, vector.transfer_read/transfer_write operations
support the callback.

Differential Revision: https://reviews.llvm.org/D127004

show more ...

Revert "[mlir][vector] Allow unroll of contraction in arbitrary order"

Reverts commit 1469ebf8382107e0344173f362b690d19e24029d (original commit)
Reverts commit a392a39f75af586e3d3cd046a8361939277e06

Revert "[mlir][vector] Allow unroll of contraction in arbitrary order"

Reverts commit 1469ebf8382107e0344173f362b690d19e24029d (original commit)
Reverts commit a392a39f75af586e3d3cd046a8361939277e067f (build fix for above commit)

The commit broke tests in out-of-tree projects, indicating that some logical
error was made in the previous change but not covered by current tests.

show more ...

[mlir][vector] Allow unroll of contraction in arbitrary order

Adds supprot for vector unroll transformations to unroll in different
orders. For example, the `vector.contract` can be unrolled into a

[mlir][vector] Allow unroll of contraction in arbitrary order

Adds supprot for vector unroll transformations to unroll in different
orders. For example, the `vector.contract` can be unrolled into a
smaller set of contractions. There is a choice of how to unroll the
decomposition based on the traversal order of (dim0, dim1, dim2).
The choice of traversal order can now be specified by a callback which
given by the caller of the transform. For now, only the
`vector.contract`, `vector.transfer_read/transfer_write` operations
support the callback.

Differential Revision: https://reviews.llvm.org/D127004

show more ...

[mlir][vector] Add new lowering mode to vector.contractionOp

Add lowering for cases where the reduction dimension is fully unrolled.
It is common to unroll the reduction dimension, therefore we woul

[mlir][vector] Add new lowering mode to vector.contractionOp

Add lowering for cases where the reduction dimension is fully unrolled.
It is common to unroll the reduction dimension, therefore we would want
to lower the contractions to an elementwise vector op in this case.

Differential Revision: https://reviews.llvm.org/D126120

show more ...

[mlir][vector] Add lowering pattern for vector.warp_execute_on_lane_0 op

Add lowering of the vector.warp_execute_on_lane_0 into scf.if plus memory
transfer for the operands and yield values.

This a

[mlir][vector] Add lowering pattern for vector.warp_execute_on_lane_0 op

Add lowering of the vector.warp_execute_on_lane_0 into scf.if plus memory
transfer for the operands and yield values.

This also add an integration test running on GPU warp. The same tests can be
later re-used with different comment lines to tests distribution
transformations.

This is mostly from @springerm contribution.

Differential Revision: https://reviews.llvm.org/D125430

show more ...

[mlir:NFC] Remove the forward declaration of FuncOp in the mlir namespace

FuncOp has been moved to the `func` namespace for a little over a month, the
using directive can be dropped now.

[mlir][vector] Add unrolling pattern for TransposeOp

Support unrolling for vector.transpose following the same interface as
other vector unrolling ops.

Differential Revision: https://reviews.llvm.o

[mlir][vector] Add unrolling pattern for TransposeOp

Support unrolling for vector.transpose following the same interface as
other vector unrolling ops.

Differential Revision: https://reviews.llvm.org/D123688

show more ...

[mlir] Rework the implementation of TypeID

This commit restructures how TypeID is implemented to ideally avoid
the current problems related to shared libraries. This is done by changing
the "implici

[mlir] Rework the implementation of TypeID

This commit restructures how TypeID is implemented to ideally avoid
the current problems related to shared libraries. This is done by changing
the "implicit" fallback path to use the name of the type, instead of using
a static template variable (which breaks shared libraries). The major downside to this
is that it adds some additional initialization costs for the implicit path. Given the
use of type names for uniqueness in the fallback, we also no longer allow types
defined in anonymous namespaces to have an implicit TypeID. To simplify defining
an ID for these classes, a new `MLIR_DEFINE_EXPLICIT_INTERNAL_INLINE_TYPE_ID` macro
was added to allow for explicitly defining a TypeID directly on an internal class.

To help identify when types are using the fallback, `-debug-only=typeid` can be
used to log which types are using implicit ids.

This change generally only requires changes to the test passes, which are all defined
in anonymous namespaces, and thus can't use the fallback any longer.

Differential Revision: https://reviews.llvm.org/D122775

show more ...

[mlir:PassOption] Rework ListOption parsing and add support for std::vector/SmallVector options

ListOption currently uses llvm::cl::list under the hood, but the usages
of ListOption are generally a

[mlir:PassOption] Rework ListOption parsing and add support for std::vector/SmallVector options

ListOption currently uses llvm::cl::list under the hood, but the usages
of ListOption are generally a tad different from llvm::cl::list. This
commit codifies this by making ListOption implicitly comma separated,
and removes the explicit flag set for all of the current list options.
The new parsing for comma separation of ListOption also adds in support
for skipping over delimited sub-ranges (i.e. {}, [], (), "", ''). This
more easily supports nested options that use those as part of the
format, and this constraint (balanced delimiters) is already codified
in the syntax of pass pipelines.

See https://discourse.llvm.org/t/list-of-lists-pass-option/5950 for
related discussion

Differential Revision: https://reviews.llvm.org/D122879

show more ...

[mlir] Flip Vector dialect accessors used to prefixed form.

This has been on _Both for a couple of weeks. Flip usages in core with
intention to flip flag to _Prefixed in follow up. Needed to add a c

[mlir] Flip Vector dialect accessors used to prefixed form.

This has been on _Both for a couple of weeks. Flip usages in core with
intention to flip flag to _Prefixed in follow up. Needed to add a couple
of helper methods in AffineOps and Linalg to facilitate a pure flag flip
in follow up as some of these classes are used in templates and so
sensitive to Vector dialect changes.

Differential Revision: https://reviews.llvm.org/D122151

show more ...

[mlir] Move the Builtin FuncOp to the Func dialect

This commit moves FuncOp out of the builtin dialect, and into the Func
dialect. This move has been planned in some capacity from the moment
we made

[mlir] Move the Builtin FuncOp to the Func dialect

This commit moves FuncOp out of the builtin dialect, and into the Func
dialect. This move has been planned in some capacity from the moment
we made FuncOp an operation (years ago). This commit handles the
functional aspects of the move, but various aspects are left untouched
to ease migration: func::FuncOp is re-exported into mlir to reduce
the actual API churn, the assembly format still accepts the unqualified
`func`. These temporary measures will remain for a little while to
simplify migration before being removed.

Differential Revision: https://reviews.llvm.org/D121266

show more ...

[mlir][vector] Implement unrolling of ReductionOp

Differential Revision: https://reviews.llvm.org/D121597

[mlir][vector] Add unrolling pattern for multidim_reduce op

Implement the vectorLoopUnroll interface for MultiDimReduceOp and add a
pattern to do the unrolling following the same interface other vec

[mlir][vector] Add unrolling pattern for multidim_reduce op

Implement the vectorLoopUnroll interface for MultiDimReduceOp and add a
pattern to do the unrolling following the same interface other vector
unroll patterns.

Differential Revision: https://reviews.llvm.org/D121263

show more ...

[mlir] Rename the Standard dialect to the Func dialect

The last remaining operations in the standard dialect all revolve around
FuncOp/function related constructs. This patch simply handles the init

[mlir] Rename the Standard dialect to the Func dialect

The last remaining operations in the standard dialect all revolve around
FuncOp/function related constructs. This patch simply handles the initial
renaming (which by itself is already huge), but there are a large number
of cleanups unlocked/necessary afterwards:

* Removing a bunch of unnecessary dependencies on Func
* Cleaning up the From/ToStandard conversion passes
* Preparing for the move of FuncOp to the Func dialect

See the discussion at https://discourse.llvm.org/t/standard-dialect-the-final-chapter/6061

Differential Revision: https://reviews.llvm.org/D120624

show more ...

[mlir] Move SelectOp from Standard to Arithmetic

This is part of splitting up the standard dialect. See https://llvm.discourse.group/t/standard-dialect-the-final-chapter/ for discussion.

Differenti

[mlir] Move SelectOp from Standard to Arithmetic

This is part of splitting up the standard dialect. See https://llvm.discourse.group/t/standard-dialect-the-final-chapter/ for discussion.

Differential Revision: https://reviews.llvm.org/D118648

show more ...