[mlir] Don't use Optional::hasValue (NFC)

[mlir] Modify SuperVectorize to generate select op->combiner op

Insert the select op before the combiner op when vectorizing a
reduction loop that needs a mask, so the vectorized reduction loop
can

[mlir] Modify SuperVectorize to generate select op->combiner op

Insert the select op before the combiner op when vectorizing a
reduction loop that needs a mask, so the vectorized reduction loop
can pass isLoopParallel check and be transformed correctly in later
passes.

Reviewed By: dcaballe

Differential Revision: https://reviews.llvm.org/D124047

show more ...

[mlir:NFC] Remove the forward declaration of FuncOp in the mlir namespace

FuncOp has been moved to the `func` namespace for a little over a month, the
using directive can be dropped now.

[mlir] Make OpBuilder::createOperation to accept raw inputs

This provides a way to create an operation without manipulating
OperationState directly. This is useful for creating unregistered ops.

Re

[mlir] Make OpBuilder::createOperation to accept raw inputs

This provides a way to create an operation without manipulating
OperationState directly. This is useful for creating unregistered ops.

Reviewed By: rriddle, mehdi_amini

Differential Revision: https://reviews.llvm.org/D120787

show more ...

[mlir][vector][NFC] Use CombiningKindAttr instead of StringAttr

This makes the op consistent with other ops in vector dialect.

Differential Revision: https://reviews.llvm.org/D119343

[mlir][Affine][Vector] NFC: fix examples in comments

s/-affine-vectorize/-affine-super-vectorize/g

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D118892

[mlir] Move SelectOp from Standard to Arithmetic

This is part of splitting up the standard dialect. See https://llvm.discourse.group/t/standard-dialect-the-final-chapter/ for discussion.

Differenti

[mlir] Move SelectOp from Standard to Arithmetic

This is part of splitting up the standard dialect. See https://llvm.discourse.group/t/standard-dialect-the-final-chapter/ for discussion.

Differential Revision: https://reviews.llvm.org/D118648

show more ...

[mlir][vector][NFC] Split into IR, Transforms and Utils

This reduces the dependencies of the MLIRVector target and makes the dialect consistent with other dialects.

Differential Revision: https://r

[mlir][vector][NFC] Split into IR, Transforms and Utils

This reduces the dependencies of the MLIRVector target and makes the dialect consistent with other dialects.

Differential Revision: https://reviews.llvm.org/D118533

show more ...

[mlir][Pass] Deprecate FunctionPass in favor of OperationPass<FuncOp>

The only benefit of FunctionPass is that it filters out function
declarations. This isn't enough to justify carrying it around,

[mlir][Pass] Deprecate FunctionPass in favor of OperationPass<FuncOp>

The only benefit of FunctionPass is that it filters out function
declarations. This isn't enough to justify carrying it around, as we can
simplify filter out declarations when necessary within the pass. We can
also explore with better scheduling primitives to filter out declarations
at the pipeline level in the future.

The definition of FunctionPass is left intact for now to allow time for downstream
users to migrate.

Differential Revision: https://reviews.llvm.org/D117182

show more ...

[mlir:Analysis] Move the LoopAnalysis library to Dialect/Affine/Analysis

The current state of the top level Analysis/ directory is that it contains two libraries;
a generic Analysis library (free fr

[mlir:Analysis] Move the LoopAnalysis library to Dialect/Affine/Analysis

The current state of the top level Analysis/ directory is that it contains two libraries;
a generic Analysis library (free from dialect dependencies), and a LoopAnalysis library
that contains various analysis utilities that originated from Affine loop transformations.
This commit moves the LoopAnalysis to the more appropriate home of `Dialect/Affine/Analysis/`,
given the use and intention of the majority of the code within it. After the move, if there
are generic utilities that would fit better in the top-level Analysis/ directory, we can move
them.

Differential Revision: https://reviews.llvm.org/D117351

show more ...

[mlir] Optimize OperationName construction and usage

When constructing an OperationName, the overwhelming majority of
cases are from registered operations. This revision adds a non-locked
lookup int

[mlir] Optimize OperationName construction and usage

When constructing an OperationName, the overwhelming majority of
cases are from registered operations. This revision adds a non-locked
lookup into the currently registered operations, which prevents locking
in the common case. This revision also optimizes several uses of
RegisteredOperationName that expect the operation to be registered,
e.g. such as in OpBuilder.

These changes provides a reasonable speedup (5-10%) in some
compilations, especially on platforms where locking is expensive.

Differential Revision: https://reviews.llvm.org/D117187

show more ...

[MLIR] Update allocs to memref.allocs in documentation.

Changed the remaining appearances of alloc to memref.alloc in several
documentation sections, since they lead to misunderstandings, if they
ar

[MLIR] Update allocs to memref.allocs in documentation.

Changed the remaining appearances of alloc to memref.alloc in several
documentation sections, since they lead to misunderstandings, if they
are used.

Differential Revision: https://reviews.llvm.org/D116999

show more ...

Remove redundant return statements (NFC)

Identified by readability-redundant-control-flow.

Fix a few unitialized class members in MLIR (NFC)

Flagged by Coverity.

[MLIR] Move AtomicRMW into MemRef dialect and enum into Arith

Per the discussion in https://reviews.llvm.org/D116345 it makes sense
to move AtomicRMWOp out of the standard dialect. This was accentua

[MLIR] Move AtomicRMW into MemRef dialect and enum into Arith

Per the discussion in https://reviews.llvm.org/D116345 it makes sense
to move AtomicRMWOp out of the standard dialect. This was accentuated by the
need to add a fold op with a memref::cast. The only dialect
that would permit this is the memref dialect (keeping it in the standard dialect
or moving it to the arithmetic dialect would require those dialects to have a
dependency on the memref dialect, which breaks linking).

As the AtomicRMWKind enum is used throughout, this has been moved to Arith.

Reviewed By: Mogball

Differential Revision: https://reviews.llvm.org/D116392

show more ...

Fix more clang-tidy cleanups in mlir/ (NFC)

Adjust "end namespace" comment in MLIR to match new agree'd coding style

See D115115 and this mailing list discussion:
https://lists.llvm.org/pipermail/llvm-dev/2021-December/154199.html

Differenti

Adjust "end namespace" comment in MLIR to match new agree'd coding style

See D115115 and this mailing list discussion:
https://lists.llvm.org/pipermail/llvm-dev/2021-December/154199.html

Differential Revision: https://reviews.llvm.org/D115309

show more ...

[mlir] Switch arith, llvm, std & shape dialects to accessors prefixed both form.

Following
https://llvm.discourse.group/t/psa-ods-generated-accessors-will-change-to-have-a-get-prefix-update-you-apis

[mlir] Switch arith, llvm, std & shape dialects to accessors prefixed both form.

Following
https://llvm.discourse.group/t/psa-ods-generated-accessors-will-change-to-have-a-get-prefix-update-you-apis/4476,
this follows flipping these dialects to _Both prefixed form. This
changes the accessors to have a prefix. This was possibly mostly without
breaking breaking changes if the existing convenience methods were used.

(https://github.com/jpienaar/llvm-project/blob/main/clang-tools-extra/clang-tidy/misc/AddGetterCheck.cpp
was used to migrate the callers post flipping, using the output from
Operator.cpp)

Differential Revision: https://reviews.llvm.org/D112383

show more ...

[MLIR] Replace std ops with arith dialect ops

Precursor: https://reviews.llvm.org/D110200

Removed redundant ops from the standard dialect that were moved to the
`arith` or `math` dialects.

Renamed

[MLIR] Replace std ops with arith dialect ops

Precursor: https://reviews.llvm.org/D110200

Removed redundant ops from the standard dialect that were moved to the
`arith` or `math` dialects.

Renamed all instances of operations in the codebase and in tests.

Reviewed By: rriddle, jpienaar

Differential Revision: https://reviews.llvm.org/D110797

show more ...

[mlir] Vectorize induction variables

1. Add support to vectorize induction variables of loops that are
not mapped to any vector dimension in SuperVectorize pass.
2. Fix a bug in getForInductionVa

[mlir] Vectorize induction variables

1. Add support to vectorize induction variables of loops that are
not mapped to any vector dimension in SuperVectorize pass.
2. Fix a bug in getForInductionVarOwner.

Reviewed By: dcaballe

Differential Revision: https://reviews.llvm.org/D111370

show more ...

[mlir] Fix typo in SuperVectorizer

NFC.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D108334

[mlir] Set insertion point of vector constant to the top of the vectorized loop body

When we vectorize a scalar constant, the vector constant is inserted before its
first user if the scalar constant

[mlir] Set insertion point of vector constant to the top of the vectorized loop body

When we vectorize a scalar constant, the vector constant is inserted before its
first user if the scalar constant is defined outside the loops to be vectorized.
It is possible that the vector constant does not dominate all its users. To fix
the problem, we find the innermost vectorized loop that encloses that first user
and insert the vector constant at the top of the loop body.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D106609

show more ...

[mlir][Affine][Vector] Support vectorizing reduction loops

This patch adds support for vectorizing loops with 'iter_args'
implementing known reductions along the vector dimension. Comparing to
the n

[mlir][Affine][Vector] Support vectorizing reduction loops

This patch adds support for vectorizing loops with 'iter_args'
implementing known reductions along the vector dimension. Comparing to
the non-vector-dimension case, two additional things are done during
vectorization of such loops:
- The resulting vector returned from the loop is reduced to a scalar
using `vector.reduce`.
- In some cases a mask is applied to the vector yielded at the end of
the loop to prevent garbage values from being written to the
accumulator.

Vectorization of reduction loops is disabled by default. To enable it, a
map from loops to array of reduction descriptors should be explicitly passed to
`vectorizeAffineLoops`, or `vectorize-reductions=true` should be passed
to the SuperVectorize pass.

Current limitations:
- Loops with a non-unit step size are not supported.
- n-D vectorization with n > 1 is not supported.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D100694

show more ...

[mlir] Affine: parallelize affine loops with reductions

Introduce a basic support for parallelizing affine loops with reductions
expressed using iteration arguments. Affine parallelism detector now

[mlir] Affine: parallelize affine loops with reductions

Introduce a basic support for parallelizing affine loops with reductions
expressed using iteration arguments. Affine parallelism detector now has a flag
to assume such reductions are parallel. The transformation handles a subset of
parallel reductions that are can be expressed using affine.parallel:
integer/float addition and multiplication. This requires to detect the
reduction operation since affine.parallel only supports a fixed set of
reduction operators.

Reviewed By: chelini, kumasento, bondhugula

Differential Revision: https://reviews.llvm.org/D101171

show more ...

[mlir] yet more iwyu fixes after ba7a92c01e8