[mlir] (NFC) run clang-format on all files

[mlir][NFC] Add a using for llvm::BitVector to LLVM.h

BitVector is becoming widespread enough that we should add a proper using.

Differential Revision: https://reviews.llvm.org/D118290

[mlir] Make locations required when adding/creating block arguments

BlockArguments gained the ability to have locations attached a while ago, but they
have always been optional. This goes against th

[mlir] Make locations required when adding/creating block arguments

BlockArguments gained the ability to have locations attached a while ago, but they
have always been optional. This goes against the core tenant of MLIR where location
information is a requirement, so this commit updates the API to require locations.

Fixes #53279

Differential Revision: https://reviews.llvm.org/D117633

show more ...

Fix clang-tidy issues in mlir/ (NFC)

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D115956

Use early exist and simplify a condition in Block SuccessorRange (NFC)

Fix verifier crashing on some invalid IR

In a region with multiple blocks the verifier will try to look for
dominance and may get successor list for blocks, even though a block
may be empty or does

Fix verifier crashing on some invalid IR

In a region with multiple blocks the verifier will try to look for
dominance and may get successor list for blocks, even though a block
may be empty or does not end with a terminator.

Differential Revision: https://reviews.llvm.org/D104411

show more ...

[IR] Add a Location to BlockArgument

This adds the ability to specify a location when creating BlockArguments.
Notably Value::getLoc() will return this correctly, which makes diagnostics
more precis

[IR] Add a Location to BlockArgument

This adds the ability to specify a location when creating BlockArguments.
Notably Value::getLoc() will return this correctly, which makes diagnostics
more precise (e.g. the example in test-legalize-type-conversion.mlir).

This is currently optional to avoid breaking any existing code - if
absent, the BlockArgument defaults to using the location of its enclosing
operation (preserving existing behavior).

The bulk of this change is plumbing location tracking through the parser
and printer to make sure it can round trip (in -mlir-print-debuginfo
mode). This is complete for generic operations, but requires manual
adoption for custom ops.

I added support for function-like ops to round trip their argument
locations - they print correctly, but when parsing the locations are
dropped on the floor. I intend to fix this, but it will require more
invasive plumbing through "function_like_impl" stuff so I think it
best to split it out to its own patch.

This is a reapply of the patch here: https://reviews.llvm.org/D102567
with an additional change: we now never defer block argument locations,
guaranteeing that we can round trip correctly.

This isn't required in all cases, but allows us to hill climb here and
works around unrelated bugs like https://bugs.llvm.org/show_bug.cgi?id=50451

Differential Revision: https://reviews.llvm.org/D102991

show more ...

Revert "[IR] Add a Location to BlockArgument." and follow-on commit
"[mlir] Speed up Lexer::getEncodedSourceLocation"

This reverts commit 3043be9d2db4d0cdf079adb5e1bdff032405e941 and commit
861d69a5

Revert "[IR] Add a Location to BlockArgument." and follow-on commit
"[mlir] Speed up Lexer::getEncodedSourceLocation"

This reverts commit 3043be9d2db4d0cdf079adb5e1bdff032405e941 and commit
861d69a5259653f60d59795597493a7939b794fe.

This change resulted in printing textual MLIR that can't be parsed; see
review thread https://reviews.llvm.org/D102567 for details.

show more ...

[IR] Add a Location to BlockArgument.

This adds the ability to specify a location when creating BlockArguments.
Notably Value::getLoc() will return this correctly, which makes diagnostics
more preci

[IR] Add a Location to BlockArgument.

This adds the ability to specify a location when creating BlockArguments.
Notably Value::getLoc() will return this correctly, which makes diagnostics
more precise (e.g. the example in test-legalize-type-conversion.mlir).

This is currently optional to avoid breaking any existing code - if
absent, the BlockArgument defaults to using the location of its enclosing
operation (preserving existing behavior).

The bulk of this change is plumbing location tracking through the parser
and printer to make sure it can round trip (in -mlir-print-debuginfo
mode). This is complete for generic operations, but requires manual
adoption for custom ops.

I added support for function-like ops to round trip their argument
locations - they print correctly, but when parsing the locations are
dropped on the floor. I intend to fix this, but it will require more
invasive plumbing through "function_like_impl" stuff so I think it
best to split it out to its own patch.

Differential Revision: https://reviews.llvm.org/D102567

show more ...

Remove unused function, fix warning (NFC)

The `mayNotHaveTerminator` was initially on Block but moved to the
verifier before landing and wasn't removed from its original place
where it is unused.

Define a `NoTerminator` traits that allows operations with a single block region to not provide a terminator

In particular for Graph Regions, the terminator needs is just a
historical artifact of th

Define a `NoTerminator` traits that allows operations with a single block region to not provide a terminator

In particular for Graph Regions, the terminator needs is just a
historical artifact of the generalization of MLIR from CFG region.
Operations like Module don't need a terminator, and before Module
migrated to be an operation with region there wasn't any needed.

To validate the feature, the ModuleOp is migrated to use this trait and
the ModuleTerminator operation is deleted.

This patch is likely to break clients, if you're in this case:

- you may iterate on a ModuleOp with `getBody()->without_terminator()`,
the solution is simple: just remove the ->without_terminator!
- you created a builder with `Builder::atBlockTerminator(module_body)`,
just use `Builder::atBlockEnd(module_body)` instead.
- you were handling ModuleTerminator: it isn't needed anymore.
- for generic code, a `Block::mayNotHaveTerminator()` may be used.

Differential Revision: https://reviews.llvm.org/D98468

show more ...

[mlir] Optimize the implementation of RegionDCE

The current implementation has some inefficiencies that become noticeable when running on large modules. This revision optimizes the code, and updates

[mlir] Optimize the implementation of RegionDCE

The current implementation has some inefficiencies that become noticeable when running on large modules. This revision optimizes the code, and updates some out-dated idioms with newer utilities. The main components of this optimization include:

* Add an overload of Block::eraseArguments that allows for O(N) erasure of disjoint arguments.
* Don't process entry block arguments given that we don't erase them at this point.
* Don't track individual operation results, given that we don't erase them. We can just track the parent operation.

Differential Revision: https://reviews.llvm.org/D98309

show more ...

Fix Block::eraseArguments: keep track the first removed element while removing

Not only this is likely more efficient than BitVector::find_first(), but
also if the BitVector is empty find_first() re

Fix Block::eraseArguments: keep track the first removed element while removing

Not only this is likely more efficient than BitVector::find_first(), but
also if the BitVector is empty find_first() returns -1, which
llvm::drop_begin isn't robust against.

show more ...

Fix Block::eraseArguments to properly update the cached positions

This is fixing correctness and ASAN failure post-ee90bb3486948.

Store (cache) the Argument number (index in the argument list) inside the BlockArgumentImpl

This avoids linear search in BlockArgument::getArgNumber().

Differential Revision: https://reviews.llvm.o

Store (cache) the Argument number (index in the argument list) inside the BlockArgumentImpl

This avoids linear search in BlockArgument::getArgNumber().

Differential Revision: https://reviews.llvm.org/D97596

show more ...

[mlir][IR] Remove the concept of `OperationProperties`

These properties were useful for a few things before traits had a better integration story, but don't really carry their weight well these days

[mlir][IR] Remove the concept of `OperationProperties`

These properties were useful for a few things before traits had a better integration story, but don't really carry their weight well these days. Most of these properties are already checked via traits in most of the code. It is better to align the system around traits, and improve the performance/cost of traits in general.

Differential Revision: https://reviews.llvm.org/D96088

show more ...

[mlir][PDL] Add support for PDL bytecode and expose PDL support to OwningRewritePatternList

PDL patterns are now supported via a new `PDLPatternModule` class. This class contains a ModuleOp with the

[mlir][PDL] Add support for PDL bytecode and expose PDL support to OwningRewritePatternList

PDL patterns are now supported via a new `PDLPatternModule` class. This class contains a ModuleOp with the pdl::PatternOp operations representing the patterns, as well as a collection of registered C++ functions for native constraints/creations/rewrites/etc. that may be invoked via the pdl patterns. Instances of this class are added to an OwningRewritePatternList in the same fashion as C++ RewritePatterns, i.e. via the `insert` method.

The PDL bytecode is an in-memory representation of the PDL interpreter dialect that can be efficiently interpreted/executed. The representation of the bytecode boils down to a code array(for opcodes/memory locations/etc) and a memory buffer(for storing attributes/operations/values/any other data necessary). The bytecode operations are effectively a 1-1 mapping to the PDLInterp dialect operations, with a few exceptions in cases where the in-memory representation of the bytecode can be more efficient than the MLIR representation. For example, a generic `AreEqual` bytecode op can be used to represent AreEqualOp, CheckAttributeOp, and CheckTypeOp.

The execution of the bytecode is split into two phases: matching and rewriting. When matching, all of the matched patterns are collected to avoid the overhead of re-running parts of the matcher. These matched patterns are then considered alongside the native C++ patterns, which rewrite immediately in-place via `RewritePattern::matchAndRewrite`, for the given root operation. When a PDL pattern is matched and has the highest benefit, it is passed back to the bytecode to execute its rewriter.

Differential Revision: https://reviews.llvm.org/D89107

show more ...

[MLIR] Move eraseArguments and eraseResults to FunctionLike

Previously, they were only defined for `FuncOp`.

To support this, `FunctionLike` needs a way to get an updated type
from the concrete ope

[MLIR] Move eraseArguments and eraseResults to FunctionLike

Previously, they were only defined for `FuncOp`.

To support this, `FunctionLike` needs a way to get an updated type
from the concrete operation. This adds a new hook for that purpose,
called `getTypeWithoutArgsAndResults`.

For now, `FunctionLike` continues to assume the type is
`FunctionType`, and concrete operations that use another type can hide
the `getType`, `setType`, and `getTypeWithoutArgsAndResults` methods.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D90363

show more ...

[MLIR] Change Operation::create() methods to use Value/Type/Block ranges.

- Introduce a new BlockRange class to represent range of blocks (constructible from
an ArrayRef<Block *> or a SuccessorRan

[MLIR] Change Operation::create() methods to use Value/Type/Block ranges.

- Introduce a new BlockRange class to represent range of blocks (constructible from
an ArrayRef<Block *> or a SuccessorRange);
- Change Operation::create() methods to use TypeRange for result types, ValueRange for
operands and BlockRange for successors.

Differential Revision: https://reviews.llvm.org/D86985

show more ...

[MLIR] Add PatternRewriter::mergeBlockBefore() to merge a block in the middle of another block.

- This utility to merge a block anywhere into another one can help inline single
block regions into

[MLIR] Add PatternRewriter::mergeBlockBefore() to merge a block in the middle of another block.

- This utility to merge a block anywhere into another one can help inline single
block regions into other blocks.
- Modified patterns test to use the new function.

Differential Revision: https://reviews.llvm.org/D86251

show more ...

[mlir][Standard] Add a canonicalization to simplify cond_br when the successors are identical

This revision adds support for canonicalizing the following:

```
cond_br %cond, ^bb1(A, ..., N), ^bb1(A

[mlir][Standard] Add a canonicalization to simplify cond_br when the successors are identical

This revision adds support for canonicalizing the following:

```
cond_br %cond, ^bb1(A, ..., N), ^bb1(A, ..., N)

br ^bb1(A, ..., N)
```

If the operands to the successor are different and the cond_br is the only predecessor, we emit selects for the branch operands.

```
cond_br %cond, ^bb1(A), ^bb1(B)

%select = select %cond, A, B
br ^bb1(%select)
```

Differential Revision: https://reviews.llvm.org/D78682

show more ...

[mlir] Refactor a few users to no longer rely on the successor operand API of Operation.

The existing API for successor operands on operations is in the process of being removed. This revision simpl

[mlir] Refactor a few users to no longer rely on the successor operand API of Operation.

The existing API for successor operands on operations is in the process of being removed. This revision simplifies a later one that completely removes the existing API.

Differential Revision: https://reviews.llvm.org/D75316

show more ...

Fix Block::eraseArgument when block arg is also a successor operand.

Summary:
This could trigger an assertion due to the block argument being used by
this block's own successor operands.

Reviewers:

Fix Block::eraseArgument when block arg is also a successor operand.

Summary:
This could trigger an assertion due to the block argument being used by
this block's own successor operands.

Reviewers: rriddle!

Subscribers: mehdi_amini, jpienaar, burmako, shauheen, antiagainst, nicolasvasilache, arpith-jacob, mgester, lucyrfox, liufengdb, Joonsoo, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D74583

show more ...

Mass update the MLIR license header to mention "Part of the LLVM project"

This is an artifact from merging MLIR into LLVM, the file headers are
now aligned with the rest of the project.

[mlir] Add in-dialect lowering of gpu.all_reduce.

Reviewers: ftynse, nicolasvasilache, herhut

Reviewed By: ftynse, herhut

Subscribers: liufengdb, aartbik, herhut, merge_guards_bot, mgorny, mehdi_a

[mlir] Add in-dialect lowering of gpu.all_reduce.

Reviewers: ftynse, nicolasvasilache, herhut

Reviewed By: ftynse, herhut

Subscribers: liufengdb, aartbik, herhut, merge_guards_bot, mgorny, mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, nicolasvasilache, arpith-jacob, mgester, lucyrfox, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D72129

show more ...