[mlir:PDL] Fix bugs in PDLPatternModule merging

* Constraints/Rewrites registered before a pattern was added were dropped
* Constraints/Rewrites may be registered multiple times (if different patter

[mlir:PDL] Fix bugs in PDLPatternModule merging

* Constraints/Rewrites registered before a pattern was added were dropped
* Constraints/Rewrites may be registered multiple times (if different pattern sets depend on them)
* ModuleOp no longer has a terminator, so we shouldn't be removing the terminator from it

Differential Revision: https://reviews.llvm.org/D114816

show more ...

Introduced iterative bytecode execution.

This is commit 2 of 4 for the multi-root matching in PDL, discussed in https://llvm.discourse.group/t/rfc-multi-root-pdl-patterns-for-kernel-matching/4148 (t

Introduced iterative bytecode execution.

This is commit 2 of 4 for the multi-root matching in PDL, discussed in https://llvm.discourse.group/t/rfc-multi-root-pdl-patterns-for-kernel-matching/4148 (topic flagged for review).

This commit implements the features needed for the execution of the new operations pdl_interp.get_accepting_ops, pdl_interp.choose_op:
1. The implementation of the generation and execution of the two ops.
2. The addition of Stack of bytecode positions within the ByteCodeExecutor. This is needed because in pdl_interp.choose_op, we iterate over the values returned by pdl_interp.get_accepting_ops until we reach finalize. When we reach finalize, we need to return back to the position marked in the stack.
3. The functionality to extend the lifetime of values that cross the nondeterministic choice. The existing bytecode generator allocates the values to memory positions by representing the liveness of values as a collection of disjoint intervals over the matcher positions. This is akin to register allocation, and substantially reduces the footprint of the bytecode executor. However, because with iterative operation pdl_interp.choose_op, execution "returns" back, so any values whose original liveness cross the nondeterminstic choice must have their lifetime executed until finalize.

Testing: pdl-bytecode.mlir test

Reviewed By: rriddle, Mogball

Differential Revision: https://reviews.llvm.org/D108547

show more ...

[mlir][Pattern] Add better support for using interfaces/traits to match root operations in rewrite patterns

To match an interface or trait, users currently have to use the `MatchAny` tag. This tag c

[mlir][Pattern] Add better support for using interfaces/traits to match root operations in rewrite patterns

To match an interface or trait, users currently have to use the `MatchAny` tag. This tag can be quite problematic for compile time for things like the canonicalizer, as the `MatchAny` patterns may get applied to *every* operation. This revision adds better support by bucketing interface/trait patterns based on which registered operations have them registered. This means that moving forward we will only attempt to match these patterns to operations that have this interface registered. Two simplify defining patterns that match traits and interfaces, two new utility classes have been added: OpTraitRewritePattern and OpInterfaceRewritePattern.

Differential Revision: https://reviews.llvm.org/D98986

show more ...

[mlir][PDL] Add support for variadic operands and results in the PDL byte code

Supporting ranges in the byte code requires additional complexity, given that a range can't be easily representable as

[mlir][PDL] Add support for variadic operands and results in the PDL byte code

Supporting ranges in the byte code requires additional complexity, given that a range can't be easily representable as an opaque void *, as is possible with the existing bytecode value types (Attribute, Type, Value, etc.). To enable representing a range with void *, an auxillary storage is used for the actual range itself, with the pointer being passed around in the normal byte code memory. For type ranges, a TypeRange is stored. For value ranges, a ValueRange is stored. The above problem represents a majority of the complexity involved in this revision, the rest is adapting/adding byte code operations to support the changes made to the PDL interpreter in the parent revision.

After this revision, PDL will have initial end-to-end support for variadic operands/results.

Differential Revision: https://reviews.llvm.org/D95723

show more ...

[mlir][pdl] Remove CreateNativeOp in favor of a more general ApplyNativeRewriteOp.

This has a numerous amount of benefits, given the overly clunky nature of CreateNativeOp:
* Users can now call into

[mlir][pdl] Remove CreateNativeOp in favor of a more general ApplyNativeRewriteOp.

This has a numerous amount of benefits, given the overly clunky nature of CreateNativeOp:
* Users can now call into arbitrary rewrite functions from inside of PDL, allowing for more natural interleaving of PDL/C++ and enabling for more of the pattern to be in PDL.
* Removes the need for an additional set of C++ functions/registry/etc. The new ApplyNativeRewriteOp will use the same PDLRewriteFunction as the existing RewriteOp. This reduces the API surface area exposed to users.

This revision also introduces a new PDLResultList class. This class is used to provide results of native rewrite functions back to PDL. We introduce a new class instead of using a SmallVector to simplify the work necessary for variadics, given that ranges will require some changes to the structure of PDLValue.

Differential Revision: https://reviews.llvm.org/D95720

show more ...

[mlir][Pattern] Create a new IRRewriter class to enable sharing code with pattern rewrites

This revision adds two new classes, RewriterBase and IRRewriter. RewriterBase is a new shared base class be

[mlir][Pattern] Create a new IRRewriter class to enable sharing code with pattern rewrites

This revision adds two new classes, RewriterBase and IRRewriter. RewriterBase is a new shared base class between IRRewriter and PatternRewriter. PatternRewriter will continue to be the base class used to perform rewrites within a rewrite pattern. IRRewriter on the other hand, is a new class that allows for tracking IR rewrites from outside of a rewrite pattern. In this revision all of the old API from PatternRewriter is moved to RewriterBase, but the distinction between IRRewriter and PatternRewriter is kept on the chance that a necessary API divergence happens in the future.

Currently if you want to have some utility that transforms a piece of IR and share it between pattern and non-pattern code, you have to duplicate it. This revision enables the creation of utilities that can be invoked from rewrite patterns and normal transformation code:

```c++
void someSharedUtility(RewriterBase &rewriter, ...) {
// Some interesting IR mutation here.
}

// Some RewritePattern
LogicalResult MyPattern::matchAndRewrite(Operation *op, PatternRewriter &rewriter) {
...
someSharedUtility(rewriter, ...);
...
}

// Some Pass
void MyPass::runOnOperation() {
...
IRRewriter rewriter(...);
someSharedUtility(rewriter, ...);
}
```

Differential Revision: https://reviews.llvm.org/D94638

show more ...

[mlir][PatternRewriter] Add a new hook to selectively replace uses of an operation

This revision adds a new `replaceOpWithIf` hook that replaces uses of an operation that satisfy a given functor. If

[mlir][PatternRewriter] Add a new hook to selectively replace uses of an operation

This revision adds a new `replaceOpWithIf` hook that replaces uses of an operation that satisfy a given functor. If all uses are replaced, the operation gets erased in a similar manner to `replaceOp`. DialectConversion support will be added in a followup as this requires adjusting how replacements are tracked there.

Differential Revision: https://reviews.llvm.org/D94632

show more ...

[mlir][PDL] Add support for PDL bytecode and expose PDL support to OwningRewritePatternList

PDL patterns are now supported via a new `PDLPatternModule` class. This class contains a ModuleOp with the

[mlir][PDL] Add support for PDL bytecode and expose PDL support to OwningRewritePatternList

PDL patterns are now supported via a new `PDLPatternModule` class. This class contains a ModuleOp with the pdl::PatternOp operations representing the patterns, as well as a collection of registered C++ functions for native constraints/creations/rewrites/etc. that may be invoked via the pdl patterns. Instances of this class are added to an OwningRewritePatternList in the same fashion as C++ RewritePatterns, i.e. via the `insert` method.

The PDL bytecode is an in-memory representation of the PDL interpreter dialect that can be efficiently interpreted/executed. The representation of the bytecode boils down to a code array(for opcodes/memory locations/etc) and a memory buffer(for storing attributes/operations/values/any other data necessary). The bytecode operations are effectively a 1-1 mapping to the PDLInterp dialect operations, with a few exceptions in cases where the in-memory representation of the bytecode can be more efficient than the MLIR representation. For example, a generic `AreEqual` bytecode op can be used to represent AreEqualOp, CheckAttributeOp, and CheckTypeOp.

The execution of the bytecode is split into two phases: matching and rewriting. When matching, all of the matched patterns are collected to avoid the overhead of re-running parts of the matcher. These matched patterns are then considered alongside the native C++ patterns, which rewrite immediately in-place via `RewritePattern::matchAndRewrite`, for the given root operation. When a PDL pattern is matched and has the highest benefit, it is passed back to the bytecode to execute its rewriter.

Differential Revision: https://reviews.llvm.org/D89107

show more ...

[mlir] NFC: fix trivial typos
fix typos in comments and documents

Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D90089

[mlir][NFC] Move around the code related to PatternRewriting to improve layering

There are several pieces of pattern rewriting infra in IR/ that really shouldn't be there. This revision moves those

[mlir][NFC] Move around the code related to PatternRewriting to improve layering

There are several pieces of pattern rewriting infra in IR/ that really shouldn't be there. This revision moves those pieces to a better location such that they are easier to evolve in the future(e.g. with PDL). More concretely this revision does the following:

* Create a Transforms/GreedyPatternRewriteDriver.h and move the apply*andFold methods there.
The definitions for these methods are already in Transforms/ so it doesn't make sense for the declarations to be in IR.

* Create a new lib/Rewrite library and move PatternApplicator there.
This new library will be focused on applying rewrites, and will also include compiling rewrites with PDL.

Differential Revision: https://reviews.llvm.org/D89103

show more ...

[mlir][Pattern] Refactor the Pattern class into a "metadata only" class

The Pattern class was originally intended to be used for solely matching operations, but that use never materialized. All of t

[mlir][Pattern] Refactor the Pattern class into a "metadata only" class

The Pattern class was originally intended to be used for solely matching operations, but that use never materialized. All of the pattern infrastructure uses RewritePattern, and the infrastructure for pure matching(Matchers.h) is implemented inline. This means that this class isn't a useful abstraction at the moment, so this revision refactors it to solely encapsulate the "metadata" of a pattern. The metadata includes the various state describing a pattern; benefit, root operation, etc. The API on PatternApplicator is updated to now operate on `Pattern`s as nothing special from `RewritePattern` is necessary.

This refactoring is also necessary for the upcoming use of PDL patterns alongside C++ rewrite patterns.

Differential Revision: https://reviews.llvm.org/D86258

show more ...

[MLIR] Add debug support for ignored patterns

The rewrite engine's cost model may determine some patterns to be irrelevant
ahead of their application. These patterns were silently ignored previously

[MLIR] Add debug support for ignored patterns

The rewrite engine's cost model may determine some patterns to be irrelevant
ahead of their application. These patterns were silently ignored previously and
now cause a message in `--debug` mode.

Differential Revision: https://reviews.llvm.org/D87290

show more ...

[MLIR] Add PatternRewriter::mergeBlockBefore() to merge a block in the middle of another block.

- This utility to merge a block anywhere into another one can help inline single
block regions into

[MLIR] Add PatternRewriter::mergeBlockBefore() to merge a block in the middle of another block.

- This utility to merge a block anywhere into another one can help inline single
block regions into other blocks.
- Modified patterns test to use the new function.

Differential Revision: https://reviews.llvm.org/D86251

show more ...

[mlir] Allow for patterns to match any root kind.

Traditionally patterns have always had the root operation kind hardcoded to a specific operation name. This has worked well for quite some time, but

[mlir] Allow for patterns to match any root kind.

Traditionally patterns have always had the root operation kind hardcoded to a specific operation name. This has worked well for quite some time, but it has certain limitations that make it undesirable. For example, some lowering have the same implementation for many different operations types with a few lowering entire dialects using the same pattern implementation. This problem has led to several "solutions":
a) Provide a template implementation to the user so that they can instantiate it for each operation combination, generally requiring the inclusion of the auto-generated operation definition file.
b) Use a non-templated pattern that allows for providing the name of the operation to match
- No one ever does this, because enumerating operation names can be cumbersome and so this quickly devolves into solution a.

This revision removes the restriction that patterns have a hardcoded root type, and allows for a class patterns that could match "any" operation type. The major downside of root-agnostic patterns is that they make certain pattern analyses more difficult, so it is still very highly encouraged that an operation specific pattern be used whenever possible.

Differential Revision: https://reviews.llvm.org/D82066

show more ...

[mlir] Refactor RewritePatternMatcher into a new PatternApplicator class.

This class enables for abstracting more of the details for the rewrite process, and will allow for clients to apply specific

[mlir] Refactor RewritePatternMatcher into a new PatternApplicator class.

This class enables for abstracting more of the details for the rewrite process, and will allow for clients to apply specific cost models to the pattern list. This allows for DialectConversion and the GreedyPatternRewriter to share the same underlying matcher implementation. This also simplifies the plumbing necessary to support dynamic patterns.

Differential Revision: https://reviews.llvm.org/D81985

show more ...

[MLIR] Add pattern rewriter util to erase block; remove dead else

Add a pattern rewriter utility to erase blocks (while notifying the
pattern rewriting driver of the erased ops). Use this to remove

[MLIR] Add pattern rewriter util to erase block; remove dead else

Add a pattern rewriter utility to erase blocks (while notifying the
pattern rewriting driver of the erased ops). Use this to remove trivial
else blocks in affine.if ops.

Differential Revision: https://reviews.llvm.org/D77083

show more ...

[mlir][NFC] Replace all usages of PatternMatchResult with LogicalResult

This also replaces usages of matchSuccess/matchFailure with success/failure respectively.

Differential Revision: https://revi

[mlir][NFC] Replace all usages of PatternMatchResult with LogicalResult

This also replaces usages of matchSuccess/matchFailure with success/failure respectively.

Differential Revision: https://reviews.llvm.org/D76313

show more ...

[mlir] Remove the PatternState class and simplify PatternMatchResult.

Summary: PatternState was a mechanism to pass state between the match and rewrite calls of a RewritePattern. With the rise of ma

[mlir] Remove the PatternState class and simplify PatternMatchResult.

Summary: PatternState was a mechanism to pass state between the match and rewrite calls of a RewritePattern. With the rise of matchAndRewrite, this class is unused and unnecessary. This revision removes PatternState and simplifies PatternMatchResult to just be a LogicalResult. A future revision will replace all usages of PatternMatchResult/matchSuccess/matchFailure with LogicalResult equivalents.

Differential Revision: https://reviews.llvm.org/D76202

show more ...

[mlir] Remove 'valuesToRemoveIfDead' from PatternRewriter API

Summary:
Remove 'valuesToRemoveIfDead' from PatternRewriter API. The removal
functionality wasn't implemented and we decided [1] not to

[mlir] Remove 'valuesToRemoveIfDead' from PatternRewriter API

Summary:
Remove 'valuesToRemoveIfDead' from PatternRewriter API. The removal
functionality wasn't implemented and we decided [1] not to implement it in
favor of having more powerful DCE approaches.

[1] https://github.com/tensorflow/mlir/pull/212

Reviewers: rriddle, bondhugula

Reviewed By: rriddle

Subscribers: liufengdb, mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, nicolasvasilache, arpith-jacob, mgester, lucyrfox, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D72545

show more ...

Mass update the MLIR license header to mention "Part of the LLVM project"

This is an artifact from merging MLIR into LLVM, the file headers are
now aligned with the rest of the project.

[mlir] NFC: Remove Value::operator* and Value::operator-> now that Value is properly value-typed.

Summary: These were temporary methods used to simplify the transition.

Reviewed By: antiagainst

Di

[mlir] NFC: Remove Value::operator* and Value::operator-> now that Value is properly value-typed.

Summary: These were temporary methods used to simplify the transition.

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D72548

show more ...

Change the `notifyRootUpdated` API to be transaction based.

This means that in-place, or root, updates need to use explicit calls to `startRootUpdate`, `finalizeRootUpdate`, and `cancelRootUpdate`.

Change the `notifyRootUpdated` API to be transaction based.

This means that in-place, or root, updates need to use explicit calls to `startRootUpdate`, `finalizeRootUpdate`, and `cancelRootUpdate`. The major benefit of this change is that it enables in-place updates in DialectConversion, which simplifies the FuncOp pattern for example. The major downside to this is that the cases that *may* modify an operation in-place will need an explicit cancel on the failure branches(assuming that they started an update before attempting the transformation).

PiperOrigin-RevId: 286933674

show more ...

Adjust License.txt file to use the LLVM license

PiperOrigin-RevId: 286906740

Add a new ValueRange class.

This class represents a generic abstraction over the different ways to represent a range of Values: ArrayRef<Value *>, operand_range, result_range. This class will allow

Add a new ValueRange class.

This class represents a generic abstraction over the different ways to represent a range of Values: ArrayRef<Value *>, operand_range, result_range. This class will allow for removing the many instances of explicit SmallVector<Value *, N> construction. It has the same memory cost as ArrayRef, and only suffers cost from indexing(if+elsing the different underlying representations).

This change only updates a few of the existing usages, with more to be changed in followups; e.g. 'build' API.

PiperOrigin-RevId: 284307996

show more ...

Add a PatternRewriter hook to merge blocks, and use it to support for folding branches.

A pattern rewriter hook, mergeBlock, is added that allows for merging the operations of one block into the end

Add a PatternRewriter hook to merge blocks, and use it to support for folding branches.

A pattern rewriter hook, mergeBlock, is added that allows for merging the operations of one block into the end of another. This is used to support a canonicalization pattern for branch operations that folds the branch when the successor has a single predecessor(the branch block).

Example:
^bb0:
%c0_i32 = constant 0 : i32
br ^bb1(%c0_i32 : i32)
^bb1(%x : i32):
return %x : i32

becomes:
^bb0:
%c0_i32 = constant 0 : i32
return %c0_i32 : i32
PiperOrigin-RevId: 278677825

show more ...