[mlir][gpu] Move GPU headers into IR/ and Transforms/

Depends on D127350

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D127352

[mlir] Don't iterate mutable user list

executeOp.operandsMutable().append(asyncTokens) in
addAsyncDependencyAfter can resize and invalidate iterators.

Fixes reports like https://reviews.llvm.org/P8

[mlir] Don't iterate mutable user list

executeOp.operandsMutable().append(asyncTokens) in
addAsyncDependencyAfter can resize and invalidate iterators.

Fixes reports like https://reviews.llvm.org/P8286

Reviewed By: herhut

Differential Revision: https://reviews.llvm.org/D124577

show more ...

[mlir:NFC] Remove the forward declaration of FuncOp in the mlir namespace

FuncOp has been moved to the `func` namespace for a little over a month, the
using directive can be dropped now.

[mlir] Remove a bunch of unnecessary dialect dependencies

A lot of dialects have dependencies that are unnecessary, either because of copy/paste
of files when creating things or some other means. Th

[mlir] Remove a bunch of unnecessary dialect dependencies

A lot of dialects have dependencies that are unnecessary, either because of copy/paste
of files when creating things or some other means. This commit cleans up a bunch of
the simple ones:

* Copy/Paste or missed during refactoring
Most of the dependencies cleaned up here look like copy/paste errors when creating
new dialects/transformations, or because the dependency wasn't removed during a
refactoring (e.g. when splitting the standard dialect).

* Unnecessary hard coding of constant operations in matchers
There are a few instances where a dialect had a dependency because it
was hardcoding checks for constant operations instead of using the better m_Constant
approach.

Differential Revision: https://reviews.llvm.org/D118062

show more ...

[mlir] Make locations required when adding/creating block arguments

BlockArguments gained the ability to have locations attached a while ago, but they
have always been optional. This goes against th

[mlir] Make locations required when adding/creating block arguments

BlockArguments gained the ability to have locations attached a while ago, but they
have always been optional. This goes against the core tenant of MLIR where location
information is a requirement, so this commit updates the API to require locations.

Fixes #53279

Differential Revision: https://reviews.llvm.org/D117633

show more ...

[mlir][Pass] Deprecate FunctionPass in favor of OperationPass<FuncOp>

The only benefit of FunctionPass is that it filters out function
declarations. This isn't enough to justify carrying it around,

[mlir][Pass] Deprecate FunctionPass in favor of OperationPass<FuncOp>

The only benefit of FunctionPass is that it filters out function
declarations. This isn't enough to justify carrying it around, as we can
simplify filter out declarations when necessary within the pass. We can
also explore with better scheduling primitives to filter out declarations
at the pipeline level in the future.

The definition of FunctionPass is left intact for now to allow time for downstream
users to migrate.

Differential Revision: https://reviews.llvm.org/D117182

show more ...

[mlir] Support pre-existing tokens in 'gpu-async-region'

Allow gpu ops implementing the async interface to already be async when running the GpuAsyncRegionPass.
That pass threads a 'current token' t

[mlir] Support pre-existing tokens in 'gpu-async-region'

Allow gpu ops implementing the async interface to already be async when running the GpuAsyncRegionPass.
That pass threads a 'current token' through a block with ops implementing the gpu async interface.

After this change, existing async ops (returning a !gpu.async.token) set the current token.
Existing synchronous `gpu.wait` ops reset the current token.

Reviewed By: herhut

Differential Revision: https://reviews.llvm.org/D103396

show more ...

[mlir] Add support for ops with regions in 'gpu-async-region' rewriter.

Reviewed By: herhut

Differential Revision: https://reviews.llvm.org/D101757

Pass GPU events instead of streams across async regions.

Lower !gpu.async.tokens returned from async.execute regions to events instead of streams.

Make !gpu.async.token returned from !async.execute

Pass GPU events instead of streams across async regions.

Lower !gpu.async.tokens returned from async.execute regions to events instead of streams.

Make !gpu.async.token returned from !async.execute single-use.
This allows creating one event per use and destroying them without leaking or ref-counting.
Technically we only need this for stream/event-based lowering. I kept the code separate
from the rest of the gpu-async-region pass so that we can make this optional or move
to a separate pass as needed.

Reviewed By: herhut

Differential Revision: https://reviews.llvm.org/D96965

show more ...

[mlir][IR] Remove the concept of `OperationProperties`

These properties were useful for a few things before traits had a better integration story, but don't really carry their weight well these days

[mlir][IR] Remove the concept of `OperationProperties`

These properties were useful for a few things before traits had a better integration story, but don't really carry their weight well these days. Most of these properties are already checked via traits in most of the code. It is better to align the system around traits, and improve the performance/cost of traits in general.

Differential Revision: https://reviews.llvm.org/D96088

show more ...

[mlir] Make GpuAsyncRegion pass depend on async dialect.

Do not cache gpu.async.token type so that the pass can be created before the GPU dialect is registered.

Reviewed By: ftynse

Differential Re

[mlir] Make GpuAsyncRegion pass depend on async dialect.

Do not cache gpu.async.token type so that the pass can be created before the GPU dialect is registered.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D94397

show more ...

[mlir] Remove the MutableDictionaryAttr class

This class used to serve a few useful purposes:
* Allowed containing a null DictionaryAttr
* Provided some simple mutable API around a DictionaryAttr

T

[mlir] Remove the MutableDictionaryAttr class

This class used to serve a few useful purposes:
* Allowed containing a null DictionaryAttr
* Provided some simple mutable API around a DictionaryAttr

The first of which is no longer an issue now that there is much better caching support for attributes in general, and a cache in the context for empty dictionaries. The second results in more trouble than it's worth because it mutates the internal dictionary on every action, leading to a potentially large number of dictionary copies. NamedAttrList is a much better alternative for the second use case, and should be modified as needed to better fit it's usage as a DictionaryAttrBuilder.

Differential Revision: https://reviews.llvm.org/D93442

show more ...

[mlir] Fix for gpu-async-region pass.

- the !gpu.async.token is the second result of 'gpu.alloc async', not the first.
- async.execute construction takes operand types not yet wrapped in !async.valu

[mlir] Fix for gpu-async-region pass.

- the !gpu.async.token is the second result of 'gpu.alloc async', not the first.
- async.execute construction takes operand types not yet wrapped in !async.value.
- fix typo

Reviewed By: herhut

Differential Revision: https://reviews.llvm.org/D93156

show more ...

[mlir] Use mlir::OpState::operator->() to get to methods of mlir::Operation. This is a preparation step to remove the corresponding methods from OpState.

Reviewed By: silvas, rriddle

Differential R

[mlir] Use mlir::OpState::operator->() to get to methods of mlir::Operation. This is a preparation step to remove the corresponding methods from OpState.

Reviewed By: silvas, rriddle

Differential Revision: https://reviews.llvm.org/D92878

show more ...

[mlir][gpu] Move gpu.wait ops from async.execute regions to its dependencies.

This can prevent unnecessary host synchronization.

Reviewed By: herhut

Differential Revision: https://reviews.llvm.org

[mlir][gpu] Move gpu.wait ops from async.execute regions to its dependencies.

This can prevent unnecessary host synchronization.

Reviewed By: herhut

Differential Revision: https://reviews.llvm.org/D90346

show more ...

[MLIR] Work around an ICE in GCC 7.

Looks like we have a blind spot in the testing matrix.

AsyncRegionRewriter.cpp: In member function ‘virtual void {anonymous}::GpuAsyncRegionPass::runOnFunction()

[MLIR] Work around an ICE in GCC 7.

Looks like we have a blind spot in the testing matrix.

AsyncRegionRewriter.cpp: In member function ‘virtual void {anonymous}::GpuAsyncRegionPass::runOnFunction()’:
AsyncRegionRewriter.cpp:113:16: internal compiler error: in replace_placeholders_r, at cp/tree.c:2804
if (getFunction()
~~~~~~~~~~~~~
.getRegion()
~~~~~~~~~~~~
.walk(Callback{OpBuilder{&getContext()}})
~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

show more ...

[mlir][gpu] Add pass to make GPU ops within a region execute asynchronously.

Do not use the pass yet, except in a test.

Reviewed By: herhut

Differential Revision: https://reviews.llvm.org/D89937