[mlir][bufferize] Bufferize after TensorCopyInsertion

This change changes the bufferization so that it utilizes the new TensorCopyInsertion pass. One-Shot Bufferize no longer calls the One-Shot Anal

[mlir][bufferize] Bufferize after TensorCopyInsertion

This change changes the bufferization so that it utilizes the new TensorCopyInsertion pass. One-Shot Bufferize no longer calls the One-Shot Analysis. Instead, it relies on the TensorCopyInsertion pass to make the entire IR fully inplacable. The `bufferize` implementations of all ops are simplified; they no longer have to account for out-of-place bufferization decisions. These were already materialized in the IR in the form of `bufferization.alloc_tensor` ops during the TensorCopyInsertion pass.

Differential Revision: https://reviews.llvm.org/D127652

show more ...

[mlir][bufferize] Fix op filter

Bufferization has an optional filter to exclude certain ops from analysis+bufferization. There were a few remaining places in the codebase where the filter was not ch

[mlir][bufferize] Fix op filter

Bufferization has an optional filter to exclude certain ops from analysis+bufferization. There were a few remaining places in the codebase where the filter was not checked.

Differential Revision: https://reviews.llvm.org/D125356

show more ...

[mlir][NFC] Update textual references of `func` to `func.func` in Linalg tests

The special case parsing of `func` operations is being removed.

[mlir][bufferize] Extract buffer hoisting into separate function

This improves the modularity of the bufferization.

From now on, all ops that do not implement BufferizableOpInterface are considered

[mlir][bufferize] Extract buffer hoisting into separate function

This improves the modularity of the bufferization.

From now on, all ops that do not implement BufferizableOpInterface are considered hoisting barriers. Previously, all ops that do not implement the interface were not considered barriers and such ops had to be marked as barriers explicitly. This was unsafe because we could've hoisted across unknown ops where it was not safe to hoist.

As a side effect, this allows for cleaning up AffineBufferizableOpInterfaceImpl. This build unit no longer needed and can be deleted.

Differential Revision: https://reviews.llvm.org/D121519

show more ...

[mlir][linalg] Replace linalg.fill by OpDSL variant.

The revision removes the linalg.fill operation and renames the OpDSL generated linalg.fill_tensor operation to replace it. After the change, all

[mlir][linalg] Replace linalg.fill by OpDSL variant.

The revision removes the linalg.fill operation and renames the OpDSL generated linalg.fill_tensor operation to replace it. After the change, all named structured operations are defined via OpDSL and there are no handwritten operations left.

A side-effect of the change is that the pretty printed form changes from:
```
%1 = linalg.fill(%cst, %0) : f32, tensor<?x?xf32> -> tensor<?x?xf32>
```
changes to
```
%1 = linalg.fill ins(%cst : f32) outs(%0 : tensor<?x?xf32>) -> tensor<?x?xf32>
```
Additionally, the builder signature now takes input and output value ranges as it is the case for all other OpDSL operations:
```
rewriter.create<linalg::FillOp>(loc, val, output)
```
changes to
```
rewriter.create<linalg::FillOp>(loc, ValueRange{val}, ValueRange{output})
```
All other changes remain minimal. In particular, the canonicalization patterns are the same and the `value()`, `output()`, and `result()` methods are now implemented by the FillOpInterface.

Depends On D120726

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D120728

show more ...

[mlir][linalg][bufferize] Migrate --linalg-bufferize to BufferizableOpInterface-based bufferization

This commit deletes the old dialect conversion-based bufferization patterns, which are now obsolet

[mlir][linalg][bufferize] Migrate --linalg-bufferize to BufferizableOpInterface-based bufferization

This commit deletes the old dialect conversion-based bufferization patterns, which are now obsolete.

Differential Revision: https://reviews.llvm.org/D120883

show more ...

[mlir][bufferize][NFC] Remove obsolete tensor bufferization patterns from Linalg/Bufferize.cpp

Differential Revision: https://reviews.llvm.org/D119824

[mlir][bufferize] Add vector-bufferize pass and remove obsolete patterns from Linalg Bufferize

Differential Revision: https://reviews.llvm.org/D119444

Revert "Revert "[mlir] Purge `linalg.copy` and use `memref.copy` instead.""

This reverts commit 25bf6a2a9bc6ecb3792199490c70c4ce50a94aea.

Revert "[mlir] Purge `linalg.copy` and use `memref.copy` instead."

This reverts commit 016956b68081705ffee511c334e31e414fa1ddbf.
Reverting it to fix NVidia build without being in a hurry.

[mlir] Purge `linalg.copy` and use `memref.copy` instead.

Differential Revision: https://reviews.llvm.org/D118028

[mlir] Move linalg::PadTensorOp to tensor::PadOp.

RFC: https://llvm.discourse.group/t/rfc-move-linalg-padtensorop-to-tensor-padop/5785

Differential Revision: https://reviews.llvm.org/D117892

[mlir] Don't print `// no predecessors` on entry blocks

Entry blocks can never have predecessors, so this is unnecessary.

Fixes #53287

Differential Revision: https://reviews.llvm.org/D117713

[mlir] Move `linalg.tensor_expand/collapse_shape` to TensorDialect.

RFC: https://llvm.discourse.group/t/rfc-reshape-ops-restructuring/3310

linalg.fill gets a canonicalizer, because `FoldFillWithTen

[mlir] Move `linalg.tensor_expand/collapse_shape` to TensorDialect.

RFC: https://llvm.discourse.group/t/rfc-reshape-ops-restructuring/3310

linalg.fill gets a canonicalizer, because `FoldFillWithTensorReshape` cannot be moved to tensorops (it uses linalg::FillOp inside). Before it was listed as a canonicalization pattern for the reshape operations, now it became a canonicalization for FillOp.

Differential Revision: https://reviews.llvm.org/D115502

show more ...

[mlir] Move memref.[tensor_load|buffer_cast|clone] to "bufferization" dialect.

https://llvm.discourse.group/t/rfc-dialect-for-bufferization-related-ops/4712

Differential Revision: https://reviews.l

[mlir] Move memref.[tensor_load|buffer_cast|clone] to "bufferization" dialect.

https://llvm.discourse.group/t/rfc-dialect-for-bufferization-related-ops/4712

Differential Revision: https://reviews.llvm.org/D114552

show more ...

[mlir:DialectConversion] Restructure how argument/target materializations get invoked

The current implementation invokes materializations
whenever an input operand does not have a mapping for the
de

[mlir:DialectConversion] Restructure how argument/target materializations get invoked

The current implementation invokes materializations
whenever an input operand does not have a mapping for the
desired type, i.e. it requires materialization at the earliest possible
point. This conflicts with goal of dialect conversion (and also the
current documentation) which states that a materialization is only
required if the materialization is supposed to persist after the
conversion process has finished.

This revision refactors this such that whenever a target
materialization "might" be necessary, we insert an
unrealized_conversion_cast to act as a temporary materialization.
This allows for deferring the invocation of the user
materialization hooks until the end of the conversion process,
where we actually have a better sense if it's actually
necessary. This has several benefits:

* In some cases a target materialization hook is no longer
necessary
When performing a full conversion, there are some situations
where a temporary materialization is necessary. Moving forward,
these users won't need to provide any target materializations,
as the temporary materializations do not require the user to
provide materialization hooks.

* getRemappedValue can now handle values that haven't been
converted yet
Before this commit, it wasn't well supported to get the remapped
value of a value that hadn't been converted yet (making it
difficult/impossible to convert multiple operations in many
situations). This commit updates getRemappedValue to properly
handle this case by inserting temporary materializations when
necessary.

Another code-health related benefit is that with this change we
can move a majority of the complexity related to materializations
to the end of the conversion process, instead of handling adhoc
while conversion is happening.

Differential Revision: https://reviews.llvm.org/D111620

show more ...

[MLIR] Replace std ops with arith dialect ops

Precursor: https://reviews.llvm.org/D110200

Removed redundant ops from the standard dialect that were moved to the
`arith` or `math` dialects.

Renamed

[MLIR] Replace std ops with arith dialect ops

Precursor: https://reviews.llvm.org/D110200

Removed redundant ops from the standard dialect that were moved to the
`arith` or `math` dialects.

Renamed all instances of operations in the codebase and in tests.

Reviewed By: rriddle, jpienaar

Differential Revision: https://reviews.llvm.org/D110797

show more ...

[mlir][linalg] Fix bufferize pattern to allow unknown operations in body of generic

The original version of the bufferization pattern for linalg.generic would
manually clone operations within the re

[mlir][linalg] Fix bufferize pattern to allow unknown operations in body of generic

The original version of the bufferization pattern for linalg.generic would
manually clone operations within the region to the bufferized clone of the
operation. This triggers legality requirements on those operations in the
conversion infra. Instead, this now uses the rewriter to inline the region
instead, avoiding those legality requirements.

Differential Revision: https://reviews.llvm.org/D109581

show more ...

[tests] Make testsuite more resilient to "order of constant" changes. NFC.

Revert "Bufferization for tiled loop."

This reverts commit edaffebcb2a62b0195e23fe7d4ead005822865c3.

Revert "[mlir] Change the pattern for TiledLoopOp bufferization."

This reverts commit 2f946eaa9d2648b883b2a1e567b23fff307f13d9.

[mlir] Change the pattern for TiledLoopOp bufferization.

This version is does not affect the patterns for Extract/InsertSliceOp and
LinalgOps.

Differential Revision: https://reviews.llvm.org/D107858

Cloned from CL 389610703 by 'g4 patch'.
Original change by pifon@pifon:tfrt_clean:6896:citc on 2021/08/09 05:30:17.

Ad b

Differential Revision: https://reviews.llvm.org/D107762

[mlir] Add patterns for vector.transfer_read/write to Linalg bufferization.

Differential Revision: https://reviews.llvm.org/D107643

[mlir] Move linalg::Expand/CollapseShapeOp to memref dialect.

RFC: https://llvm.discourse.group/t/rfc-reshape-ops-restructuring/3310

Differential Revision: https://reviews.llvm.org/D106141