[mlir][tensor][bufferize][NFC] Remove duplicate code

InsertSliceOp and ParallelInsertSliceOp are very similar and can share some of the bufferization analysis code.

Differential Revision: https://r

[mlir][tensor][bufferize][NFC] Remove duplicate code

InsertSliceOp and ParallelInsertSliceOp are very similar and can share some of the bufferization analysis code.

Differential Revision: https://reviews.llvm.org/D130465

show more ...

[mlir][tensor][bufferize] Fix deallocation of GenerateOp/FromElementsOp

Both ops allocate a buffer. There were cases in which the buffer was not deallocated.

Differential Revision: https://reviews.

[mlir][tensor][bufferize] Fix deallocation of GenerateOp/FromElementsOp

Both ops allocate a buffer. There were cases in which the buffer was not deallocated.

Differential Revision: https://reviews.llvm.org/D130469

show more ...

[mlir][tensor][bufferize] Load dependent dialects

Load dialects that will be generated by the extension. (Except for BufferizationDialect and MemrefDialect which are loaded already.)

Differential R

[mlir][tensor][bufferize] Load dependent dialects

Load dialects that will be generated by the extension. (Except for BufferizationDialect and MemrefDialect which are loaded already.)

Differential Revision: https://reviews.llvm.org/D130463

show more ...

[mlir] Flip accessors to prefixed form (NFC)

Another mechanical sweep to keep diff small for flip to _Prefixed.

[mlir][bufferization][NFC] Move more unknown type conversion logic into BufferizationOptions

The `unknownTypeConversion` bufferization option (enum) is now a type converter function option. Some log

[mlir][bufferization][NFC] Move more unknown type conversion logic into BufferizationOptions

The `unknownTypeConversion` bufferization option (enum) is now a type converter function option. Some logic of `getMemRefType` is now handled by that function.

This change makes type conversion more controllable. Previously, there were only two options when generating memref types for non-bufferizable ops: Static identity layout or fully dynamic layout. With this change, users of One-Shot Bufferize can provide a function with custom logic.

Differential Revision: https://reviews.llvm.org/D129273

show more ...

[mlir][memref] Improve type inference for rank-reducing subviews

The result shape of a rank-reducing subview cannot be inferred in the general case. Just the result rank is not enough. The only thin

[mlir][memref] Improve type inference for rank-reducing subviews

The result shape of a rank-reducing subview cannot be inferred in the general case. Just the result rank is not enough. The only thing that we can infer is the layout map.

This change also improves the bufferization patterns of tensor.extract_slice and tensor.insert_slice to fully support rank-reducing operations.

Differential Revision: https://reviews.llvm.org/D129144

show more ...

[mlir][Tensor] Move ParallelInsertSlice to the tensor dialect

This is moslty NFC and will allow tensor.parallel_insert_slice to gain
rank-reducing semantics by reusing the vast majority of the tenso

[mlir][Tensor] Move ParallelInsertSlice to the tensor dialect

This is moslty NFC and will allow tensor.parallel_insert_slice to gain
rank-reducing semantics by reusing the vast majority of the tensor.insert_slice impl.

Depends on D128857

Differential Revision: https://reviews.llvm.org/D128920

show more ...

[mlir][bufferize] Infer memory space in all bufferization patterns

This change updates all remaining bufferization patterns (except for scf.while) and the remaining bufferization infrastructure to i

[mlir][bufferize] Infer memory space in all bufferization patterns

This change updates all remaining bufferization patterns (except for scf.while) and the remaining bufferization infrastructure to infer the memory space whenever possible instead of falling back to "0". (If a default memory space is set in the bufferization options, we still fall back to that value if the memory space could not be inferred.)

Differential Revision: https://reviews.llvm.org/D128423

show more ...

[mlir][bufferize][NFC] Change signature of allocateTensorForShapedValue

Add a failure return value and bufferization options argument. This is to keep a subsequent change smaller.

Differential Revi

[mlir][bufferize][NFC] Change signature of allocateTensorForShapedValue

Add a failure return value and bufferization options argument. This is to keep a subsequent change smaller.

Differential Revision: https://reviews.llvm.org/D128278

show more ...

[mlir][bufferization][NFC] Add error handling to getBuffer

This is in preparation of adding memory space support.

Differential Revision: https://reviews.llvm.org/D128277

[mlir][bufferization][NFC] Change signature of getMemRefType

These functions now accep unsigned attributes for address spaces instead of Attributes.

Differential Revision: https://reviews.llvm.org/

[mlir][bufferization][NFC] Change signature of getMemRefType

These functions now accep unsigned attributes for address spaces instead of Attributes.

Differential Revision: https://reviews.llvm.org/D128275

show more ...

[mlir] move SCF headers to SCF/{IR,Transforms} respectively

This aligns the SCF dialect file layout with the majority of the dialects.

Reviewed By: jpienaar

Differential Revision: https://reviews.

[mlir] move SCF headers to SCF/{IR,Transforms} respectively

This aligns the SCF dialect file layout with the majority of the dialects.

Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D128049

show more ...

[mlir] Update accessors to prefixed form (NFC)

Follow up from flipping dialects to both, flip accessor used to prefixed
variant ahead to flipping from _Both to _Prefixed. This just flips to
the acce

[mlir] Update accessors to prefixed form (NFC)

Follow up from flipping dialects to both, flip accessor used to prefixed
variant ahead to flipping from _Both to _Prefixed. This just flips to
the accessors introduced in the preceding change which are just prefixed
forms of the existing accessor changed from.

Mechanical change using helper script
https://github.com/jpienaar/llvm-project/blob/main/clang-tools-extra/clang-tidy/misc/AddGetterCheck.cpp and clang-format.

show more ...

[mlir] Start migrating more dialects to prefixed form

Marked all dialects that could be (reasonably) easily flipped to _Both
prefix. Updating the accessors to prefixed form will happen in follow
up,

[mlir] Start migrating more dialects to prefixed form

Marked all dialects that could be (reasonably) easily flipped to _Both
prefix. Updating the accessors to prefixed form will happen in follow
up, this was to flush out conflicts and to mark all dialects explicitly
as I plan to flip OpBase default to _Prefixed to avoid needing to
migrate new dialects.

Except for Standalone example which got flipped to _Prefixed.

Differential Revision: https://reviews.llvm.org/D128027

show more ...

[mlir][bufferize][NFC] Remove BufferizationState

With the recent refactorings, this class is no longer needed. We can use BufferizationOptions in all places were BufferizationState was used.

Differ

[mlir][bufferize][NFC] Remove BufferizationState

With the recent refactorings, this class is no longer needed. We can use BufferizationOptions in all places were BufferizationState was used.

Differential Revision: https://reviews.llvm.org/D127653

show more ...

[mlir][bufferize] Bufferize after TensorCopyInsertion

This change changes the bufferization so that it utilizes the new TensorCopyInsertion pass. One-Shot Bufferize no longer calls the One-Shot Anal

[mlir][bufferize] Bufferize after TensorCopyInsertion

This change changes the bufferization so that it utilizes the new TensorCopyInsertion pass. One-Shot Bufferize no longer calls the One-Shot Analysis. Instead, it relies on the TensorCopyInsertion pass to make the entire IR fully inplacable. The `bufferize` implementations of all ops are simplified; they no longer have to account for out-of-place bufferization decisions. These were already materialized in the IR in the form of `bufferization.alloc_tensor` ops during the TensorCopyInsertion pass.

Differential Revision: https://reviews.llvm.org/D127652

show more ...

[mlir][SCF] Fix scf.while bufferization

Before this fix, the bufferization implementation made the incorrect assumption that the values yielded from the "before" region must match with the values yi

[mlir][SCF] Fix scf.while bufferization

Before this fix, the bufferization implementation made the incorrect assumption that the values yielded from the "before" region must match with the values yielded from the "after" region.

Differential Revision: https://reviews.llvm.org/D125835

show more ...

[mlir][bufferize][NFC] Make getContiguousMemRefType a static function

No need to expose this as public API anymore.

Differential Revision: https://reviews.llvm.org/D125361

[mlir][bufferize][NFC] Move helper functions to BufferizationOptions

Move helper functions for creating allocs/deallocs/memcpys to BufferizationOptions.

Differential Revision: https://reviews.llvm.

[mlir][bufferize][NFC] Move helper functions to BufferizationOptions

Move helper functions for creating allocs/deallocs/memcpys to BufferizationOptions.

Differential Revision: https://reviews.llvm.org/D125375

show more ...

[mlir] Fail early if AnalysisState::getBuffer() returns failure

This patch updates calls to AnalysisState::getBuffer() so that we return
early with a failure if the call does not succeed.

Reviewed

[mlir] Fail early if AnalysisState::getBuffer() returns failure

This patch updates calls to AnalysisState::getBuffer() so that we return
early with a failure if the call does not succeed.

Reviewed By: springerm

Differential Revision: https://reviews.llvm.org/D125251

show more ...

[mlir] Add translation from tensor.reshape to memref.reshape

This patch augments the `tensor-bufferize` pass by adding a conversion
rule to translate ReshapeOp from the `tensor` dialect to the `memr

[mlir] Add translation from tensor.reshape to memref.reshape

This patch augments the `tensor-bufferize` pass by adding a conversion
rule to translate ReshapeOp from the `tensor` dialect to the `memref`
dialect, in addition to adding a unit test to validate the translation.

Reviewed By: springerm

Differential Revision: https://reviews.llvm.org/D125031

show more ...

[mlir][NFC] Shift a bunch of dialect includes from the .h to the .cpp

Now that dialect constructors are generated in the .cpp file, we can
drop all of the dependent dialect includes from the .h file

[mlir][NFC] Shift a bunch of dialect includes from the .h to the .cpp

Now that dialect constructors are generated in the .cpp file, we can
drop all of the dependent dialect includes from the .h file.

Differential Revision: https://reviews.llvm.org/D124298

show more ...

[mlir] Move isGuaranteedCollapsible to CollapseShapeOp (NFC).

It seems more natural than to have it as a static method of ExpandShapeOp.
Also fix a typo ("the the" -> "the").

Differential Revision:

[mlir] Move isGuaranteedCollapsible to CollapseShapeOp (NFC).

It seems more natural than to have it as a static method of ExpandShapeOp.
Also fix a typo ("the the" -> "the").

Differential Revision: https://reviews.llvm.org/D124234

show more ...

[mlir][tensor] Fix verifier and bufferization of collapse_shape

Insert a buffer copy unless the dims are guaranteed to be collapsible. In the verifier, accept collapses unless they are guaranteed to

[mlir][tensor] Fix verifier and bufferization of collapse_shape

Insert a buffer copy unless the dims are guaranteed to be collapsible. In the verifier, accept collapses unless they are guaranteed to be non-collapsible.

Differential Revision: https://reviews.llvm.org/D123316

show more ...

[mlir][tensor][bufferize] Support 0-d collapse_shape with offset

Differential Revision: https://reviews.llvm.org/D122901