[mlir][vector] Extend transfer_write to read propagation

Folding of transfer_write into transfer_read is already supported but
this requires the read and write to have the same permuation map.
After

[mlir][vector] Extend transfer_write to read propagation

Folding of transfer_write into transfer_read is already supported but
this requires the read and write to have the same permuation map.
After linalg vectorization it is common to have different ppermuation
map for write followed by read even though the cases could be
propagated.
This canonicalization handle cases where the permuation maps are
different but the data read and written match and replace the transfer
ops with broadcast and permuation

Differential Revision: https://reviews.llvm.org/D130135

show more ...

[mlir] Flip LinAlg dialect to _Both

This one required more changes than ideal due to overlapping generated name
with different return types. Changed getIndexingMaps to getIndexingMapsArray to
move i

[mlir] Flip LinAlg dialect to _Both

This one required more changes than ideal due to overlapping generated name
with different return types. Changed getIndexingMaps to getIndexingMapsArray to
move it out of the way/highlight that it returns (more expensively) a
SmallVector and uses the prefixed name for the Attribute.

Differential Revision: https://reviews.llvm.org/D129919

show more ...

[mlir] Use value instead of getValue (NFC)

[mlir][vector] Fix crash in vector.reduction canonicalization

since vector.reduce support accumulator in all the cases remove the
assert assuming old definition.

Differential Revision: https://revi

[mlir][vector] Fix crash in vector.reduction canonicalization

since vector.reduce support accumulator in all the cases remove the
assert assuming old definition.

Differential Revision: https://reviews.llvm.org/D129602

show more ...

[mlir][vector] Add accumulator operand to MultiDimReduce op

This allows vectorizing linalg reductions without changing the operation
order. Therefore this produce a valid vectorization even if opera

[mlir][vector] Add accumulator operand to MultiDimReduce op

This allows vectorizing linalg reductions without changing the operation
order. Therefore this produce a valid vectorization even if operations
are not associative.

Differential Revision: https://reviews.llvm.org/D129535

show more ...

[mlir][Vector] Fold InsertOp(SplatOp(X), SplatOp(X)) to SplatOp(X).

This patch folds InsertOp(SplatOp(X), SplatOp(X)) to SplatOp(X).

Differential Revision: https://reviews.llvm.org/D129058

CombineContractBroadcast should not create dims unused in LHS+RHS

Differential Revision: https://reviews.llvm.org/D129087

[mlir][Vector] Fold ShuffleOp(SplatOp(X), SplatOp(X)) to SplatOp(X).

This patch folds ShuffleOp(SplatOp(X), SplatOp(X)) to SplatOp(X).

Differential Revision: https://reviews.llvm.org/D128969

[mlir][Vector] Fold InsertStridedSliceOp of ExtractStridedSliceOp.

This patch supports to fold InsertStridedSliceOp(ExtractStridedSliceOp(dst), dst) to dst.

Differential Revision: https://reviews.l

[mlir][Vector] Fold InsertStridedSliceOp of ExtractStridedSliceOp.

This patch supports to fold InsertStridedSliceOp(ExtractStridedSliceOp(dst), dst) to dst.

Differential Revision: https://reviews.llvm.org/D128903

show more ...

[mlir][Vector] Fold InsertStridedSliceOp of two splat with the same input to splat.

This patch folds InsertStridedSliceOp(SplatOp(X):src_type, SplatOp(X):dst_type) to SplatOp(X):dst_type.

Reviewed

[mlir][Vector] Fold InsertStridedSliceOp of two splat with the same input to splat.

This patch folds InsertStridedSliceOp(SplatOp(X):src_type, SplatOp(X):dst_type) to SplatOp(X):dst_type.

Reviewed By: Mogball

Differential Revision: https://reviews.llvm.org/D128891

show more ...

Useful error when input dim is unused by LHS/RHS.

Differential Revision: https://reviews.llvm.org/D128925

[mlir] Update flipped accessors (NFC)

Follow up with memref flipped and flipping any intermediate changes
made.

q[mlir][Vector] Add a ShapeCastOp(BroadcastOp) canonicalization pattern

This pattern can kick in when the source of the broadcast has a shape
that is a prefix/suffix of the result of the shape_cast.

q[mlir][Vector] Add a ShapeCastOp(BroadcastOp) canonicalization pattern

This pattern can kick in when the source of the broadcast has a shape
that is a prefix/suffix of the result of the shape_cast.

Differential Revision: https://reviews.llvm.org/D128734

show more ...

[mlir][Vector] Fix reordering of floating point adds during lower of `vector.contract`.

Adding the accumulator value after the `vector.contract` changes the
precision of the operation. This makes su

[mlir][Vector] Fix reordering of floating point adds during lower of `vector.contract`.

Adding the accumulator value after the `vector.contract` changes the
precision of the operation. This makes sure the accumulator is carried
through to `vector.reduce` (and down to LLVM).

Differential Revision: https://reviews.llvm.org/D128674

show more ...

Revert "Don't use Optional::hasValue (NFC)"

This reverts commit aa8feeefd3ac6c78ee8f67bf033976fc7d68bc6d.

Don't use Optional::hasValue (NFC)

[mlir] Don't use Optional::getValue (NFC)

[mlir] Don't use Optional::hasValue (NFC)

[mlir][Vector] Fold consecutive bitcast.

This patch supports to fold consecutive bitcast into one bitcast.

Differential Revision: https://reviews.llvm.org/D127723

[mlir][Vector] Support vectorize to vector.reduction or/and.

This patch supports to vectorize affine.for of ori/andi to vector.reduction or/and.

Differential Revision: https://reviews.llvm.org/D127

[mlir][Vector] Support vectorize to vector.reduction or/and.

This patch supports to vectorize affine.for of ori/andi to vector.reduction or/and.

Differential Revision: https://reviews.llvm.org/D127090

show more ...

[ParseResult] Mark this as LLVM_NODISCARD (like LogicalResult) and fix issues.

There are a lot of cases where we accidentally ignored the result of some
parsing hook. Mark ParseResult as LLVM_NODIS

[ParseResult] Mark this as LLVM_NODISCARD (like LogicalResult) and fix issues.

There are a lot of cases where we accidentally ignored the result of some
parsing hook. Mark ParseResult as LLVM_NODISCARD just like ParseResult is.
This exposed some stuff to clean up, so do.

Differential Revision: https://reviews.llvm.org/D125549

show more ...

[AsmParser] Rework logic around "region argument parsing"

The asm parser had a notional distinction between parsing an
operand (like "%foo" or "%4#3") and parsing a region argument
(which isn't supp

[AsmParser] Rework logic around "region argument parsing"

The asm parser had a notional distinction between parsing an
operand (like "%foo" or "%4#3") and parsing a region argument
(which isn't supposed to allow a result number like #3).

Unfortunately the implementation has two problems:

1) It didn't actually check for the result number and reject
it. parseRegionArgument and parseOperand were identical.
2) It had a lot of machinery built up around it that paralleled
operand parsing. This also was functionally identical, but
also had some subtle differences (e.g. the parseOptional
stuff had a different result type).

I thought about just removing all of this, but decided that the
missing error checking was important, so I reimplemented it with
a `allowResultNumber` flag on parseOperand. This keeps the
codepaths unified and adds the missing error checks.

Differential Revision: https://reviews.llvm.org/D124470

show more ...

[mlir][vector] Fold 1-element reduction into extract or arith ops

If there is only one single element in the vector, then we can
just extract the element to compute the final result.

Reviewed By: m

[mlir][vector] Fold 1-element reduction into extract or arith ops

If there is only one single element in the vector, then we can
just extract the element to compute the final result.

Reviewed By: mravishankar

Differential Revision: https://reviews.llvm.org/D124129

show more ...

[mlir][vector] Fold cancelling vector.shape_cast(vector.broadcast)

vector.broadcast can inject all size one dimensions. If it's
followed by a vector.shape_cast to the original type, we can
cancel th

[mlir][vector] Fold cancelling vector.shape_cast(vector.broadcast)

vector.broadcast can inject all size one dimensions. If it's
followed by a vector.shape_cast to the original type, we can
cancel the op pair, like cancelling consecutive shape_cast ops.

Reviewed By: mravishankar

Differential Revision: https://reviews.llvm.org/D124094

show more ...

[mlir][Vector] Add check of supported reduction kind for ScanOp.

This patch adds check of supported reduction kind for ScanOp to avoid using and/or/xor for floating point type.

Reviewed By: ftynse

[mlir][Vector] Add check of supported reduction kind for ScanOp.

This patch adds check of supported reduction kind for ScanOp to avoid using and/or/xor for floating point type.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D123977

show more ...