CombineContractBroadcast should not create dims unused in LHS+RHS

Differential Revision: https://reviews.llvm.org/D129087

[mlir] Bubble up tensor.extract_slice above linalg operation

Bubble up extract_slice above Linalg operation.

A sequence of operations

%0 = linalg.<op> ... arg0, arg1, ...
%1 = tensor.extra

[mlir] Bubble up tensor.extract_slice above linalg operation

Bubble up extract_slice above Linalg operation.

A sequence of operations

%0 = linalg.<op> ... arg0, arg1, ...
%1 = tensor.extract_slice %0 ...

can be replaced with

%0 = tensor.extract_slice %arg0
%1 = tensor.extract_slice %arg1
%2 = linalg.<op> ... %0, %1, ...

This results in the reduce computation of the linalg operation.

The implementation uses the tiling utility functions. One difference
from the tiling process is that we don't need to insert the checking
code for the out-of-bound accesses. The use of the slice itself
represents that the code writer is sure about the boundary condition.
To avoid adding the boundary condtion check code, `omitPartialTileCheck`
is introduced for the tiling utility functions.

Differential Revision: https://reviews.llvm.org/D122437

show more ...

Use llvm::append_range where applicable

It knows the size, so no need to call reserve beforehand. NFCI.

[AffineMap] Move result exprs into trailing storage. NFCI.

Use AffineMap::getSliceMap where applicable. NFCI.

[mlir] Use SmallBitVector instead of SmallDenseSet for AffineMap::compressSymbols

This is both more efficient and more ergonomic to use, as inverting a
bit vector is trivial while inverting a set is

[mlir] Use SmallBitVector instead of SmallDenseSet for AffineMap::compressSymbols

This is both more efficient and more ergonomic to use, as inverting a
bit vector is trivial while inverting a set is annoying.

Sadly this leaks into a bunch of APIs downstream, so adapt them as well.

This would be NFC, but there is an ordering dependency in MemRefOps's
computeMemRefRankReductionMask. This is now deterministic, previously it
was dependent on SmallDenseSet's unspecified iteration order.

Differential Revision: https://reviews.llvm.org/D119076

show more ...

Apply clang-tidy fixes for performance-for-range-copy to MLIR (NFC)

Apply clang-tidy fixes for performance-unnecessary-value-param to MLIR (NFC)

Reviewed By: Mogball

Differential Revision: https://reviews.llvm.org/D116250

Fix clang-tidy issues in mlir/ (NFC)

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D115956

Adjust "end namespace" comment in MLIR to match new agree'd coding style

See D115115 and this mailing list discussion:
https://lists.llvm.org/pipermail/llvm-dev/2021-December/154199.html

Differenti

Adjust "end namespace" comment in MLIR to match new agree'd coding style

See D115115 and this mailing list discussion:
https://lists.llvm.org/pipermail/llvm-dev/2021-December/154199.html

Differential Revision: https://reviews.llvm.org/D115309

show more ...

[MLIR] Avoid creation of buggy affine maps when incorrect values of number of dimensions and number of symbols are provided.

We check whether the maximum index of dimensional identifier present
in t

[MLIR] Avoid creation of buggy affine maps when incorrect values of number of dimensions and number of symbols are provided.

We check whether the maximum index of dimensional identifier present
in the result expressions is less than dimCount (number of dimensional
identifiers) argument passed in the AffineMap::get() and the maximum index
of symbolic identifier present in the result expressions is less than
symbolCount (number of symbolic identifiers) argument passed in AffineMap::get().

Reviewed By: nicolasvasilache, bondhugula

Differential Revision: https://reviews.llvm.org/D114238

show more ...

[mlir][Linalg] Enable vectorization of explicit broadcasts

This patch teaches `isProjectedPermutation` and `inverseAndBroadcastProjectedPermutation`
utilities to deal with maps representing an expli

[mlir][Linalg] Enable vectorization of explicit broadcasts

This patch teaches `isProjectedPermutation` and `inverseAndBroadcastProjectedPermutation`
utilities to deal with maps representing an explicit broadcast, e.g., (d0, d1) -> (d0, 0).
This extension is needed to enable vectorization of such explicit broadcast in Linalg.

Reviewed By: pifon2a, nicolasvasilache

Differential Revision: https://reviews.llvm.org/D111563

show more ...

[mlir] add permutation utility

I found myself typing this code several times at different places
by now, so time to make this a general utility instead. Given
a permutation, it returns the permuted

[mlir] add permutation utility

I found myself typing this code several times at different places
by now, so time to make this a general utility instead. Given
a permutation, it returns the permuted position of the input,
for example (i,j,k) -> (k,i,j) yields position 1 for input 0.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D108347

show more ...

[mlir] NFC - Add AffineMap::replace variant with dim/symbol inference

[mlir][linalg] Add scalar operands inlining pattern

This pattern inlines operands to a linalg.generic operation that use a constant
index and hence are loop-invariant scalars. This reduces the numbe

[mlir][linalg] Add scalar operands inlining pattern

This pattern inlines operands to a linalg.generic operation that use a constant
index and hence are loop-invariant scalars. This reduces the number of
linalg.generic operands and unlocks some canonicalizations that rely on seeing
an explicit tensor.extract.

Differential Revision: https://reviews.llvm.org/D102682

show more ...

[mlir][linalg] Fix vectorization bug in vector transfer indexing map calculation

The current implementation had a bug as it was relying on the target vector
dimension sizes to calculate where to ins

[mlir][linalg] Fix vectorization bug in vector transfer indexing map calculation

The current implementation had a bug as it was relying on the target vector
dimension sizes to calculate where to insert broadcast. If several dimensions
have the same size we may insert the broadcast on the wrong dimension. The
correct broadcast cannot be inferred from the type of the source and
destination vector.

Instead when we want to extend transfer ops we calculate an "inverse" map to the
projected permutation and insert broadcast in place of the projected dimensions.

Differential Revision: https://reviews.llvm.org/D101738

show more ...

[mlir] support max/min lower/upper bounds in affine.parallel

This enables to express more complex parallel loops in the affine framework,
for example, in cases of tiling by sizes not dividing loop t

[mlir] support max/min lower/upper bounds in affine.parallel

This enables to express more complex parallel loops in the affine framework,
for example, in cases of tiling by sizes not dividing loop trip counts perfectly
or inner wavefront parallelism, among others. One can't use affine.max/min
and supply values to the nested loop bounds since the results of such
affine.max/min operations aren't valid symbols. Making them valid symbols
isn't an option since they would introduce selection trees into memref
subscript arithmetic as an unintended and undesired consequence. Also
add support for converting such loops to SCF. Drop some API that isn't used in
the core repo from AffineParallelOp since its semantics becomes ambiguous in
presence of max/min bounds. Loop normalization is currently unavailable for
such loops.

Depends On D101171

Reviewed By: bondhugula

Differential Revision: https://reviews.llvm.org/D101172

show more ...

[mlir][Python][Linalg] Add missing attributes to linalg ops

This revision tightens up the handling of attributes for both named
and generic linalg ops.
To demonstrate the IR validity, a working e2e

[mlir][Python][Linalg] Add missing attributes to linalg ops

This revision tightens up the handling of attributes for both named
and generic linalg ops.
To demonstrate the IR validity, a working e2e Linalg example is added.

Differential Revision: https://reviews.llvm.org/D99430

show more ...

[mlir][vector] Add lowering of Transfer_read with broadcast and permutation map

Convert transfer_read ops with permutation maps into simpler
transfer_read with minority map + vector.braodcast and ve

[mlir][vector] Add lowering of Transfer_read with broadcast and permutation map

Convert transfer_read ops with permutation maps into simpler
transfer_read with minority map + vector.braodcast and vector.transpose.
And transfer_read with leading dimensions broacast into transfer_read of
lower rank.

Differential Revision: https://reviews.llvm.org/D99019

show more ...

[mlir][linalg] Extend linalg vectorization to support non-identity input maps

This propagates the affine map to transfer_read op in case it is not a
minor identity map.

Differential Revision: https

[mlir][linalg] Extend linalg vectorization to support non-identity input maps

This propagates the affine map to transfer_read op in case it is not a
minor identity map.

Differential Revision: https://reviews.llvm.org/D98523

show more ...

[mlir][Vector] Lowering of transfer_read/write to vector.load/store

This patch introduces progressive lowering patterns for rewriting
vector.transfer_read/write to vector.load/store and vector.broad

[mlir][Vector] Lowering of transfer_read/write to vector.load/store

This patch introduces progressive lowering patterns for rewriting
vector.transfer_read/write to vector.load/store and vector.broadcast
in certain supported cases.

Reviewed By: dcaballe, nicolasvasilache

Differential Revision: https://reviews.llvm.org/D97822

show more ...

s[mlir] Tighten computation of inferred SubView result type.

The AffineMap in the MemRef inferred by SubViewOp may have uncompressed symbols which result in type mismatch on otherwise unused symbols

s[mlir] Tighten computation of inferred SubView result type.

The AffineMap in the MemRef inferred by SubViewOp may have uncompressed symbols which result in type mismatch on otherwise unused symbols. Make the computation of the AffineMap compress those unused symbols which results in better canonical types.
Additionally, improve the error message to report which inferred type was expected.

Differential Revision: https://reviews.llvm.org/D96551

show more ...

[mlir] Add `const` qualifiers to `AffineMap` methods

The `AffineMap` class follows the same semantic as Type and Attribute.
It is immutable object, so it make sence to mark its methods as const.
Als

[mlir] Add `const` qualifiers to `AffineMap` methods

The `AffineMap` class follows the same semantic as Type and Attribute.
It is immutable object, so it make sence to mark its methods as const.
Also part of its API is already marked as const, this change just make the API consistent.

Reviewed By: ftynse, bondhugula

Differential Revision: https://reviews.llvm.org/D96026

show more ...

[mlir][Affine] Revisit and simplify composeAffineMapAndOperands.

In prehistorical times, AffineApplyOp was allowed to produce multiple values.
This allowed the creation of intricate SSA use-def chai

[mlir][Affine] Revisit and simplify composeAffineMapAndOperands.

In prehistorical times, AffineApplyOp was allowed to produce multiple values.
This allowed the creation of intricate SSA use-def chains.
AffineApplyNormalizer was originally introduced as a means of reusing the AffineMap::compose method to write SSA use-def chains.
Unfortunately, symbols that were produced by an AffineApplyOp needed to be promoted to dims and reordered for the mathematical composition to be valid.

Since then, single result AffineApplyOp became the law of the land but the original assumptions were not revisited.

This revision revisits these assumptions and retires AffineApplyNormalizer.

Differential Revision: https://reviews.llvm.org/D94920

show more ...

[MLIR] Fix affine_map compose with multi-symbols

Fix bug: https://bugs.llvm.org/show_bug.cgi?id=46845

Differential Revision: https://reviews.llvm.org/D93831