Use llvm::sort instead of std::sort where possible

llvm::sort is beneficial even when we use the iterator-based overload,
since it can optionally shuffle the elements (to detect
non-determinism). Ho

Use llvm::sort instead of std::sort where possible

llvm::sort is beneficial even when we use the iterator-based overload,
since it can optionally shuffle the elements (to detect
non-determinism). However llvm::sort is not usable everywhere, for
example, in compiler-rt.

Reviewed By: nhaehnle

Differential Revision: https://reviews.llvm.org/D130406

show more ...

Use llvm::is_contained (NFC)

[mlir] (NFC) run clang-format on all files

[mlir] Don't use Optional::getValue (NFC)

Apply clang-tidy fixes for readability-simplify-boolean-expr to MLIR (NFC)

Apply clang-tidy fixes for performance-for-range-copy to MLIR (NFC)

Apply clang-tidy fixes for performance-unnecessary-value-param to MLIR (NFC)

Reviewed By: Mogball

Differential Revision: https://reviews.llvm.org/D116250

[MLIR] Simplify Semi-affine expressions by rule based matching and replacing "expr - q * (expr floordiv q)" with "expr mod q" expression.

Add rule based matching for detecting and transforming "exp

[MLIR] Simplify Semi-affine expressions by rule based matching and replacing "expr - q * (expr floordiv q)" with "expr mod q" expression.

Add rule based matching for detecting and transforming "expr - q * (expr floordiv q)"
to "expr mod q", where q is a symbolic exxpression, in simplifyAdd function.

Reviewed By: bondhugula, dcaballe

Differential Revision: https://reviews.llvm.org/D112985

show more ...

[MLIR] Simplify semi-affine expressions using flattening

For the semi affine expressions, whenever rhs of a floordiv, ceildiv, mod
or product expression is a symbolic expression, we introduce a loca

[MLIR] Simplify semi-affine expressions using flattening

For the semi affine expressions, whenever rhs of a floordiv, ceildiv, mod
or product expression is a symbolic expression, we introduce a local variable
representing the result, and store the floordiv/ceildiv, mod or product
affine expression in LocalExprs. In this way the expression is flattened,
and trivial addition and subtraction related simplifications are performed.
Also rule based matching for detecting and transforming "expr - q * (expr floordiv q)"
to "expr mod q", where q is a symbolic exxpression, in simplifyAdd function.

Differential Revision: https://reviews.llvm.org/D112808

show more ...

[MLIR] Fix AffineExpr getLargestKnownDivisor for ceildiv and floordiv

Fix AffineExpr `getLargestKnownDivisor` for ceil/floor div cases.
In these cases, nothing can be inferred on the divisor of the

[MLIR] Fix AffineExpr getLargestKnownDivisor for ceildiv and floordiv

Fix AffineExpr `getLargestKnownDivisor` for ceil/floor div cases.
In these cases, nothing can be inferred on the divisor of the
result.

Add test case for `mod` as well.

Differential Revision: https://reviews.llvm.org/D112523

show more ...

[MLIR][Affine] Simplify nested modulo operations when able

It is the case that, for all positive a and b such that b divides a
(e mod (a * b)) mod b = e mod b. For example, ((d0 mod 35) mod 5) can
b

[MLIR][Affine] Simplify nested modulo operations when able

It is the case that, for all positive a and b such that b divides a
(e mod (a * b)) mod b = e mod b. For example, ((d0 mod 35) mod 5) can
be simplified to (d0 mod 5), but ((d0 mod 35) mod 4) cannot be simplified
further (x = 36 is a counterexample).

This change enables more complex simplifications. For example,
((d0 * 72 + d1) mod 144) mod 9 can now simplify to (d0 * 72 + d1) mod 9
and thus to d1 mod 9. Expressions with chained modulus operators are
reasonably common in tensor applications, and this change _should_
improve code generation for such expressions.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D109930

show more ...

[mlir][scf] Simplify affine.min ops after loop peeling

Simplify affine.min ops, enabling various other canonicalizations inside the peeled loop body.

affine.min ops such as:
```
map = affine_map<(d

[mlir][scf] Simplify affine.min ops after loop peeling

Simplify affine.min ops, enabling various other canonicalizations inside the peeled loop body.

affine.min ops such as:
```
map = affine_map<(d0)[s0, s1] -> (s0, -d0 + s1)>
%r = affine.min #affine.min #map(%iv)[%step, %ub]
```
are rewritten them into (in the case the peeled loop):
```
%r = %step
```

To determine how an affine.min op should be rewritten and to prove its correctness, FlatAffineConstraints is utilized.

Differential Revision: https://reviews.llvm.org/D107222

show more ...

s[mlir] Tighten computation of inferred SubView result type.

The AffineMap in the MemRef inferred by SubViewOp may have uncompressed symbols which result in type mismatch on otherwise unused symbols

s[mlir] Tighten computation of inferred SubView result type.

The AffineMap in the MemRef inferred by SubViewOp may have uncompressed symbols which result in type mismatch on otherwise unused symbols. Make the computation of the AffineMap compress those unused symbols which results in better canonical types.
Additionally, improve the error message to report which inferred type was expected.

Differential Revision: https://reviews.llvm.org/D96551

show more ...

[mlir][Affine] Revisit and simplify composeAffineMapAndOperands.

In prehistorical times, AffineApplyOp was allowed to produce multiple values.
This allowed the creation of intricate SSA use-def chai

[mlir][Affine] Revisit and simplify composeAffineMapAndOperands.

In prehistorical times, AffineApplyOp was allowed to produce multiple values.
This allowed the creation of intricate SSA use-def chains.
AffineApplyNormalizer was originally introduced as a means of reusing the AffineMap::compose method to write SSA use-def chains.
Unfortunately, symbols that were produced by an AffineApplyOp needed to be promoted to dims and reordered for the mathematical composition to be valid.

Since then, single result AffineApplyOp became the law of the land but the original assumptions were not revisited.

This revision revisits these assumptions and retires AffineApplyNormalizer.

Differential Revision: https://reviews.llvm.org/D94920

show more ...

[mlir] Remove the use of "kinds" from Attributes and Types

This greatly simplifies a large portion of the underlying infrastructure, allows for lookups of singleton classes to be much more efficient

[mlir] Remove the use of "kinds" from Attributes and Types

This greatly simplifies a large portion of the underlying infrastructure, allows for lookups of singleton classes to be much more efficient and always thread-safe(no locking). As a result of this, the dialect symbol registry has been removed as it is no longer necessary.

For users broken by this change, an alert was sent out(https://llvm.discourse.group/t/removing-kinds-from-attributes-and-types) that helps prevent a majority of the breakage surface area. All that should be necessary, if the advice in that alert was followed, is removing the kind passed to the ::get methods.

Differential Revision: https://reviews.llvm.org/D86121

show more ...

[mlir] Refactor StorageUniquer to require registration of possible storage types

This allows for bucketing the different possible storage types, with each bucket having its own allocator/mutex/insta

[mlir] Refactor StorageUniquer to require registration of possible storage types

This allows for bucketing the different possible storage types, with each bucket having its own allocator/mutex/instance map. This greatly reduces the amount of lock contention when multi-threading is enabled. On some non-trivial .mlir modules (>300K operations), this led to a compile time decrease of a single conversion pass by around half a second(>25%).

Differential Revision: https://reviews.llvm.org/D82596

show more ...

[mlir] Introduce AffineMinSCF folding as a pattern

This revision adds a folding pattern to replace affine.min ops by the actual min value, when it can be determined statically from the strides and b

[mlir] Introduce AffineMinSCF folding as a pattern

This revision adds a folding pattern to replace affine.min ops by the actual min value, when it can be determined statically from the strides and bounds of enclosing scf loop .

This matches the type of expressions that Linalg produces during tiling and simplifies boundary checks. For now Linalg depends both on Affine and SCF but they do not depend on each other, so the pattern is added there.
In the future this will move to a more appropriate place when it is determined.

The canonicalization of AffineMinOp operations in the context of enclosing scf.for and scf.parallel proceeds by:
1. building an affine map where uses of the induction variable of a loop
are replaced by `%lb + %step * floordiv(%iv - %lb, %step)` expressions.
2. checking if any of the results of this affine map divides all the other
results (in which case it is also guaranteed to be the min).
3. replacing the AffineMinOp by the result of (2).

The algorithm is functional in simple parametric tiling cases by using semi-affine maps. However simplifications of such semi-affine maps are not yet available and the canonicalization does not succeed yet.

Differential Revision: https://reviews.llvm.org/D82009

show more ...

[MLIR] Simplify semi-affine expressions

Simplify semi-affine expression for the operations like ceildiv,
floordiv and modulo by any given symbol by checking divisibilty by that
symbol.

Some propert

[MLIR] Simplify semi-affine expressions

Simplify semi-affine expression for the operations like ceildiv,
floordiv and modulo by any given symbol by checking divisibilty by that
symbol.

Some properties used in simplification are:

1) Commutative property of the floordiv and ceildiv:
((expr1 floordiv expr2) floordiv expr3 ) = ((expr1 floordiv expr3) floordiv expr2)
((expr1 ceildiv expr2) ceildiv expr3 ) = ((expr1 ceildiv expr3) ceildiv expr2)

While simplification if operations are different no simplification is
possible as there is no property that simplify expressions like these:
((expr1 ceildiv expr2) floordiv expr3) or ((expr1 floordiv expr2)
ceildiv expr3).

2) If both expr1 and expr2 are divisible by the expr3 then:
(expr1 % expr2) / expr3 = ((expr1 / expr3) % (expr2 / expr3))
where / is divide symbol.

3) If expr1 is divisible by expr2 then expr1 % expr2 = 0.

Signed-off-by: Yash Jain <[email protected]>

Differential Revision: https://reviews.llvm.org/D84920

show more ...

[mlir] Added support for symbols inside linalg.generic and map concatenation

This commit adds functionality needed for implementation of convolutions with
linalg.generic op. Since linalg.generic rig

[mlir] Added support for symbols inside linalg.generic and map concatenation

This commit adds functionality needed for implementation of convolutions with
linalg.generic op. Since linalg.generic right now expects indexing maps to be
just permutations, offset indexing needed in convolutions is not possible.
Therefore in this commit we address the issue by adding support for symbols inside
indexing maps which enables more advanced indexing. The upcoming commit will
solve the problem of computing loop bounds from such maps.

Differential Revision: https://reviews.llvm.org/D83158

show more ...

[mlir][NFC] Remove usernames and google bug numbers from TODO comments.

These were largely leftover from when MLIR was a google project, and don't really follow LLVM guidelines.

[mlir][NFC] Remove the STLExtras.h header file now that it has been merged into LLVM.

Now that no more utilities exist within, this file can be deleted.

Differential Revision: https://reviews.llvm.

[mlir][NFC] Remove the STLExtras.h header file now that it has been merged into LLVM.

Now that no more utilities exist within, this file can be deleted.

Differential Revision: https://reviews.llvm.org/D78079

show more ...

[mlir][Linalg] Create a tool to generate named Linalg ops from a Tensor Comprehensions-like specification.

Summary:

This revision adds a tool that generates the ODS and C++ implementation for "name

[mlir][Linalg] Create a tool to generate named Linalg ops from a Tensor Comprehensions-like specification.

Summary:

This revision adds a tool that generates the ODS and C++ implementation for "named" Linalg ops according to the [RFC discussion](https://llvm.discourse.group/t/rfc-declarative-named-ops-in-the-linalg-dialect/745).

While the mechanisms and language aspects are by no means set in stone, this revision allows connecting the pieces end-to-end from a mathematical-like specification.

Some implementation details and short-term decisions taken for the purpose of bootstrapping and that are not set in stone include:

1. using a "[Tensor Comprehension](https://arxiv.org/abs/1802.04730)-inspired" syntax
2. implicit and eager discovery of dims and symbols when parsing
3. using EDSC ops to specify the computation (e.g. std_addf, std_mul_f, ...)

A followup revision will connect this tool to tablegen mechanisms and allow the emission of named Linalg ops that automatically lower to various loop forms and run end to end.

For the following "Tensor Comprehension-inspired" string:

```
def batch_matmul(A: f32(Batch, M, K), B: f32(K, N)) -> (C: f32(Batch, M, N)) {
C(b, m, n) = std_addf<k>(std_mulf(A(b, m, k), B(k, n)));
}
```

With -gen-ods-decl=1, this emits (modulo formatting):

```
def batch_matmulOp : LinalgNamedStructured_Op<"batch_matmul", [
NInputs<2>,
NOutputs<1>,
NamedStructuredOpTraits]> {
let arguments = (ins Variadic<LinalgOperand>:$views);
let results = (outs Variadic<AnyRankedTensor>:$output_tensors);
let extraClassDeclaration = [{
llvm::Optional<SmallVector<StringRef, 8>> referenceIterators();
llvm::Optional<SmallVector<AffineMap, 8>> referenceIndexingMaps();
void regionBuilder(ArrayRef<BlockArgument> args);
}];
let hasFolder = 1;
}
```

With -gen-ods-impl, this emits (modulo formatting):

```
llvm::Optional<SmallVector<StringRef, 8>> batch_matmul::referenceIterators() {
return SmallVector<StringRef, 8>{ getParallelIteratorTypeName(),
getParallelIteratorTypeName(),
getParallelIteratorTypeName(),
getReductionIteratorTypeName() };
}
llvm::Optional<SmallVector<AffineMap, 8>> batch_matmul::referenceIndexingMaps()
{
MLIRContext *context = getContext();
AffineExpr d0, d1, d2, d3;
bindDims(context, d0, d1, d2, d3);
return SmallVector<AffineMap, 8>{
AffineMap::get(4, 0, {d0, d1, d3}),
AffineMap::get(4, 0, {d3, d2}),
AffineMap::get(4, 0, {d0, d1, d2}) };
}
void batch_matmul::regionBuilder(ArrayRef<BlockArgument> args) {
using namespace edsc;
using namespace intrinsics;
ValueHandle _0(args[0]), _1(args[1]), _2(args[2]);

ValueHandle _4 = std_mulf(_0, _1);
ValueHandle _5 = std_addf(_2, _4);
(linalg_yield(ValueRange{ _5 }));
}
```

Differential Revision: https://reviews.llvm.org/D77067

show more ...

Affine expr simplification for add of const multiple of same expression

- Detect "c_1 * expr + c_2 * expr" as (c_1 + c_2) * expr
- subsumes things like 'expr - expr' and "expr * -1 + expr" as 0.
- c

Affine expr simplification for add of const multiple of same expression

- Detect "c_1 * expr + c_2 * expr" as (c_1 + c_2) * expr
- subsumes things like 'expr - expr' and "expr * -1 + expr" as 0.
- change AffineConstantExpr ctor to allow default null init

Signed-off-by: Uday Bondhugula <[email protected]>

Differential Revision: https://reviews.llvm.org/D76233

show more ...

[mlir][NFC] remove stray decl of toAffineExpr, rename for readability

Summary:
- remove stray toAffineExpr decl in affine analysis (name duplicate of
mlir::toAffineExpr)

- rename mlir::toAffineEx

[mlir][NFC] remove stray decl of toAffineExpr, rename for readability

Summary:
- remove stray toAffineExpr decl in affine analysis (name duplicate of
mlir::toAffineExpr)

- rename mlir::toAffineExpr for better readability

- related NFC changes

Signed-off-by: Uday Bondhugula <[email protected]>

Differential Revision: https://reviews.llvm.org/D75694

show more ...

[MLIR][Affine][NFC] Remove obsolete and ambiguous definitions

Summary:
Looks like a refactor that was never completed.

This change removes some unused and ambiguous definitions.

Reviewed By: bondh

[MLIR][Affine][NFC] Remove obsolete and ambiguous definitions

Summary:
Looks like a refactor that was never completed.

This change removes some unused and ambiguous definitions.

Reviewed By: bondhugula, nicolasvasilache, rriddle

Differential Revision: https://reviews.llvm.org/D75586

show more ...