[mlir] (NFC) run clang-format on all files

[mlir][CSE] Add ability to remove commutative operations

This patch takes advantage of the Commutative trait on operation
to remove identical commutative operations where the operands are swapped.

[mlir][CSE] Add ability to remove commutative operations

This patch takes advantage of the Commutative trait on operation
to remove identical commutative operations where the operands are swapped.

The second operation below can be removed since `arith.addi` is commutative.
```
%1 = arith.addi %a, %b : i32
%2 = arith.addi %b, %a : i32
```

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D123492

show more ...

[mlir][NFC] Add a using for llvm::BitVector to LLVM.h

BitVector is becoming widespread enough that we should add a proper using.

Differential Revision: https://reviews.llvm.org/D118290

[mlir] Optimize OperationName construction and usage

When constructing an OperationName, the overwhelming majority of
cases are from registered operations. This revision adds a non-locked
lookup int

[mlir] Optimize OperationName construction and usage

When constructing an OperationName, the overwhelming majority of
cases are from registered operations. This revision adds a non-locked
lookup into the currently registered operations, which prevents locking
in the common case. This revision also optimizes several uses of
RegisteredOperationName that expect the operation to be registered,
e.g. such as in OpBuilder.

These changes provides a reasonable speedup (5-10%) in some
compilations, especially on platforms where locking is expensive.

Differential Revision: https://reviews.llvm.org/D117187

show more ...

Fix clang-tidy issues in mlir/ (NFC)

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D115956

[mlir] Convert NamedAttribute to be a class

NamedAttribute is currently represented as an std::pair, but this
creates an extremely clunky .first/.second API. This commit
converts it to a class, with

[mlir] Convert NamedAttribute to be a class

NamedAttribute is currently represented as an std::pair, but this
creates an extremely clunky .first/.second API. This commit
converts it to a class, with better accessors (getName/getValue)
and also opens the door for more convenient API in the future.

Differential Revision: https://reviews.llvm.org/D113956

show more ...

[mlir][NFC] Replace references to Identifier with StringAttr

This is part of the replacement of Identifier with StringAttr.

Differential Revision: https://reviews.llvm.org/D113953

[mlir] Replace usages of Identifier with StringAttr

Identifier and StringAttr essentially serve the same purpose, i.e. to hold a string value. Keeping these seemingly identical pieces of functionali

[mlir] Replace usages of Identifier with StringAttr

Identifier and StringAttr essentially serve the same purpose, i.e. to hold a string value. Keeping these seemingly identical pieces of functionality separate has caused problems in certain situations:

* Identifier has nice accessors that StringAttr doesn't
* Identifier can't be used as an Attribute, meaning strings are often duplicated between Identifier/StringAttr (e.g. in PDL)

The only thing that Identifier has that StringAttr doesn't is support for caching a dialect that is referenced by the string (e.g. dialect.foo). This functionality is added to StringAttr, as this is useful for StringAttr in generally the same ways it was useful for Identifier.

Differential Revision: https://reviews.llvm.org/D113536

show more ...

[mlir][core] Slightly improved attribute lookup

- String binary search does 1 less string comparison
- Identifier linear scan on large attribute list is switched to string binary search

Reviewed By

[mlir][core] Slightly improved attribute lookup

- String binary search does 1 less string comparison
- Identifier linear scan on large attribute list is switched to string binary search

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D112970

show more ...

[mlir] Move the Operation OperandStorage to the first trailing object

The main benefits of this change are faster access to operands
(no need to compute the offset, as it is now right after the
oper

[mlir] Move the Operation OperandStorage to the first trailing object

The main benefits of this change are faster access to operands
(no need to compute the offset, as it is now right after the
operation), simpler code(no need to manage a lot of the "is the
operand storage trailing" logic we had to before). The major
downside to this though, is that operand holding operations now
grow in size by 1 word (as no matter how we do this change, there
will need to be some additional book keeping).

Differential Revision: https://reviews.llvm.org/D111695

show more ...

[mlir:DialectConversion] Restructure how argument/target materializations get invoked

The current implementation invokes materializations
whenever an input operand does not have a mapping for the
de

[mlir:DialectConversion] Restructure how argument/target materializations get invoked

The current implementation invokes materializations
whenever an input operand does not have a mapping for the
desired type, i.e. it requires materialization at the earliest possible
point. This conflicts with goal of dialect conversion (and also the
current documentation) which states that a materialization is only
required if the materialization is supposed to persist after the
conversion process has finished.

This revision refactors this such that whenever a target
materialization "might" be necessary, we insert an
unrealized_conversion_cast to act as a temporary materialization.
This allows for deferring the invocation of the user
materialization hooks until the end of the conversion process,
where we actually have a better sense if it's actually
necessary. This has several benefits:

* In some cases a target materialization hook is no longer
necessary
When performing a full conversion, there are some situations
where a temporary materialization is necessary. Moving forward,
these users won't need to provide any target materializations,
as the temporary materializations do not require the user to
provide materialization hooks.

* getRemappedValue can now handle values that haven't been
converted yet
Before this commit, it wasn't well supported to get the remapped
value of a value that hadn't been converted yet (making it
difficult/impossible to convert multiple operations in many
situations). This commit updates getRemappedValue to properly
handle this case by inserting temporary materializations when
necessary.

Another code-health related benefit is that with this change we
can move a majority of the complexity related to materializations
to the end of the conversion process, instead of handling adhoc
while conversion is happening.

Differential Revision: https://reviews.llvm.org/D111620

show more ...

[mlir] Move the Operation use iteration utilities to ResultRange

This allows for iterating and interacting with the uses of a specific subset of
results as opposed to just the full range.

Different

[mlir] Move the Operation use iteration utilities to ResultRange

This allows for iterating and interacting with the uses of a specific subset of
results as opposed to just the full range.

Differential Revision: https://reviews.llvm.org/D108586

show more ...

[mlir] Add assertion in NamedAttrList to prevent adding null attributes

Differential Revision: https://reviews.llvm.org/D108570

[mlir] Add support for VariadicOfVariadic operands

This revision adds native ODS support for VariadicOfVariadic operand
groups. An example of this is the SwitchOp, which has a variadic number
of nes

[mlir] Add support for VariadicOfVariadic operands

This revision adds native ODS support for VariadicOfVariadic operand
groups. An example of this is the SwitchOp, which has a variadic number
of nested operand ranges for each of the case statements, where the
number of case statements is variadic. Builtin ODS support allows for
generating proper accessors for the nested operand ranges, builder
support, and declarative format support. VariadicOfVariadic operands
are supported by providing a segment attribute to use to store the
operand groups, mapping similarly to the AttrSizedOperand trait
(but with a user defined attribute name).

`build` methods for VariadicOfVariadic operand expect inputs of the
form `ArrayRef<ValueRange>`. Accessors for the variadic ranges
return a new `OperandRangeRange` type, which represents a
contiguous range of `OperandRange`. In the declarative assembly
format, VariadicOfVariadic operands and types are by default
formatted as a comma delimited list of value lists:
`(<value>, <value>), (), (<value>)`.

Differential Revision: https://reviews.llvm.org/D107774

show more ...

Implement recursive support into OperationEquivalence::isEquivalentTo()

This allows to use OperationEquivalence to track structural comparison for equality
between two operations.

Differential Revi

Implement recursive support into OperationEquivalence::isEquivalentTo()

This allows to use OperationEquivalence to track structural comparison for equality
between two operations.

Differential Revision: https://reviews.llvm.org/D106422

show more ...

[mlir] Add workaround for false positive in -Wfree-nonheap-object

Restore 499571ea835daf786626a0db1e12f890b6cd8f8d
reverted by 0082764605cc0e7e0363a41ffa77d214c3157aa6.

A compiler slightly older th

[mlir] Add workaround for false positive in -Wfree-nonheap-object

Restore 499571ea835daf786626a0db1e12f890b6cd8f8d
reverted by 0082764605cc0e7e0363a41ffa77d214c3157aa6.

A compiler slightly older than
"[clang][Sema] removes -Wfree-nonheap-object reference param false positive"
may report the false positive.
We need to retain the workaround a bit longer so that such compilers
can be used to compile MLIR in a warning-free way.

show more ...

Revert "Add workaround for false positive in -Wfree-nonheap-object"

This reverts commit 499571ea835daf786626a0db1e12f890b6cd8f8d.

[mlir] Store the flag for dynamic operand storage in the low bits

It is currently stored in the high bits, which is disallowed on certain
platforms (e.g. android). This revision switches the represe

[mlir] Store the flag for dynamic operand storage in the low bits

It is currently stored in the high bits, which is disallowed on certain
platforms (e.g. android). This revision switches the representation to use
the low bits instead, fixing crashes/breakages on those platforms.

Differential Revision: https://reviews.llvm.org/D101969

show more ...

Add workaround for false positive in -Wfree-nonheap-object

[mlir][IR] Add an Operation::eraseOperands that supports batch erasure

This method allows for removing multiple disjoint operands at once, reducing the need to erase operands individually (which res

[mlir][IR] Add an Operation::eraseOperands that supports batch erasure

This method allows for removing multiple disjoint operands at once, reducing the need to erase operands individually (which results in shifting the operand list).

Differential Revision: https://reviews.llvm.org/D98290

show more ...

[mlir][IR][NFC] Move a majority of the builtin attributes to ODS

Now that attributes can be generated using ODS, we can move the builtin attributes as well. This revision removes a majority of the b

[mlir][IR][NFC] Move a majority of the builtin attributes to ODS

Now that attributes can be generated using ODS, we can move the builtin attributes as well. This revision removes a majority of the builtin attributes with a few left for followup revisions. The attributes moved to ODS in this revision are: AffineMapAttr, ArrayAttr, DictionaryAttr, IntegerSetAttr, StringAttr, SymbolRefAttr, TypeAttr, and UnitAttr.

Differential Revision: https://reviews.llvm.org/D97591

show more ...

[mlir][IR] Refactor the internal implementation of Value

The current implementation of Value involves a pointer int pair with several different kinds of owners, i.e. BlockArgumentImpl*, Operation *,

[mlir][IR] Refactor the internal implementation of Value

The current implementation of Value involves a pointer int pair with several different kinds of owners, i.e. BlockArgumentImpl*, Operation *, TrailingOpResult*. This design arose from the desire to save memory overhead for operations that have a very small number of results (generally 0-2). There are, unfortunately, many problematic aspects of the current implementation that make Values difficult to work with or just inefficient.

Operation result types are stored as a separate array on the Operation. This is very inefficient for many reasons: we use TupleType for multiple results, which can lead to huge amounts of memory usage if multi-result operations change types frequently(they do). It also means that simple methods like Value::getType/Value::setType now require complex logic to get to the desired type.

Value only has one pointer bit free, severely limiting the ability to use it in things like PointerUnion/PointerIntPair. Given that we store the kind of a Value along with the "owner" pointer, we only leave one bit free for users of Value. This creates situations where we end up nesting PointerUnions to be able to use Value in one.

As noted above, most of the methods in Value need to branch on at least 3 different cases which is both inefficient, possibly error prone, and verbose. The current storage of results also creates problems for utilities like ValueRange/TypeRange, which want to efficiently store base pointers to ranges (of which Operation* isn't really useful as one).

This revision greatly simplifies the implementation of Value by the introduction of a new ValueImpl class. This class contains all of the state shared between all of the various derived value classes; i.e. the use list, the type, and the kind. This shared implementation class provides several large benefits:

* Most of the methods on value are now branchless, and often one-liners.

* The "kind" of the value is now stored in ValueImpl instead of Value
This frees up all of Value's pointer bits, allowing for users to take full advantage of PointerUnion/PointerIntPair/etc. It also allows for storing more operation results as "inline", 6 now instead of 2, freeing up 1 word per new inline result.

* Operation result types are now stored in the result, instead of a side array
This drops the size of zero-result operations by 1 word. It also removes the memory crushing use of TupleType for operations results (which could lead up to hundreds of megabytes of "dead" TupleTypes in the context). This also allowed restructured ValueRange, making it simpler and one word smaller.

This revision does come with two conceptual downsides:
* Operation::getResultTypes no longer returns an ArrayRef<Type>
This conceptually makes some usages slower, as the iterator increment is slightly more complex.
* OpResult::getOwner is slightly more expensive, as it now requires a little bit of arithmetic

From profiling, neither of the conceptual downsides have resulted in any perceivable hit to performance. Given the advantages of the new design, most compiles are slightly faster.

Differential Revision: https://reviews.llvm.org/D97804

show more ...

[mlir] Remove the MutableDictionaryAttr class

This class used to serve a few useful purposes:
* Allowed containing a null DictionaryAttr
* Provided some simple mutable API around a DictionaryAttr

T

[mlir] Remove the MutableDictionaryAttr class

This class used to serve a few useful purposes:
* Allowed containing a null DictionaryAttr
* Provided some simple mutable API around a DictionaryAttr

The first of which is no longer an issue now that there is much better caching support for attributes in general, and a cache in the context for empty dictionaries. The second results in more trouble than it's worth because it mutates the internal dictionary on every action, leading to a potentially large number of dictionary copies. NamedAttrList is a much better alternative for the second use case, and should be modified as needed to better fit it's usage as a DictionaryAttrBuilder.

Differential Revision: https://reviews.llvm.org/D93442

show more ...

[mlir][IR] Move the storage for results to before the Operation instead of after.

Trailing objects are really nice for storing additional data inline with the main class, and is something that we he

[mlir][IR] Move the storage for results to before the Operation instead of after.

Trailing objects are really nice for storing additional data inline with the main class, and is something that we heavily take advantage of for Operation(and many other classes). To get the address of the inline data you need to compute the address by doing some pointer arithmetic taking into account any objects stored before the object you want to access. Most classes keep the count of the number of objects, so this is relatively cheap to compute. This is not the case for results though, which have two different types(inline and trailing) that are not necessarily as cheap to compute as the count for other objects. This revision moves the storage for results to before the operation and stores them in reverse order. This allows for getting results to still be very fast given that they are never iterated directly in order, and also greatly improves the speed when accessing the other trailing objects of an operation(operands/regions/blocks/etc).

This reduced compile time when compiling a decently sized mlir module by about ~400ms, or 2.17s -> 1.76s.

Differential Revision: https://reviews.llvm.org/D92687

show more ...

[mlir][Types][NFC] Move all of the builtin Type classes to BuiltinTypes.h

This is part of a larger refactoring the better congregates the builtin structures under the BuiltinDialect. This also remov

[mlir][Types][NFC] Move all of the builtin Type classes to BuiltinTypes.h

This is part of a larger refactoring the better congregates the builtin structures under the BuiltinDialect. This also removes the problematic "standard" naming that clashes with the "standard" dialect, which is not defined within IR/. A temporary forward is placed in StandardTypes.h to allow time for downstream users to replaced references.

Differential Revision: https://reviews.llvm.org/D92435

show more ...