[mlir] Fix verification order of nested ops.

In order to increase parallism, certain ops with regions and have the
IsIsolatedFromAbove trait will have their verification delayed. That
means the regi

[mlir] Fix verification order of nested ops.

In order to increase parallism, certain ops with regions and have the
IsIsolatedFromAbove trait will have their verification delayed. That
means the region verifier may access the invalid ops and may lead to a
crash.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D122771

show more ...

[mlir] Fix missing verification after running an OpToOpAdaptorPass

The current decision of when to run the verifier is running on the
assumption that nested passes can't affect the validity of the p

[mlir] Fix missing verification after running an OpToOpAdaptorPass

The current decision of when to run the verifier is running on the
assumption that nested passes can't affect the validity of the parent
operation, which isn't true. Parent operations may attach any number
of constraints on nested operations, which may not necessarily be
captured (or shouldn't be captured) at a smaller granularity.

This commit rectifies this by properly running the verifier after an
OpToOpAdaptor pass. To avoid an explosive increase in compile time,
we only run verification on the parent operation itself. To do this, a
flag to mlir::verify is added to avoid recursive verification if it isn't
desired.

Fixes #54288

Differential Revision: https://reviews.llvm.org/D121836

show more ...

[mlir] Support verification order (2/3)

This change gives explicit order of verifier execution and adds
`hasRegionVerifier` and `verifyWithRegions` to increase the granularity
of verifie

[mlir] Support verification order (2/3)

This change gives explicit order of verifier execution and adds
`hasRegionVerifier` and `verifyWithRegions` to increase the granularity
of verifier classification. The orders are as below,

1. InternalOpTrait will be verified first, they can be run independently.
2. `verifyInvariants` which is constructed by ODS, it verifies the type,
attributes, .etc.
3. Other Traits/Interfaces that have marked their verifier as
`verifyTrait` or `verifyWithRegions=0`.
4. Custom verifier which is defined in the op and has marked
`hasVerifier=1`

If an operation has regions, then it may have the second phase,

5. Traits/Interfaces that have marked their verifier as
`verifyRegionTrait` or
`verifyWithRegions=1`. This implies the verifier needs to access the
operations in its regions.
6. Custom verifier which is defined in the op and has marked
`hasRegionVerifier=1`

Note that the second phase will be run after the operations in the
region are verified. Based on the verification order, you will be able to
avoid verifying duplicate things.

Reviewed By: Mogball

Differential Revision: https://reviews.llvm.org/D116789

show more ...

Apply clang-tidy fixes for performance-for-range-copy to MLIR (NFC)

Adjust "end namespace" comment in MLIR to match new agree'd coding style

See D115115 and this mailing list discussion:
https://lists.llvm.org/pipermail/llvm-dev/2021-December/154199.html

Differenti

Adjust "end namespace" comment in MLIR to match new agree'd coding style

See D115115 and this mailing list discussion:
https://lists.llvm.org/pipermail/llvm-dev/2021-December/154199.html

Differential Revision: https://reviews.llvm.org/D115309

show more ...

[mlir] Convert NamedAttribute to be a class

NamedAttribute is currently represented as an std::pair, but this
creates an extremely clunky .first/.second API. This commit
converts it to a class, with

[mlir] Convert NamedAttribute to be a class

NamedAttribute is currently represented as an std::pair, but this
creates an extremely clunky .first/.second API. This commit
converts it to a class, with better accessors (getName/getValue)
and also opens the door for more convenient API in the future.

Differential Revision: https://reviews.llvm.org/D113956

show more ...

[mlir] Refactor AbstractOperation and OperationName

The current implementation is quite clunky; OperationName stores either an Identifier
or an AbstractOperation that corresponds to an operation. Th

[mlir] Refactor AbstractOperation and OperationName

The current implementation is quite clunky; OperationName stores either an Identifier
or an AbstractOperation that corresponds to an operation. This has several problems:

* OperationNames created before and after an operation are registered are different
* Accessing the identifier name/dialect/etc. from an OperationName are overly branchy
- they need to dyn_cast a PointerUnion to check the state

This commit refactors this such that we create a single information struct for every
operation name, even operations that aren't registered yet. When an OperationName is
created for an unregistered operation, we only populate the name field. When the
operation is registered, we populate the remaining fields. With this we now have two
new classes: OperationName and RegisteredOperationName. These both point to the
same underlying operation information struct, but only RegisteredOperationName can
assume that the operation is actually registered. This leads to a much cleaner API, and
we can also move some AbstractOperation functionality directly to OperationName.

Differential Revision: https://reviews.llvm.org/D114049

show more ...

[mlir] Replace usages of Identifier with StringAttr

Identifier and StringAttr essentially serve the same purpose, i.e. to hold a string value. Keeping these seemingly identical pieces of functionali

[mlir] Replace usages of Identifier with StringAttr

Identifier and StringAttr essentially serve the same purpose, i.e. to hold a string value. Keeping these seemingly identical pieces of functionality separate has caused problems in certain situations:

* Identifier has nice accessors that StringAttr doesn't
* Identifier can't be used as an Attribute, meaning strings are often duplicated between Identifier/StringAttr (e.g. in PDL)

The only thing that Identifier has that StringAttr doesn't is support for caching a dialect that is referenced by the string (e.g. dialect.foo). This functionality is added to StringAttr, as this is useful for StringAttr in generally the same ways it was useful for Identifier.

Differential Revision: https://reviews.llvm.org/D113536

show more ...

Remove unused llvm/Support/Parallel.h from MLIR (NFC)

This header aren't needed anymore: MLIR is using a thread pool
injected in the context instead of a global one.

Defend early against operation created without a registered dialect

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D105961

Revert "Defend early against operation created without a registered dialect"

This reverts commit 58018858e887320e2432e2e00ace13273b8a1f29.

The Python bindings test are broken.

Defend early against operation created without a registered dialect

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D105961

[mlir] Add a ThreadPool to MLIRContext and refactor MLIR threading usage

This revision refactors the usage of multithreaded utilities in MLIR to use a common
thread pool within the MLIR context, in

[mlir] Add a ThreadPool to MLIRContext and refactor MLIR threading usage

This revision refactors the usage of multithreaded utilities in MLIR to use a common
thread pool within the MLIR context, in addition to a new utility that makes writing
multi-threaded code in MLIR less error prone. Using a unified thread pool brings about
several advantages:

* Better thread usage and more control
We currently use the static llvm threading utilities, which do not allow multiple
levels of asynchronous scheduling (even if there are open threads). This is due to
how the current TaskGroup structure works, which only allows one truly multithreaded
instance at a time. By having our own ThreadPool we gain more control and flexibility
over our job/thread scheduling, and in a followup can enable threading more parts of
the compiler.

* The static nature of TaskGroup causes issues in certain configurations
Due to the static nature of TaskGroup, there have been quite a few problems related to
destruction that have caused several downstream projects to disable threading. See
D104207 for discussion on some related fallout. By having a ThreadPool scoped to
the context, we don't have to worry about destruction and can ensure that any
additional MLIR thread usage ends when the context is destroyed.

Differential Revision: https://reviews.llvm.org/D104516

show more ...

Partial rollback: Disable MLIR verifier parallelism.

Deadlocks have been found in several downstream projects as noted on the original patch: https://reviews.llvm.org/D104207

Disabling pending full

Partial rollback: Disable MLIR verifier parallelism.

Deadlocks have been found in several downstream projects as noted on the original patch: https://reviews.llvm.org/D104207

Disabling pending full root cause analysis.

Differential Revision: https://reviews.llvm.org/D104570

show more ...

[Verifier] Parallelize verification and dom checking. NFC.

This changes the outer verification loop to not recurse into
IsolatedFromAbove operations - instead return them up to a place
where a para

[Verifier] Parallelize verification and dom checking. NFC.

This changes the outer verification loop to not recurse into
IsolatedFromAbove operations - instead return them up to a place
where a parallel for loop can process them all in parallel. This
also changes Dominance checking to happen on IsolatedFromAbove
chunks of the region tree, which makes it easy to fold operation
and dominance verification into a single simple parallel regime.

This speeds up firtool in CIRCT from ~40s to 31s on a large
testcase in -verify-each mode (the default). The .fir parser and
module passes in particular benefit from this - FModule passes
(roughly analogous to function passes) were already running the
verifier in parallel as part of the pass manager. This allows
the whole-module passes to verify their enclosed functions /
FModules in parallel.

-verify-each mode is still faster (26.3s on the same testcase),
but we do expect the verifier to take *some* time.

Differential Revision: https://reviews.llvm.org/D104207

show more ...

[DominanceInfo] Make the ctor take a defaulted value for the operand. NFC.

This allows it to be default constructible, which makes sense given it
ignores the operand.

Revert "[Verifier] Speed up and parallelize dominance checking. NFC"

This reverts commit 08664d005c02003180371049b19c7e5d01541c58, which according to
https://reviews.llvm.org/D103373 was pushed acc

Revert "[Verifier] Speed up and parallelize dominance checking. NFC"

This reverts commit 08664d005c02003180371049b19c7e5d01541c58, which according to
https://reviews.llvm.org/D103373 was pushed accidentally, and I believe it
causes timeouts in some internal mlir tests.

show more ...

Add missing header <atomic> in lib/IR/Verifier.cpp (NFC)

Fix the build on some platform.

[Verifier] Speed up and parallelize dominance checking. NFC

One of the key algorithms used in the "mlir::verify(op)" method is the
dominance checker, which ensures that operand values properly domi

[Verifier] Speed up and parallelize dominance checking. NFC

One of the key algorithms used in the "mlir::verify(op)" method is the
dominance checker, which ensures that operand values properly dominate
the operations that use them.

The MLIR dominance implementation has a number of algorithmic problems,
and is not really set up in general to answer dense queries: it's constant
factors are really slow with multiple map lookups and scans, even in the
easy cases. Furthermore, when calling mlir::verify(module) or some other
high level operation, it makes sense to parallelize the dominator
verification of all the functions within the module.

This patch has a few changes to enact this:
1) It splits dominance checking into "IsolatedFromAbove" units. Instead
of building a monolithic DominanceInfo for everything in a module,
for example, it checks dominance for the module to all the functions
within it (noop, since there are no operands at this level) then each
function gets their own DominanceInfo for each of their scope.
2) It adds the ability for mlir::DominanceInfo (and post dom) to be
constrained to an IsolatedFromAbove region. There is no reason to
recurse into IsolatedFromAbove regions since use/def relationships
can't span this region anyway. This is already checked by the time
the verifier gets here.
3) It avoids querying DominanceInfo for trivial checks (e.g. intra Block
references) to eliminate constant factor issues).
4) It switches to lazily constructing DominanceInfo because the trivial
check case handles the vast majority of the cases and avoids
constructing DominanceInfo entirely in some cases (e.g. at the module
level or for many Regions's that contain a single Block).
5) It parallelizes analysis of collections IsolatedFromAbove operations,
e.g. each of the functions within a Module.

All together this is more than a 10% speedup on `firtool` in circt on a
large design when run in -verify-each mode (our default) since the verifier
is invoked after each pass.

Still todo is to parallelize the main verifier pass. I decided to split
this out to its own thing since this patch is already large-ish.

Differential Revision: https://reviews.llvm.org/D103373

show more ...

[Verifier] Inline a method to simplify the code in preparation for bigger changes, NFC.

Differential Revision: https://reviews.llvm.org/D103365

[Verifier] Slightly refactor code to reduce duplication, NFC.

[Verifier] Tidy up the code a bit, NFC.

This tidies up the code a bit:
* Eliminate the ctx member, which doesn't need to be stored.
* Rename verify(Operation) to make it more clear that it is
d

[Verifier] Tidy up the code a bit, NFC.

This tidies up the code a bit:
* Eliminate the ctx member, which doesn't need to be stored.
* Rename verify(Operation) to make it more clear that it is
doing more than verifyOperation and that the dominance check
isn't being done multiple times.
* Rename mayNotHaveTerminator which was confusing about whether
it wasn't known whether it had a terminator, when it is really
about whether it is legal to have a terminator.
* Some minor optimizations: don't check for RegionKindInterface
if there are no regions. Don't do two passes over the
operations in a block in OperationVerifier::verifyDominance when
one will do.

The optimizations are actually a measurable (but minor) win in some
CIRCT cases.

Differential Revision: https://reviews.llvm.org/D101267

show more ...

Define a `NoTerminator` traits that allows operations with a single block region to not provide a terminator

In particular for Graph Regions, the terminator needs is just a
historical artifact of th

Define a `NoTerminator` traits that allows operations with a single block region to not provide a terminator

In particular for Graph Regions, the terminator needs is just a
historical artifact of the generalization of MLIR from CFG region.
Operations like Module don't need a terminator, and before Module
migrated to be an operation with region there wasn't any needed.

To validate the feature, the ModuleOp is migrated to use this trait and
the ModuleTerminator operation is deleted.

This patch is likely to break clients, if you're in this case:

- you may iterate on a ModuleOp with `getBody()->without_terminator()`,
the solution is simple: just remove the ->without_terminator!
- you created a builder with `Builder::atBlockTerminator(module_body)`,
just use `Builder::atBlockEnd(module_body)` instead.
- you were handling ModuleTerminator: it isn't needed anymore.
- for generic code, a `Block::mayNotHaveTerminator()` may be used.

Differential Revision: https://reviews.llvm.org/D98468

show more ...

[mlir] Simplify various pieces of code now that Identifier has access to the Context/Dialect

This also exposed a bug in Dialect loading where it was not correctly identifying identifiers that had th

[mlir] Simplify various pieces of code now that Identifier has access to the Context/Dialect

This also exposed a bug in Dialect loading where it was not correctly identifying identifiers that had the dialect namespace as a prefix.

Differential Revision: https://reviews.llvm.org/D97431

show more ...

[mlir][IR] Remove the concept of `OperationProperties`

These properties were useful for a few things before traits had a better integration story, but don't really carry their weight well these days

[mlir][IR] Remove the concept of `OperationProperties`

These properties were useful for a few things before traits had a better integration story, but don't really carry their weight well these days. Most of these properties are already checked via traits in most of the code. It is better to align the system around traits, and improve the performance/cost of traits in general.

Differential Revision: https://reviews.llvm.org/D96088

show more ...