[async] Get the number of worker threads from the runtime.

Reviewed By: ezhulenev

Differential Revision: https://reviews.llvm.org/D117751

Fix more clang-tidy cleanups in mlir/ (NFC)

Fix clang-tidy issues in mlir/ (NFC)

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D115956

[mlir] AsyncRuntime: use int64_t for ref counting operations

Workaround for SystemZ ABI problem: https://bugs.llvm.org/show_bug.cgi?id=51898

Reviewed By: ftynse

Differential Revision: https://revi

[mlir] AsyncRuntime: use int64_t for ref counting operations

Workaround for SystemZ ABI problem: https://bugs.llvm.org/show_bug.cgi?id=51898

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D110550

show more ...

[mlir:Async] Add the size parameter to the async.group

Specify the `!async.group` size (the number of tokens that will be added to it) at construction time. `async.await_all` operation can potential

[mlir:Async] Add the size parameter to the async.group

Specify the `!async.group` size (the number of tokens that will be added to it) at construction time. `async.await_all` operation can potentially race with `async.execute` operations that keep updating the group, for this reason it is required to know upfront how many tokens will be added to the group.

Reviewed By: ftynse, herhut

Differential Revision: https://reviews.llvm.org/D104780

show more ...

[mlir] Async: Add error propagation support to async groups

Depends On D103109

If any of the tokens/values added to the `!async.group` switches to the error state, than the group itself switches to

[mlir] Async: Add error propagation support to async groups

Depends On D103109

If any of the tokens/values added to the `!async.group` switches to the error state, than the group itself switches to the error state.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D103203

show more ...

[mlir] Add error state and error propagation to async runtime values

Depends On D103102

Not yet implemented:
1. Error handling after synchronous await
2. Error handling for async groups

Will be ad

[mlir] Add error state and error propagation to async runtime values

Depends On D103102

Not yet implemented:
1. Error handling after synchronous await
2. Error handling for async groups

Will be addressed in the followup PRs

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D103109

show more ...

Fix MLIR Async Runtime DLL on Windows

The AsyncRuntime declares prototypes for extern "C" functions inside a
namespace in the header, but not inside that namespace in the
definition. This causes Vis

Fix MLIR Async Runtime DLL on Windows

The AsyncRuntime declares prototypes for extern "C" functions inside a
namespace in the header, but not inside that namespace in the
definition. This causes Visual Studio to treat them as different
entities and thus the dllexport is ignored for the definitions.

Using the same namespace fixes this issue.

Secondly, this commit moves the dllexport to be consistent with the
JITs expectation.

This is an update to https://reviews.llvm.org/D95386 that fixes the
compile issues in old versions of Visual studio.

Differential Revision: https://reviews.llvm.org/D95933

show more ...

Revert "Fix namespace for MLIR Async Runtime"

This reverts commit b7d80058ff4649d1a15ef930269458dbb17145d9.

The mlir-windows buildbot is broken.

Fix namespace for MLIR Async Runtime

The MLIR Async runtime uses different namespacing for the header file,
and the definitions of its C API. The header file places the extern "C"
functions inside n

Fix namespace for MLIR Async Runtime

The MLIR Async runtime uses different namespacing for the header file,
and the definitions of its C API. The header file places the extern "C"
functions inside namespace mlir::runtime, and the definitions are not
in a namespace. This causes issues in cl.exe. It treats the declaration
and definition as different, and thus does not apply dllexport to the
definition, which leads to the mlir_async_runtime.dll containing no
definitions, and the mlir_async_runtime.lib not being generated.

This patch moves the namespace to cover the definitions, and thus
generates the dll correctly on Windows with cl.exe.

This was tested with Visual Studio C++ 19.28.29336.

Differential Revision: https://reviews.llvm.org/D95386

show more ...

[mlir:async] Fix deadlock in async runtime await-and-execute functions

`emplace???` functions running concurrently can set the ready flag and then pending awaiter will never be executed

Differentia

[mlir:async] Fix deadlock in async runtime await-and-execute functions

`emplace???` functions running concurrently can set the ready flag and then pending awaiter will never be executed

Differential Revision: https://reviews.llvm.org/D95517

show more ...

[mlir:async] Fix data races in AsyncRuntime

Resumed coroutine potentially can deallocate the token/value/group and destroy the mutex before the std::unique_ptr destructor.

Reviewed By: mehdi_amini

[mlir:async] Fix data races in AsyncRuntime

Resumed coroutine potentially can deallocate the token/value/group and destroy the mutex before the std::unique_ptr destructor.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D95037

show more ...

[mlir] AsyncRuntime: use LLVM ThreadPool to run async tasks

Revert https://reviews.llvm.org/D92368 after the dynamic library unloading was fixed in https://reviews.llvm.org/D94312

Reviewed By: mehd

[mlir] AsyncRuntime: use LLVM ThreadPool to run async tasks

Revert https://reviews.llvm.org/D92368 after the dynamic library unloading was fixed in https://reviews.llvm.org/D94312

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D94346

show more ...

[mlir] AsyncRuntime: disable mlir-runner init/disable for WIN32

Differential Revision: https://reviews.llvm.org/D94339

[mlir:JitRunner] Use custom shared library init/destroy functions if available

Use custom mlir runner init/destroy functions to safely init and destroy shared libraries loaded by the JitRunner.

Thi

[mlir:JitRunner] Use custom shared library init/destroy functions if available

Use custom mlir runner init/destroy functions to safely init and destroy shared libraries loaded by the JitRunner.

This mechanism is ignored for Windows builds (for now) because init/destroy functions are not exported, and library unloading relies on static destructors.

Re-submit https://reviews.llvm.org/D94270 with a temporary workaround for windows

Differential Revision: https://reviews.llvm.org/D94312

show more ...

Revert "[mlir:JitRunner] Use custom shared library init/destroy functions if available"

This reverts commit 84dc9b451bfd62474f44dd1af0e4955a0110d523.

Fix Windows breakage: http://lab.llvm.org:8011/

Revert "[mlir:JitRunner] Use custom shared library init/destroy functions if available"

This reverts commit 84dc9b451bfd62474f44dd1af0e4955a0110d523.

Fix Windows breakage: http://lab.llvm.org:8011/#/builders/13/builds/3658/steps/6/logs/stdio

Differential Revision: https://reviews.llvm.org/D94309

show more ...

[mlir:JitRunner] Use custom shared library init/destroy functions if available

Use custom mlir runner init/destroy functions to safely init and destroy shared libraries loaded by the JitRunner.

Rev

[mlir:JitRunner] Use custom shared library init/destroy functions if available

Use custom mlir runner init/destroy functions to safely init and destroy shared libraries loaded by the JitRunner.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D94270

show more ...

[mlir] Async: lowering async.value to LLVM

1. Add new methods to Async runtime API to support yielding async values
2. Add lowering from `async.yield` with value payload to the new runtime API calls

[mlir] Async: lowering async.value to LLVM

1. Add new methods to Async runtime API to support yielding async values
2. Add lowering from `async.yield` with value payload to the new runtime API calls

`async.value` lowering requires that payload type is convertible to LLVM and supported by `llvm.mlir.cast` (DialectCast) operation.

Reviewed By: csigg

Differential Revision: https://reviews.llvm.org/D93592

show more ...

[mlir] AsyncRuntime: mode runtime declarations to mlir::runtime namespace

Define Async runtime related typedefs in the `mlir::runtime` namespace.

Reviewed By: mehdi_amini

Differential Revision: ht

[mlir] AsyncRuntime: mode runtime declarations to mlir::runtime namespace

Define Async runtime related typedefs in the `mlir::runtime` namespace.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D93391

show more ...

[mlir] AsyncRuntime: disable threading until test flakiness is fixed

ExecutionEngine/LLJIT do not run globals destructors in loaded dynamic libraries when destroyed, and threads managed by ThreadPoo

[mlir] AsyncRuntime: disable threading until test flakiness is fixed

ExecutionEngine/LLJIT do not run globals destructors in loaded dynamic libraries when destroyed, and threads managed by ThreadPool can race with program termination, and it leads to segfaults.

TODO: Re-enable threading after fixing a problem with destructors, or removing static globals from dynamic library.

Differential Revision: https://reviews.llvm.org/D92368

show more ...

[mlir] AsyncRuntime: fix concurrency bugs + fix exports in methods definitions

1. Move ThreadPool ownership to the runtime, and wait for the async tasks completion in the destructor.
2. Remove MLIR_

[mlir] AsyncRuntime: fix concurrency bugs + fix exports in methods definitions

1. Move ThreadPool ownership to the runtime, and wait for the async tasks completion in the destructor.
2. Remove MLIR_ASYNCRUNTIME_EXPORT from method definitions because they are unnecessary in .cpp files, as only function declarations need to be exported, not their definitions.
3. Fix concurrency bugs in group emplace and potential use-after-free in token emplace.

Tested internally 10k runs in `async.mlir` and `async-group.mlir`.

Fixed: https://bugs.llvm.org/show_bug.cgi?id=48267

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D91988

show more ...

[mlir] Add microbenchmark for linalg+async-parallel-for

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D91896

[mlir] Automatic reference counting for Async values + runtime support for ref counted objects

Depends On D89963

**Automatic reference counting algorithm outline:**

1. `ReturnLike` operations forw

[mlir] Automatic reference counting for Async values + runtime support for ref counted objects

Depends On D89963

**Automatic reference counting algorithm outline:**

1. `ReturnLike` operations forward the reference counted values without
modifying the reference count.
2. Use liveness analysis to find blocks in the CFG where the lifetime of
reference counted values ends, and insert `drop_ref` operations after
the last use of the value.
3. Insert `add_ref` before the `async.execute` operation capturing the
value, and pairing `drop_ref` before the async body region terminator,
to release the captured reference counted value when execution
completes.
4. If the reference counted value is passed only to some of the block
successors, insert `drop_ref` operations in the beginning of the blocks
that do not have reference coutned value uses.

Reviewed By: silvas

Differential Revision: https://reviews.llvm.org/D90716

show more ...

[mlir] Transform scf.parallel to scf.for + async.execute

Depends On D89958

1. Adds `async.group`/`async.awaitall` to group together multiple async tokens/values
2. Rewrite scf.parallel operation in

[mlir] Transform scf.parallel to scf.for + async.execute

Depends On D89958

1. Adds `async.group`/`async.awaitall` to group together multiple async tokens/values
2. Rewrite scf.parallel operation into multiple concurrent async.execute operations over non overlapping subranges of the original loop.

Example:

```
scf.for (%i, %j) = (%lbi, %lbj) to (%ubi, %ubj) step (%si, %sj) {
"do_some_compute"(%i, %j): () -> ()
}
```

Converted to:

```
%c0 = constant 0 : index
%c1 = constant 1 : index

// Compute blocks sizes for each induction variable.
%num_blocks_i = ... : index
%num_blocks_j = ... : index
%block_size_i = ... : index
%block_size_j = ... : index

// Create an async group to track async execute ops.
%group = async.create_group

scf.for %bi = %c0 to %num_blocks_i step %c1 {
%block_start_i = ... : index
%block_end_i = ... : index

scf.for %bj = %c0 t0 %num_blocks_j step %c1 {
%block_start_j = ... : index
%block_end_j = ... : index

// Execute the body of original parallel operation for the current
// block.
%token = async.execute {
scf.for %i = %block_start_i to %block_end_i step %si {
scf.for %j = %block_start_j to %block_end_j step %sj {
"do_some_compute"(%i, %j): () -> ()
}
}
}

// Add produced async token to the group.
async.add_to_group %token, %group
}
}

// Await completion of all async.execute operations.
async.await_all %group
```
In this example outer loop launches inner block level loops as separate async
execute operations which will be executed concurrently.

At the end it waits for the completiom of all async execute operations.

Reviewed By: ftynse, mehdi_amini

Differential Revision: https://reviews.llvm.org/D89963

show more ...

[mlir] Fix exports in mlir_async_runtime

The MLIR_ASYNCRUNTIME_EXPORT macro was being defined to be either
__declspec(dllexport) or __declspec(dllimport), depending on whether
mlir_c_runner_utils_EX

[mlir] Fix exports in mlir_async_runtime

The MLIR_ASYNCRUNTIME_EXPORT macro was being defined to be either
__declspec(dllexport) or __declspec(dllimport), depending on whether
mlir_c_runner_utils_EXPORTS is defined. The latter was a copy/paste
error and should have been mlir_async_runtime_EXPORTS.

Additionally, the uses of that macro in the .cpp file were unnecessary,
as only function declarations need to be exported, not their definitions.

Differential Revision: https://reviews.llvm.org/D91196

show more ...