[OpenMP][IRBuilder] Add support for taskgroup

This patch adds support for generating taskgroup construct.

Reviewed By: Meinersbur

Differential Revision: https://reviews.llvm.org/D128203

[OMPIRBuilder] Add support for simdlen clause

This patch adds OMPIRBuilder support for the simdlen clause for the
simd directive. It uses the simdlen support in OpenMPIRBuilder when
it is enabled in

[OMPIRBuilder] Add support for simdlen clause

This patch adds OMPIRBuilder support for the simdlen clause for the
simd directive. It uses the simdlen support in OpenMPIRBuilder when
it is enabled in Clang. Simdlen is lowered by OpenMPIRBuilder by
generating the loop.vectorize.width metadata.

Reviewed By: jdoerfert, Meinersbur

Differential Revision: https://reviews.llvm.org/D129149

show more ...

[OpenMP] Change OpenMP code generation for target region entries

This patch changes the code we generate to enter a target region on the
device. This is in-line with the new definition in the runtim

[OpenMP] Change OpenMP code generation for target region entries

This patch changes the code we generate to enter a target region on the
device. This is in-line with the new definition in the runtime that was
added previously. Additionally we implement this in the OpenMPIRBuilder
so that this code can be shared with Flang in the future.

Reviewed By: ABataev

Differential Revision: https://reviews.llvm.org/D128550

show more ...

[Clang][OpenMP] Enable floating-point operation for `atomic compare` series

D127041 introduced the support for `fmax` and `fmin` such that we can also reprent
`atomic compare` and `atomic compare ca

[Clang][OpenMP] Enable floating-point operation for `atomic compare` series

D127041 introduced the support for `fmax` and `fmin` such that we can also reprent
`atomic compare` and `atomic compare capture` with `atomicrmw` instruction. This
patch simply lifts the limitation we set before.

Depend on D127041.

Reviewed By: ABataev

Differential Revision: https://reviews.llvm.org/D127042

show more ...

[LLVM] Add the support for fmax and fmin in atomicrmw instruction

This patch adds the support for `fmax` and `fmin` operations in `atomicrmw`
instruction. For now (at least in this patch), the instr

[LLVM] Add the support for fmax and fmin in atomicrmw instruction

This patch adds the support for `fmax` and `fmin` operations in `atomicrmw`
instruction. For now (at least in this patch), the instruction will be expanded
to CAS loop. There are already a couple of targets supporting the feature. I'll
create another patch(es) to enable them accordingly.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D127041

show more ...

[mlir][OpenMP] omp.task translation to LLVM IR

This patch adds translation for omp.task from OpenMPDialect to LLVM IR
Dialect and adds tests for the same.

Depends on D71989

Reviewed By: ftynse, ki

[mlir][OpenMP] omp.task translation to LLVM IR

This patch adds translation for omp.task from OpenMPDialect to LLVM IR
Dialect and adds tests for the same.

Depends on D71989

Reviewed By: ftynse, kiranchandramohan, peixin, Meinersbur

Differential Revision: https://reviews.llvm.org/D123919

show more ...

[OpenMP][IRBuilder] Add final clause to task

This patch adds final clause to OpenMP IR Builder.

Reviewed By: Meinersbur

Differential Revision: https://reviews.llvm.org/D126626

Plumb InstructionCost through unroll costing

Teach the unroller(s) how to handle an invalid cost. This avoids crashes when the backend can't provide a cost due to either a fundemental limitation or

Plumb InstructionCost through unroll costing

Teach the unroller(s) how to handle an invalid cost. This avoids crashes when the backend can't provide a cost due to either a fundemental limitation or an unimplemented cost model case.

Differential Revision: https://reviews.llvm.org/D127305

show more ...

[OMPIRBuilder] Add the support for compare capture

This patch adds the support for `compare capture` in `OMPIRBuilder`.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D1200

[OMPIRBuilder] Add the support for compare capture

This patch adds the support for `compare capture` in `OMPIRBuilder`.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D120007

show more ...

[OpenMP][IRBuilder] `omp task` support

This patch adds basic support for `omp task` to the OpenMPIRBuilder.

The outlined function after code extraction is called from a wrapper function with approp

[OpenMP][IRBuilder] `omp task` support

This patch adds basic support for `omp task` to the OpenMPIRBuilder.

The outlined function after code extraction is called from a wrapper function with appropriate arguments. This wrapper function is passed to the runtime calls for task allocation.

This approach is different from the Clang approach - clang directly emits the runtime call to the outlined function. The outlining utility (OutlineInfo) simply outlines the code and generates a function call to the outlined function. After the function has been generated by the outlining utility, there is no easy way to alter the function arguments without meddling with the outlining itself. Hence the wrapper function approach is taken.

Reviewed By: Meinersbur

Differential Revision: https://reviews.llvm.org/D71989

show more ...

[OpenMP] Make generating offloading entries more generic

This patch moves the logic for generating the offloading entries to the
OpenMPIRBuilder. This makes it easier to re-use in other places, such

[OpenMP] Make generating offloading entries more generic

This patch moves the logic for generating the offloading entries to the
OpenMPIRBuilder. This makes it easier to re-use in other places, such as
for OpenMP support in Flang or using the same method for generating
offloading entires for other languages like Cuda.

Reviewed By: tianshilei1992

Differential Revision: https://reviews.llvm.org/D123460

show more ...

[OpenMPIRBuilder] Remove ContinuationBB argument from Body callback.

The callback is expected to create a branch to the ContinuationBB (sometimes called FiniBB in some lambdas) argument when finishi

[OpenMPIRBuilder] Remove ContinuationBB argument from Body callback.

The callback is expected to create a branch to the ContinuationBB (sometimes called FiniBB in some lambdas) argument when finishing. This creates problems:

1. The InsertPoint used for CodeGenIP does not need to be the end of a block. If it is not, a naive callback will insert a branch instruction into the middle of the block.

2. The BasicBlock the CodeGenIP is pointing to may or may not have a terminator. There is an conflict where to branch to if the block already has a terminator.

3. Some API functions work only with block having a terminator. Some workarounds have been used to insert a temporary terminator that is removed again.

4. Some callbacks are sensitive to whether the BasicBlock has a terminator or not. This creates a callback ordering problem where different callback may have different behaviour depending on whether a previous callback created a terminator or not. The problem also exists for FinalizeCallbackTy where some callbacks do create branch to another "continue" block, but unlike BodyGenCallbackTy does not receive the target as argument. This is not addressed in this patch.

With this patch, the callback receives an CodeGenIP into a BasicBlock where to insert instructions. If it has to insert control flow, it can split the block at that position as needed but otherwise no separate ContinuationBB is needed. In particular, a callback can be empty without breaking the emitted IR. If the caller needs the control flow to branch to a specific target, it can insert the branch instruction itself and pass an InsertPoint before the terminator to the callback.

Certain frontends such as Clang may expect the current IRBuilder position to be at the end of a basic block. In this case its callbacks must split the block at CodeGenIP before setting the IRBuilder position such that the instructions after CodeGenIP are moved to another basic block and before returning create a new branch instruction to the split block.

Some utility functions such as `splitBB` are supporting correct splitting of BasicBlocks, independent of whether they have a terminator or not, returning/setting the InsertPoint of an IRBuilder to the end of split predecessor block, and optionally omitting creating a branch to the split successor block to be added later.

Reviewed By: kiranchandramohan

Differential Revision: https://reviews.llvm.org/D118409

show more ...

Reapply "[OpenMP] Refactor OMPScheduleType enum."

This reverts commit af0285122f306573d9bcc4c4ad7f904cfdd4d869.

The test "libomp::loop_dispatch.c" on builder
openmp-gcc-x86_64-linux-debian fails fr

Reapply "[OpenMP] Refactor OMPScheduleType enum."

This reverts commit af0285122f306573d9bcc4c4ad7f904cfdd4d869.

The test "libomp::loop_dispatch.c" on builder
openmp-gcc-x86_64-linux-debian fails from time-to-time.
See #54969. This patch is unrelated.

show more ...

Revert "[OpenMP] Refactor OMPScheduleType enum."

This reverts commit 9ec501da76fc1559cadd6d6dac32766bf4376a3d.

It may have caused the openmp-gcc-x86_64-linux-debian buildbot to fail.
https://lab.ll

Revert "[OpenMP] Refactor OMPScheduleType enum."

This reverts commit 9ec501da76fc1559cadd6d6dac32766bf4376a3d.

It may have caused the openmp-gcc-x86_64-linux-debian buildbot to fail.
https://lab.llvm.org/buildbot/#/builders/4/builds/20377

show more ...

[OpenMP] Refactor OMPScheduleType enum.

The OMPScheduleType enum stores the constants from libomp's internal sched_type in kmp.h and are used by several kmp API functions. The enum values have an in

[OpenMP] Refactor OMPScheduleType enum.

The OMPScheduleType enum stores the constants from libomp's internal sched_type in kmp.h and are used by several kmp API functions. The enum values have an internal structure, namely each scheduling algorithm (e.g.) exists in four variants: unordered, orderend, normerge unordered, and nomerge ordered.

This patch (basically a followup to D114940) splits the "ordered" and "nomerge" bits into separate flags, as was already done for the "monotonic" and "nonmonotonic", so we can apply bit flags operations on them. It also now contains all possible combinations according to kmp's sched_type. Deriving of the OMPScheduleType enum from clause parameters has been moved form MLIR's OpenMPToLLVMIRTranslation.cpp to OpenMPIRBuilder to make available for clang as well. Since the primary purpose of the flag is the binary interface to libomp, it has been made more private to LLVMFrontend. The primary interface for generating worksharing-loop using OpenMPIRBuilder code becomes `applyWorkshareLoop` which derives the OMPScheduleType automatically and calls the appropriate emitter function.

While this is mostly a NFC refactor, it still applies the following functional changes:
* The logic from OpenMPToLLVMIRTranslation to derive the OMPScheduleType also applies to clang. Most notably, it now applies the nonmonotonic flag for non-static schedules by default.
* In OpenMPToLLVMIRTranslation, the nonmonotonic default flag was previously not applied if the simd modifier was used. I assume this was a bug, since the effect was due to `loop.schedule_modifier()` returning `mlir::omp::ScheduleModifier::none` instead of `llvm::Optional::None`.
* In OpenMPToLLVMIRTranslation, the nonmonotonic default flag was set even if ordered was specified, in breach to what the comment before citing the OpenMP specification says. I assume this was an oversight.

The ordered flag with parameter was not considered in this patch. Changes will need to be made (e.g. adding/modifying function parameters) when support for it is added. The lengthy names of the enum values can be discussed, for the moment this is avoiding reusing previously existing enum value names such as `StaticChunked` to avoid confusion.

Reviewed By: peixin

Differential Revision: https://reviews.llvm.org/D123403

show more ...

[OpenMPIRBuilder] Detect and fix ambiguous InsertPoints for createSections.

Follow-up on D117226 for createSections.

Reviewed By: shraiysh

Differential Revision: https://reviews.llvm.org/D117835

[OMPIRBuilder] Support ordered clause specified without parameter

This patch supports ordered clause specified without parameter in
worksharing-loop directive in the OpenMPIRBuilder and lowering MLI

[OMPIRBuilder] Support ordered clause specified without parameter

This patch supports ordered clause specified without parameter in
worksharing-loop directive in the OpenMPIRBuilder and lowering MLIR to
LLVM IR.

Reviewed By: Meinersbur

Differential Revision: https://reviews.llvm.org/D114940

show more ...

[mlir][OpenMP][IRBuilder] Add support for nowait on single construct

This patch adds the nowait parameter to `createSingle` in
OpenMPIRBuilder and handling for IR generation from OpenMP Dialect.

Al

[mlir][OpenMP][IRBuilder] Add support for nowait on single construct

This patch adds the nowait parameter to `createSingle` in
OpenMPIRBuilder and handling for IR generation from OpenMP Dialect.

Also added tests for the same.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D122371

show more ...

[mlir][OpenMP] Added translation from `omp.atomic.capture` to LLVM IR

This patch adds translation from `omp.atomic.capture` to LLVM IR. Also
added tests for the same.

Depends on D121546

Reviewed B

[mlir][OpenMP] Added translation from `omp.atomic.capture` to LLVM IR

This patch adds translation from `omp.atomic.capture` to LLVM IR. Also
added tests for the same.

Depends on D121546

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D121554

show more ...

[OpenMP][IRBuilder] Fix emitAtomicUpdate conditions

This patch fixes the condition for emitting atomic update using
`atomicrmw` instruction or compare-exchange loop.

Reviewed By: ftynse

Differenti

[OpenMP][IRBuilder] Fix emitAtomicUpdate conditions

This patch fixes the condition for emitting atomic update using
`atomicrmw` instruction or compare-exchange loop.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D121546

show more ...

Cleanup includes: DebugInfo & CodeGen

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D121332

[OpenMP][mlir] Lowering for omp.atomic.update

This patch adds lowering from omp.atomic.update to LLVM IR. Whenever a
special LLVM IR instruction is available for the operation, `atomicrmw`
instructi

[OpenMP][mlir] Lowering for omp.atomic.update

This patch adds lowering from omp.atomic.update to LLVM IR. Whenever a
special LLVM IR instruction is available for the operation, `atomicrmw`
instruction is emitted, otherwise a compare-exchange loop based update
is emitted.

Depends on D119522

Reviewed By: ftynse, peixin

Differential Revision: https://reviews.llvm.org/D119657

show more ...

[OpenMPIRBuilder] Allocate temporary at the correct block in a nested parallel

The OpenMPIRBuilder has a bug. Specifically, suppose you have two nested openmp parallel regions (writing with MLIR for

[OpenMPIRBuilder] Allocate temporary at the correct block in a nested parallel

The OpenMPIRBuilder has a bug. Specifically, suppose you have two nested openmp parallel regions (writing with MLIR for ease)

```
omp.parallel {
%a = ...
omp.parallel {
use(%a)
}
}
```

As OpenMP only permits pointer-like inputs, the builder will wrap all of the inputs into a stack allocation, and then pass this
allocation to the inner parallel. For example, we would want to get something like the following:

```
omp.parallel {
%a = ...
%tmp = alloc
store %tmp[] = %a
kmpc_fork(outlined, %tmp)
}
```

However, in practice, this is not what currently occurs in the context of nested parallel regions. Specifically to the OpenMPIRBuilder,
the entirety of the function (at the LLVM level) is currently inlined with blocks marking the corresponding start and end of each
region.

```
entry:
...

parallel1:
%a = ...
...

parallel2:
use(%a)
...

endparallel2:
...

endparallel1:
...
```

When the allocation is inserted, it presently inserted into the parent of the entire function (e.g. entry) rather than the parent
allocation scope to the function being outlined. If we were outlining parallel2, the corresponding alloca location would be parallel1.

This causes a variety of bugs, including https://github.com/llvm/llvm-project/issues/54165 as one example.

This PR allows the stack allocation to be created at the correct allocation block, and thus remedies such issues.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D121061

show more ...

Cleanup includes: TransformsUtils

Estimation on the impact on preprocessor output:
before: 1065307662
after: 1064800684

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-

Cleanup includes: TransformsUtils

Estimation on the impact on preprocessor output:
before: 1065307662
after: 1064800684

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D120741

show more ...

[OpenMPIRBuilder] Implement static-chunked workshare-loop schedules.

Add applyStaticChunkedWorkshareLoop method implementing static schedule when chunk-size is specified. Unlike a static schedule wi

[OpenMPIRBuilder] Implement static-chunked workshare-loop schedules.

Add applyStaticChunkedWorkshareLoop method implementing static schedule when chunk-size is specified. Unlike a static schedule without chunk-size (where chunk-size is chosen by the runtime such that each thread receives one chunk), we need two nested loops: one for looping over the iterations of a chunk, and a second for looping over all chunks assigned to the threads.

This patch includes the following related changes:
* Adapt applyWorkshareLoop to triage between the schedule types, now possible since all schedules have been implemented. The default schedule is assumed to be non-chunked static, as without OpenMPIRBuilder.
* Remove the chunk parameter from applyStaticWorkshareLoop, it is ignored by the runtime. Change the value for the value passed to the init function to 0, as without OpenMPIRBuilder.
* Refactor CanonicalLoopInfo::setTripCount and CanonicalLoopInfo::mapIndVar as used by both, applyStaticWorkshareLoop and applyStaticChunkedWorkshareLoop.
* Enable Clang to use the OpenMPIRBuilder in the presence of the schedule clause.

Differential Revision: https://reviews.llvm.org/D114413

show more ...