[Clang][OpenMP] Fix the issue that `llvm.lifetime.end` is emitted too early for variables captured in linear clause

Currently if an OpenMP program uses `linear` clause, and is compiled with
optimiza

[Clang][OpenMP] Fix the issue that `llvm.lifetime.end` is emitted too early for variables captured in linear clause

Currently if an OpenMP program uses `linear` clause, and is compiled with
optimization, `llvm.lifetime.end` for variables listed in `linear` clause are
emitted too early such that there could still be uses after that. Let's take the
following code as example:
```
// loop.c
int j;
int *u;

void loop(int n) {
int i;
for (i = 0; i < n; ++i) {
++j;
u = &j;
}
}
```
We compile using the command:
```
clang -cc1 -fopenmp-simd -O3 -x c -triple x86_64-apple-darwin10 -emit-llvm loop.c -o loop.ll
```
The following IR (simplified) will be generated:
```
@j = local_unnamed_addr global i32 0, align 4
@u = local_unnamed_addr global ptr null, align 8

define void @loop(i32 noundef %n) local_unnamed_addr {
entry:
%j = alloca i32, align 4
%cmp = icmp sgt i32 %n, 0
br i1 %cmp, label %simd.if.then, label %simd.if.end

simd.if.then: ; preds = %entry
call void @llvm.lifetime.start.p0(i64 4, ptr nonnull %j)
store ptr %j, ptr @u, align 8
call void @llvm.lifetime.end.p0(i64 4, ptr nonnull %j)
%0 = load i32, ptr %j, align 4
store i32 %0, ptr @j, align 4
br label %simd.if.end

simd.if.end: ; preds = %simd.if.then, %entry
ret void
}
```
The most important part is:
```
call void @llvm.lifetime.end.p0(i64 4, ptr nonnull %j)
%0 = load i32, ptr %j, align 4
store i32 %0, ptr @j, align 4
```
`%j` is still loaded after `@llvm.lifetime.end.p0(i64 4, ptr nonnull %j)`. This
could cause the backend incorrectly optimizes the code and further generates
incorrect code. The root cause is, when we emit a construct that could have
`linear` clause, it usually has the following pattern:
```
EmitOMPLinearClauseInit(S)
{
OMPPrivateScope LoopScope(*this);
...
EmitOMPLinearClause(S, LoopScope);
...
(void)LoopScope.Privatize();
...
}
EmitOMPLinearClauseFinal(S, [](CodeGenFunction &) { return nullptr; });
```
Variables that need to be privatized are added into `LoopScope`, which also
serves as a RAII object. When `LoopScope` is destructed and if optimization is
enabled, a `@llvm.lifetime.end` is also emitted for each privatized variable.
However, the writing back to original variables in `linear` clause happens after
the scope in `EmitOMPLinearClauseFinal`, causing the issue we see above.

A quick "fix" seems to be, moving `EmitOMPLinearClauseFinal` inside the scope.
However, it doesn't work. That's because the local variable map has been updated
by `LoopScope` such that a variable declaration is mapped to the privatized
variable, instead of the actual one. In that way, the following code will be
generated:
```
%0 = load i32, ptr %j, align 4
store i32 %0, ptr %j, align 4
call void @llvm.lifetime.end.p0(i64 4, ptr nonnull %j)
```
Well, now the life time is correct, but apparently the writing back is broken.

In this patch, a new function `OMPPrivateScope::restoreMap` is added and called
before calling `EmitOMPLinearClauseFinal`. This can make sure that
`EmitOMPLinearClauseFinal` can find the orignal varaibls to write back.

Fixes #56913.

Reviewed By: ABataev

Differential Revision: https://reviews.llvm.org/D131272

(cherry picked from commit e21202dac18ed7f718d26a0e131f96b399b4891c)

show more ...

[clang][OpenMP] Add IRBuilder support for taskgroup

This patch makes use of OMPIRBuilder support for codegen of taskgroup
construct in clang.

Depends on D128203

Reviewed By: Meinersbur

Differenti

[clang][OpenMP] Add IRBuilder support for taskgroup

This patch makes use of OMPIRBuilder support for codegen of taskgroup
construct in clang.

Depends on D128203

Reviewed By: Meinersbur

Differential Revision: https://reviews.llvm.org/D129992

show more ...

[OMPIRBuilder] Add support for simdlen clause

This patch adds OMPIRBuilder support for the simdlen clause for the
simd directive. It uses the simdlen support in OpenMPIRBuilder when
it is enabled in

[OMPIRBuilder] Add support for simdlen clause

This patch adds OMPIRBuilder support for the simdlen clause for the
simd directive. It uses the simdlen support in OpenMPIRBuilder when
it is enabled in Clang. Simdlen is lowered by OpenMPIRBuilder by
generating the loop.vectorize.width metadata.

Reviewed By: jdoerfert, Meinersbur

Differential Revision: https://reviews.llvm.org/D129149

show more ...

[Clang][OpenMP] Enable floating-point operation for `atomic compare` series

D127041 introduced the support for `fmax` and `fmin` such that we can also reprent
`atomic compare` and `atomic compare ca

[Clang][OpenMP] Enable floating-point operation for `atomic compare` series

D127041 introduced the support for `fmax` and `fmin` such that we can also reprent
`atomic compare` and `atomic compare capture` with `atomicrmw` instruction. This
patch simply lifts the limitation we set before.

Depend on D127041.

Reviewed By: ABataev

Differential Revision: https://reviews.llvm.org/D127042

show more ...

Adding support for target in_reduction

Implementing target in_reduction by wrapping target task with host task with in_reduction and if clause. This is in compliance with OpenMP 5.0 section: 2.19.5.

Adding support for target in_reduction

Implementing target in_reduction by wrapping target task with host task with in_reduction and if clause. This is in compliance with OpenMP 5.0 section: 2.19.5.6.
So, this

```
for (int i=0; i<N; i++) {
res = res+i
}
```

will become

```

#pragma omp task in_reduction(+:res) if(0)
#pragma omp target map(res)
for (int i=0; i<N; i++) {
res = res+i
}
```

Reviewed By: ABataev

Differential Revision: https://reviews.llvm.org/D125669

show more ...

[clang] Don't use Optional::hasValue (NFC)

[Clang][OpenMP] Add the codegen support for `atomic compare capture`

This patch adds the codegen support for `atomic compare capture` in clang.

Reviewed By: ABataev

Differential Revision: https://

[Clang][OpenMP] Add the codegen support for `atomic compare capture`

This patch adds the codegen support for `atomic compare capture` in clang.

Reviewed By: ABataev

Differential Revision: https://reviews.llvm.org/D120290

show more ...

[Clang][OpenMP] Avoid using `IgnoreImpCasts` if possible

This patch removes all `IgnoreImpCasts` in Sema, and only uses it if necessary. If the expression is not of the same type as the pointer valu

[Clang][OpenMP] Avoid using `IgnoreImpCasts` if possible

This patch removes all `IgnoreImpCasts` in Sema, and only uses it if necessary. If the expression is not of the same type as the pointer value, a cast is inserted.

Reviewed By: ABataev

Differential Revision: https://reviews.llvm.org/D126602

show more ...

[OMPIRBuilder] Add the support for compare capture

This patch adds the support for `compare capture` in `OMPIRBuilder`.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D1200

[OMPIRBuilder] Add the support for compare capture

This patch adds the support for `compare capture` in `OMPIRBuilder`.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D120007

show more ...

[OpenMP][Clang] Fix atomic compare for signed vs. unsigned

Without this patch, arguments to the
`llvm::OpenMPIRBuilder::AtomicOpValue` initializer are reversed.

Reviewed By: ABataev, tianshilei1992

[OpenMP][Clang] Fix atomic compare for signed vs. unsigned

Without this patch, arguments to the
`llvm::OpenMPIRBuilder::AtomicOpValue` initializer are reversed.

Reviewed By: ABataev, tianshilei1992

Differential Revision: https://reviews.llvm.org/D126619

show more ...

Removing this as part of the revert done in 69da3b6aead2e7a18a2578aad661d6d36b8d30cf

This appears to have been added in a follow-up commit that I missed.

Fix warning by handling OMPC_fail in switch statement.

[OpenMP] Add codegen for 'omp_all_memory' reserved locator.

This creates an entry with address=nullptr and flag=0x80.
When an 'omp_all_memory' entry is specified any other 'out' or
'inout' entries a

[OpenMP] Add codegen for 'omp_all_memory' reserved locator.

This creates an entry with address=nullptr and flag=0x80.
When an 'omp_all_memory' entry is specified any other 'out' or
'inout' entries are not needed and are not passed to the runtime.

Differential Revision: https://reviews.llvm.org/D126321

show more ...

[Clang][OpenMP] Add the support for floating-point variables for specific atomic clauses

Currently when using `atomic update` with floating-point variables, if
the operation is add or sub, `cmpxchg`

[Clang][OpenMP] Add the support for floating-point variables for specific atomic clauses

Currently when using `atomic update` with floating-point variables, if
the operation is add or sub, `cmpxchg`, instead of `atomicrmw` is emitted, as
shown in [1]. In fact, about three years ago, llvm-svn: 351850 added the
support for FP operations. This patch adds the support in OpenMP as well.

[1] https://godbolt.org/z/M7b4ba9na

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D124724

show more ...

[OpenMP] Fix -Wswitch (due to new OMPC_cancellation_construct_type) after D123828

[OPENMP]PR53344: Emit code for final update of the inscan reduction vars in worksharing loops.

Need to emit final update of the inscan reduction variables. For
worksharing loops, the reduction value

[OPENMP]PR53344: Emit code for final update of the inscan reduction vars in worksharing loops.

Need to emit final update of the inscan reduction variables. For
worksharing loops, the reduction values are stored in the temp array,
need to copy the last element to the original var at the end of the
construct.

Differential Revision: https://reviews.llvm.org/D121156

show more ...

[OpenMPIRBuilder] Remove ContinuationBB argument from Body callback.

The callback is expected to create a branch to the ContinuationBB (sometimes called FiniBB in some lambdas) argument when finishi

[OpenMPIRBuilder] Remove ContinuationBB argument from Body callback.

The callback is expected to create a branch to the ContinuationBB (sometimes called FiniBB in some lambdas) argument when finishing. This creates problems:

1. The InsertPoint used for CodeGenIP does not need to be the end of a block. If it is not, a naive callback will insert a branch instruction into the middle of the block.

2. The BasicBlock the CodeGenIP is pointing to may or may not have a terminator. There is an conflict where to branch to if the block already has a terminator.

3. Some API functions work only with block having a terminator. Some workarounds have been used to insert a temporary terminator that is removed again.

4. Some callbacks are sensitive to whether the BasicBlock has a terminator or not. This creates a callback ordering problem where different callback may have different behaviour depending on whether a previous callback created a terminator or not. The problem also exists for FinalizeCallbackTy where some callbacks do create branch to another "continue" block, but unlike BodyGenCallbackTy does not receive the target as argument. This is not addressed in this patch.

With this patch, the callback receives an CodeGenIP into a BasicBlock where to insert instructions. If it has to insert control flow, it can split the block at that position as needed but otherwise no separate ContinuationBB is needed. In particular, a callback can be empty without breaking the emitted IR. If the caller needs the control flow to branch to a specific target, it can insert the branch instruction itself and pass an InsertPoint before the terminator to the callback.

Certain frontends such as Clang may expect the current IRBuilder position to be at the end of a basic block. In this case its callbacks must split the block at CodeGenIP before setting the IRBuilder position such that the instructions after CodeGenIP are moved to another basic block and before returning create a new branch instruction to the split block.

Some utility functions such as `splitBB` are supporting correct splitting of BasicBlocks, independent of whether they have a terminator or not, returning/setting the InsertPoint of an IRBuilder to the end of split predecessor block, and optionally omitting creating a branch to the split successor block to be added later.

Reviewed By: kiranchandramohan

Differential Revision: https://reviews.llvm.org/D118409

show more ...

[clang][OpenMP][DebugInfo] Debug support for TLS variables present in OpenMP consruct

In case of OpenMP programs, thread local variables can be present in
any clause pertaining to OpenMP constructs,

[clang][OpenMP][DebugInfo] Debug support for TLS variables present in OpenMP consruct

In case of OpenMP programs, thread local variables can be present in
any clause pertaining to OpenMP constructs, as we know that compiler
generates artificial functions and in some cases values are passed to
those artificial functions thru parameters. For an example, if thread
local variable is present in copyin clause (testcase attached with the
patch), parameter with same name is generated as parameter to artificial
function. When user inquires the thread Local variable, its debug info
is hidden by the parameter. User never gets the actual TLS variable
when inquires it, instead gets the artificial parameter.

Current patch suppresses the debug info for such artificial parameter to
enable correct debugging of TLS variables.

Reviewed By: aprantl

Differential Revision: https://reviews.llvm.org/D123787

show more ...

Reapply "[OpenMP] Refactor OMPScheduleType enum."

This reverts commit af0285122f306573d9bcc4c4ad7f904cfdd4d869.

The test "libomp::loop_dispatch.c" on builder
openmp-gcc-x86_64-linux-debian fails fr

Reapply "[OpenMP] Refactor OMPScheduleType enum."

This reverts commit af0285122f306573d9bcc4c4ad7f904cfdd4d869.

The test "libomp::loop_dispatch.c" on builder
openmp-gcc-x86_64-linux-debian fails from time-to-time.
See #54969. This patch is unrelated.

show more ...

Revert "[OpenMP] Refactor OMPScheduleType enum."

This reverts commit 9ec501da76fc1559cadd6d6dac32766bf4376a3d.

It may have caused the openmp-gcc-x86_64-linux-debian buildbot to fail.
https://lab.ll

Revert "[OpenMP] Refactor OMPScheduleType enum."

This reverts commit 9ec501da76fc1559cadd6d6dac32766bf4376a3d.

It may have caused the openmp-gcc-x86_64-linux-debian buildbot to fail.
https://lab.llvm.org/buildbot/#/builders/4/builds/20377

show more ...

[OpenMP] Refactor OMPScheduleType enum.

The OMPScheduleType enum stores the constants from libomp's internal sched_type in kmp.h and are used by several kmp API functions. The enum values have an in

[OpenMP] Refactor OMPScheduleType enum.

The OMPScheduleType enum stores the constants from libomp's internal sched_type in kmp.h and are used by several kmp API functions. The enum values have an internal structure, namely each scheduling algorithm (e.g.) exists in four variants: unordered, orderend, normerge unordered, and nomerge ordered.

This patch (basically a followup to D114940) splits the "ordered" and "nomerge" bits into separate flags, as was already done for the "monotonic" and "nonmonotonic", so we can apply bit flags operations on them. It also now contains all possible combinations according to kmp's sched_type. Deriving of the OMPScheduleType enum from clause parameters has been moved form MLIR's OpenMPToLLVMIRTranslation.cpp to OpenMPIRBuilder to make available for clang as well. Since the primary purpose of the flag is the binary interface to libomp, it has been made more private to LLVMFrontend. The primary interface for generating worksharing-loop using OpenMPIRBuilder code becomes `applyWorkshareLoop` which derives the OMPScheduleType automatically and calls the appropriate emitter function.

While this is mostly a NFC refactor, it still applies the following functional changes:
* The logic from OpenMPToLLVMIRTranslation to derive the OMPScheduleType also applies to clang. Most notably, it now applies the nonmonotonic flag for non-static schedules by default.
* In OpenMPToLLVMIRTranslation, the nonmonotonic default flag was previously not applied if the simd modifier was used. I assume this was a bug, since the effect was due to `loop.schedule_modifier()` returning `mlir::omp::ScheduleModifier::none` instead of `llvm::Optional::None`.
* In OpenMPToLLVMIRTranslation, the nonmonotonic default flag was set even if ordered was specified, in breach to what the comment before citing the OpenMP specification says. I assume this was an oversight.

The ordered flag with parameter was not considered in this patch. Changes will need to be made (e.g. adding/modifying function parameters) when support for it is added. The lengthy names of the enum values can be discussed, for the moment this is avoiding reusing previously existing enum value names such as `StaticChunked` to avoid confusion.

Reviewed By: peixin

Differential Revision: https://reviews.llvm.org/D123403

show more ...

[clang][OpenMP5.1] Initial parsing/sema for has_device_addr

Added basic parsing/sema/ support for the 'has_device_addr' clause.

Differential Revision: https://reviews.llvm.org/D123402

[CGStmtOpenMP] Remove uses of deprecated Address constructor

[CodeGen][OpenMP] Make EmitLoadOfPointer() type consistent

If necessary insert a bitcast beforehand, so the LLVM-level pointer
type and the Clang-level pointer type line up.

Simplify OpenMP Lambda use

* Use default ref capture for non-escaping lambdas (this makes
maintenance easier by allowing new uses, removing uses, having
conditional uses (such as in assertions)

Simplify OpenMP Lambda use

* Use default ref capture for non-escaping lambdas (this makes
maintenance easier by allowing new uses, removing uses, having
conditional uses (such as in assertions) not require updates to an
explicit capture list)
* Simplify addPrivate API not to take a lambda, since it calls it
unconditionally/immediately anyway - most callers are simply passing
in a named value or short expression anyway and the lambda syntax just
adds noise/overhead

Reviewed By: ABataev

Differential Revision: https://reviews.llvm.org/D121077

show more ...