[Libomptarget][NFC] Move legacy functions to a separate file

This patch moves the old legacy interfaces into `libomptarget` to a
separate file. These do not need to be included anywhere and are simp

[Libomptarget][NFC] Move legacy functions to a separate file

This patch moves the old legacy interfaces into `libomptarget` to a
separate file. These do not need to be included anywhere and are simply
provided for backwards compatibility with the ABI. This cleans up the
interface greatly.

Depends on D128817

Reviewed By: JonChesterfield

Differential Revision: https://reviews.llvm.org/D128818

show more ...

[Libomptarget] Use new tripcount argument in the runtime.

The previous patch added an argument to the `__tgt_target_kernel`
runtime function which includes the tripcount used for the loop clause.
Th

[Libomptarget] Use new tripcount argument in the runtime.

The previous patch added an argument to the `__tgt_target_kernel`
runtime function which includes the tripcount used for the loop clause.
This was originally passed in via the `__kmpc_push_target_tripcount`
function. Now we move this logic to the kernel launch itself and remove
the need for the push function.

Depends on D128816

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D128817

show more ...

[Libomptarget] Implement a unified kernel entry function

This patch implements a unified kernel entry function that will be
targeted from both teams and non-teams clauses. We introduce a new
interfa

[Libomptarget] Implement a unified kernel entry function

This patch implements a unified kernel entry function that will be
targeted from both teams and non-teams clauses. We introduce a new
interface and make the old functions call in using the new one. A
following patch will include the necessary changes to Clang to call
these new functions instead.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D128549

show more ...

[Libomptarget][NFC] Make Libomptarget use the LLVM naming convention

Libomptarget grew out of a project that was originally not in LLVM. As
we develop libomptarget this has led to an increasingly la

[Libomptarget][NFC] Make Libomptarget use the LLVM naming convention

Libomptarget grew out of a project that was originally not in LLVM. As
we develop libomptarget this has led to an increasingly large clash
between the naming conventions used. This patch fixes most of the
variable names that did not confrom to the LLVM standard, that is
`VariableName` for variables and `functionName` for functions.

This patch was primarily done using my editor's linting messages, if
there are any issues I missed arising from the automation let me know.

Reviewed By: saiislam

Differential Revision: https://reviews.llvm.org/D128997

show more ...

[OpenMP][libomptarget] Add __tgt_target_return_t enum for __tgt_target_XXX return int

The defintion of OFFLOAD_SUCCESS and OFFLOAD_FAIL used in plugin APIs and libomptarget public APIs are not consi

[OpenMP][libomptarget] Add __tgt_target_return_t enum for __tgt_target_XXX return int

The defintion of OFFLOAD_SUCCESS and OFFLOAD_FAIL used in plugin APIs and libomptarget public APIs are not consistent.
Create __tgt_target_return_t for libomptarget public APIs.

Differential Revision: https://reviews.llvm.org/D109304

show more ...

[OpenMP][libomptarget][NFC] Change checkDeviceAndCtors return type to bool.

What is exactly needed is only a boolean. Pulling OFFLOAD_SUCCESS/FAIL only adds confusion.

Differential Revision: https:

[OpenMP][libomptarget][NFC] Change checkDeviceAndCtors return type to bool.

What is exactly needed is only a boolean. Pulling OFFLOAD_SUCCESS/FAIL only adds confusion.

Differential Revision: https://reviews.llvm.org/D109303

show more ...

[OpenMP][libomptarget] Change device vector elements to unique_ptr type

Using std::vector<DeviceTy> requires implementing copy constructor and copied assign operator for DeviceTy.
Indeed DeviceTy sh

[OpenMP][libomptarget] Change device vector elements to unique_ptr type

Using std::vector<DeviceTy> requires implementing copy constructor and copied assign operator for DeviceTy.
Indeed DeviceTy should never be copied. After changing to std::vector<std::unique_ptr<DeviceTy>>,
All the unsafe copy constructor and copy assign operator implementations can be removed.
Compilers mark them deleted due to mutex or underlying objects and this is the desired behavior.

Differential Revision: https://reviews.llvm.org/D109276

show more ...

[OpenMP][Offloading] Remove task wait in nowait interfaces

All `nowait` series of interfaces in `libomptarget` accept four more arguments (`int32_t depNum, void *depList, int32_t noAliasDepNum, void

[OpenMP][Offloading] Remove task wait in nowait interfaces

All `nowait` series of interfaces in `libomptarget` accept four more arguments (`int32_t depNum, void *depList, int32_t noAliasDepNum, void *noAliasDepList`) compared with their counterparts w/o `nowait`. These extra arguments were expected for dependence resolution, potentially lowered to device side. Current implementation calls `libomp` function `__kmpc_omp_taskwait`. However, the front end simply ignores them, that these four arguments are not emitted at all. As a consequence, the `depNum` and `noAliasDepNum` are garbage, which could lead to unnecessary task wait.

Reviewed By: grokos

Differential Revision: https://reviews.llvm.org/D107164

show more ...

[OpenMP][Tool] Introducing the `llvm-omp-device-info` tool

This patch introduces the `llvm-omp-device-info` tool, which uses the
omptarget library and interface to query the device info from all the

[OpenMP][Tool] Introducing the `llvm-omp-device-info` tool

This patch introduces the `llvm-omp-device-info` tool, which uses the
omptarget library and interface to query the device info from all the
available devices as seen by OpenMP. This is inspired by PGI's `pgaccelinfo`

Since omptarget usually requires a description structure with executable
kernels, I split the initialization of the RTLs and Devices to be able to
initialize all possible devices and query each of them.

This revision relies on the patch that introduces the print device info.

A limitation is that the order in which the devices are initialized, and the
corresponding device ID is not necesarily the one seen by OpenMP.

The changes are as follows:
1. Separate the RTL initialization that was performed in `RegisterLib` to its own `initRTLonce` function
2. Create an `initAllRTLs` method that initializes all available RTLs at runtime
3. Created the `llvm-deviceinfo.cpp` tool that uses `omptarget` to query each device and prints its information.

Example Output:
```
Device (0):
print_device_info not implemented

Device (1):
print_device_info not implemented

Device (2):
print_device_info not implemented

Device (3):
print_device_info not implemented

Device (4):
CUDA Driver Version: 11000
CUDA Device Number: 0
Device Name: Quadro P1000
Global Memory Size: 4236312576 bytes
Number of Multiprocessors: 5
Concurrent Copy and Execution: Yes
Total Constant Memory: 65536 bytes
Max Shared Memory per Block: 49152 bytes
Registers per Block: 65536
Warp Size: 32 Threads
Maximum Threads per Block: 1024
Maximum Block Dimensions: 1024, 1024, 64
Maximum Grid Dimensions: 2147483647 x 65535 x 65535
Maximum Memory Pitch: 2147483647 bytes
Texture Alignment: 512 bytes
Clock Rate: 1480500 kHz
Execution Timeout: Yes
Integrated Device: No
Can Map Host Memory: Yes
Compute Mode: DEFAULT
Concurrent Kernels: Yes
ECC Enabled: No
Memory Clock Rate: 2505000 kHz
Memory Bus Width: 128 bits
L2 Cache Size: 1048576 bytes
Max Threads Per SMP: 2048
Async Engines: Yes (2)
Unified Addressing: Yes
Managed Memory: Yes
Concurrent Managed Memory: Yes
Preemption Supported: Yes
Cooperative Launch: Yes
Multi-Device Boars: No
Compute Capabilities: 61
```

Reviewed By: tianshilei1992

Differential Revision: https://reviews.llvm.org/D106752

show more ...

[OpenMP][Libomptarget] Adding `print_device_info` to RTL and `omptarget`

This patch introduces a function in the device's plugin to print the
device information. This patch relates to another patch

[OpenMP][Libomptarget] Adding `print_device_info` to RTL and `omptarget`

This patch introduces a function in the device's plugin to print the
device information. This patch relates to another patch that introduces
a CLI tool to obtain the device information from the omplibrary directly.
It is inspired by PGI's pgaccelinfo.

The modifications are as follows:
1. Introduce the optional `void __tgt_rtl_print_device_info(RTLdevID)` function into the RTL.
2. Introduce the `bool __tgt_print_device_info(devID)` function into `omptarget` interface. Returns false if the RTL is not implemented
3. Added `bool printDeviceInfo(RTLDevID)` to the `DeviceTy`
4. Implement the `__tgt_rtl_print_device_info` for CUDA. Added additional CUDA Runtime calls.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D106751

show more ...

[OpenMP] Replace global InfoLevel with a reference to an internal one.

Summary:
This patch improves the implementation of D100774 by replacing the global
variable introduced with a function that ret

[OpenMP] Replace global InfoLevel with a reference to an internal one.

Summary:
This patch improves the implementation of D100774 by replacing the global
variable introduced with a function that returns a reference to an internal
one. This removes the need to define the variable in every plugin that uses it.

Reviewed By: JonChesterfield

Differential Revision: https://reviews.llvm.org/D101102

show more ...

[OpenMP] Add function for setting LIBOMPTARGET_INFO at runtime

Summary:
This patch adds a new runtime function __tgt_set_info_flag that allows the
user to set the information level at runtime withou

[OpenMP] Add function for setting LIBOMPTARGET_INFO at runtime

Summary:
This patch adds a new runtime function __tgt_set_info_flag that allows the
user to set the information level at runtime without using the environment
variable. Using this will require an extern function, but will eventually be
added into an auxilliary library for OpenMP support functions.

This patch required moving the current InfoLevel to a global variable which must
be instantiated by each plugin.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D100774

show more ...

[OpenMP] Restore backwards compatibility for libomptarget

Summary:
The changes introduced in D87946 changed the API for libomptarget
functions. `__kmpc_push_target_tripcount` was a function in Clang

[OpenMP] Restore backwards compatibility for libomptarget

Summary:
The changes introduced in D87946 changed the API for libomptarget
functions. `__kmpc_push_target_tripcount` was a function in Clang 11.x
but was not given a backward-compatible interface. This change will
require people using Clang 13.x or 12.x to recompile their offloading
programs.

Reviewed By: jdoerfert cchen

Differential Revision: https://reviews.llvm.org/D98358

show more ...

[OpenMP] Encapsulate more in checkDeviceAndCtors

This patch just encapsulates some repeated code. To do so, it
relocates some functions from interface.cpp to omptarget.cpp. It also
adjusts them to

[OpenMP] Encapsulate more in checkDeviceAndCtors

This patch just encapsulates some repeated code. To do so, it
relocates some functions from interface.cpp to omptarget.cpp. It also
adjusts them to the LLVM coding style.

This patch is almost NFC except some `DP` messages are a bit
different. For example, messages like "Entering target region" are
now emitted even if offload is disabled, but a subsequent "Offload is
disabled" is then emitted.

Reviewed By: jdoerfert, grokos

Differential Revision: https://reviews.llvm.org/D97908

show more ...

[OpenMP] Fix lone target exit data

Without this patch, an `omp target exit data` before the runtime is
initialized produces a runtime error. This patch fixes that by
changing `__tgt_target_data_end

[OpenMP] Fix lone target exit data

Without this patch, an `omp target exit data` before the runtime is
initialized produces a runtime error. This patch fixes that by
changing `__tgt_target_data_end_mapper` to call `CheckDeviceAndCtors`
like many other runtime routines.

Discussed at
<https://lists.llvm.org/pipermail/openmp-dev/2021-March/003920.html>.

Reviewed By: grokos

Differential Revision: https://reviews.llvm.org/D97907

show more ...

[OpenMP] Fix support for device as host

Without this patch, when the offload device is set to
`omp_get_initial_device()`, the runtime fails with an error diagnostic
when entering target regions or t

[OpenMP] Fix support for device as host

Without this patch, when the offload device is set to
`omp_get_initial_device()`, the runtime fails with an error diagnostic
when entering target regions or target data regions.

However, OpenMP 5.1, sec. 2.14.5 "target Construct", "Restrictions",
p. 203, L3-5 states:

> The device clause expression must evaluate to a non-negative integer
> value that is less than or equal to the value of
> omp_get_num_devices().

Sec. 3.7.7 "omp_get_initial_device", p. 412, L2-3 states:

> The value of the device number is the value returned by the
> omp_get_num_devices routine.

Similarly, OpenMP 5.0, sec. 2.12.5 "target Construct", "Restrictions",
p. 174 L30-32 states:

> The device clause expression must evaluate to a non-negative integer
> value less than the value of omp_get_num_devices() or to the value
> of omp_get_initial_device().

This patch fixes this behavior by changing the runtime to behave as if
offloading is disabled whenever it finds the offload device (either
from a `device` clause or the default device) is set to the host
device. In the case of mandatory offloading when
`omp_get_num_devices() == 0`, it incorporates the behavior proposed
for OpenMP 5.2 in OpenMP spec github issue 2669.

Reviewed By: grokos, RaviNarayanaswamy

Differential Revision: https://reviews.llvm.org/D97616

show more ...

[OpenMP] Unify omptarget API and usage wrt. `__tgt_async_info`

This patch unifies our libomptarget API in two ways:
- always pass a `__tgt_async_info` object, the Queue member decides if
it is

[OpenMP] Unify omptarget API and usage wrt. `__tgt_async_info`

This patch unifies our libomptarget API in two ways:
- always pass a `__tgt_async_info` object, the Queue member decides if
it is in use or not.
- (almost) always synchronize in the interface layer and not in the
omptarget layer.

A side effect is that we now put all constructor and static initializer
kernels in a stream too, if the device utilizes `__tgt_async_info`.

The patch contains a TODO which can be addressed as we add support for
asynchronous malloc and free in the plugin API. This is the only
`synchronizeAsyncInfo` left in the omptarget layer.

Site note: On a V100 system the GridMini performance for small sizes
more than doubled.

Reviewed By: tianshilei1992

Differential Revision: https://reviews.llvm.org/D96379

show more ...

[OpenMP][NFC] Unify `target` API with other by passing a `__tgt_async_info` pointer

Reviewed By: tianshilei1992

Differential Revision: https://reviews.llvm.org/D96430

[OpenMP][NFC] Pass a DeviceTy, not the device number to `target`

This unifies the API of `target` relative to `targetUpdateData` and
such.

Reviewed By: tianshilei1992, grokos

Differential Revision

[OpenMP][NFC] Pass a DeviceTy, not the device number to `target`

This unifies the API of `target` relative to `targetUpdateData` and
such.

Reviewed By: tianshilei1992, grokos

Differential Revision: https://reviews.llvm.org/D96429

show more ...

[OpenMP][NFC] Clang format libomptarget code (src & include)

The struct and enum alignments are kept by disabling clang-format for
that code region.

Reviewed By: tianshilei1992, JonChesterfield, gr

[OpenMP][NFC] Clang format libomptarget code (src & include)

The struct and enum alignments are kept by disabling clang-format for
that code region.

Reviewed By: tianshilei1992, JonChesterfield, grokos

Differential Revision: https://reviews.llvm.org/D96428

show more ...

[OpenMP][Libomptarget] Remove possible harmful copy constructor call for RTLsTy

From https://bugs.llvm.org/show_bug.cgi?id=48973, we know that
`std::call_once(PM->RTLs.initFlag, &RTLsTy::LoadRTLs, P

[OpenMP][Libomptarget] Remove possible harmful copy constructor call for RTLsTy

From https://bugs.llvm.org/show_bug.cgi?id=48973, we know that
`std::call_once(PM->RTLs.initFlag, &RTLsTy::LoadRTLs, PM->RTLs)` causes compile
time problems in libstdc++v3 5.3.1. This is because there was a defect in the
standard regarding the `call_once` (LWG 2442). This was fixed in libstdc++ soon
thereafter, but there are likely other standard libraries where this will fail.

By matching this function call with the other one, we fix this bug.

Differential Revision: https://reviews.llvm.org/D95769

show more ...

[OpenMP] Fix seg fault in libomptarget when using Info with multiple threads

Summary:
One option for the LIBOMPTARGET_INFO environment variable is to print the current status of the device's data ma

[OpenMP] Fix seg fault in libomptarget when using Info with multiple threads

Summary:
One option for the LIBOMPTARGET_INFO environment variable is to print the current status of the device's data mappings. These are a shared resource among threads so this needs to be protected when using multiple streams.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D95786

show more ...

[OpenMP][Libomptarget] Introduce changes to support remote plugin

In order to support remote execution, we need to be able to send the
target binary description to the remote host for registration (

[OpenMP][Libomptarget] Introduce changes to support remote plugin

In order to support remote execution, we need to be able to send the
target binary description to the remote host for registration (and
consequent deregistration). To support this, I added these two
optional new functions to the plugin API:
- `__tgt_rtl_register_lib`
- `__tgt_rtl_unregister_lib`

These functions will be called to properly manage the instance of
libomptarget running on the remote host.

Reviewed By: JonChesterfield

Differential Revision: https://reviews.llvm.org/D93293

show more ...

[OpenMP] Add source location information to the libomptarget profile

In much of the libomptarget interface we have an ident_t object now, if
it is not null we can use it to improve the profile outpu

[OpenMP] Add source location information to the libomptarget profile

In much of the libomptarget interface we have an ident_t object now, if
it is not null we can use it to improve the profile output. For now, we
simply use the ident_t "source information string" as generated by the
FE.

Reviewed By: tianshilei1992

Differential Revision: https://reviews.llvm.org/D95282

show more ...

[OpenMP] Add support for mapping names in mapper API

Summary:
The custom mapper API did not previously support the mapping names added previously. This means they were not present if a user requeste

[OpenMP] Add support for mapping names in mapper API

Summary:
The custom mapper API did not previously support the mapping names added previously. This means they were not present if a user requested debugging information while using the mapper functions. This adds basic support for passing the mapped names to the runtime library.

Reviewers: jdoerfert

Differential Revision: https://reviews.llvm.org/D94806

show more ...