1Support, Getting Involved, and FAQ
2==================================
3
4Please do not hesitate to reach out to us via [email protected] or join
5one of our :ref:`regular calls <calls>`. Some common questions are answered in
6the :ref:`faq`.
7
8.. _calls:
9
10Calls
11-----
12
13OpenMP in LLVM Technical Call
14^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
15
16-   Development updates on OpenMP (and OpenACC) in the LLVM Project, including Clang, optimization, and runtime work.
17-   Join `OpenMP in LLVM Technical Call <https://bluejeans.com/544112769//webrtc>`__.
18-   Time: Weekly call on every Wednesday 7:00 AM Pacific time.
19-   Meeting minutes are `here <https://docs.google.com/document/d/1Tz8WFN13n7yJ-SCE0Qjqf9LmjGUw0dWO9Ts1ss4YOdg/edit>`__.
20-   Status tracking `page <https://openmp.llvm.org/docs>`__.
21
22
23OpenMP in Flang Technical Call
24^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
25-   Development updates on OpenMP and OpenACC in the Flang Project.
26-   Join `OpenMP in Flang Technical Call <https://bit.ly/39eQW3o>`_
27-   Time: Weekly call on every Thursdays 8:00 AM Pacific time.
28-   Meeting minutes are `here <https://docs.google.com/document/d/1yA-MeJf6RYY-ZXpdol0t7YoDoqtwAyBhFLr5thu5pFI>`__.
29-   Status tracking `page <https://docs.google.com/spreadsheets/d/1FvHPuSkGbl4mQZRAwCIndvQx9dQboffiD-xD0oqxgU0/edit#gid=0>`__.
30
31
32.. _faq:
33
34FAQ
35---
36
37.. note::
38   The FAQ is a work in progress and most of the expected content is not
39   yet available. While you can expect changes, we always welcome feedback and
40   additions. Please contact, e.g., through ``[email protected]``.
41
42
43Q: How to contribute a patch to the webpage or any other part?
44^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
45
46All patches go through the regular `LLVM review process
47<https://llvm.org/docs/Contributing.html#how-to-submit-a-patch>`_.
48
49
50.. _build_offload_capable_compiler:
51
52Q: How to build an OpenMP GPU offload capable compiler?
53^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
54To build an *effective* OpenMP offload capable compiler, only one extra CMake
55option, `LLVM_ENABLE_RUNTIMES="openmp"`, is needed when building LLVM (Generic
56information about building LLVM is available `here <https://llvm.org/docs/GettingStarted.html>`__.).
57Make sure all backends that are targeted by OpenMP to be enabled. By default,
58Clang will be built with all backends enabled.
59When building with `LLVM_ENABLE_RUNTIMES="openmp"` OpenMP should not be enabled
60in `LLVM_ENABLE_PROJECTS` because it is enabled by default.
61
62For Nvidia offload, please see :ref:`_build_nvidia_offload_capable_compiler`.
63For AMDGPU offload, please see :ref:`_build_amdgpu_offload_capable_compiler`.
64
65.. note::
66  The compiler that generates the offload code should be the same (version) as
67  the compiler that builds the OpenMP device runtimes. The OpenMP host runtime
68  can be built by a different compiler.
69
70.. _advanced_builds: https://llvm.org//docs/AdvancedBuilds.html
71
72.. _build_nvidia_offload_capable_compiler:
73
74Q: How to build an OpenMP NVidia offload capable compiler?
75^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
76The Cuda SDK is required on the machine that will execute the openmp application.
77
78If your build machine is not the target machine or automatic detection of the
79available GPUs failed, you should also set:
80
81- `CLANG_OPENMP_NVPTX_DEFAULT_ARCH=sm_XX` where `XX` is the architecture of your GPU, e.g, 80.
82- `LIBOMPTARGET_NVPTX_COMPUTE_CAPABILITIES=YY` where `YY` is the numeric compute capacity of your GPU, e.g., 75.
83
84
85.. _build_amdgpu_offload_capable_compiler:
86
87Q: How to build an OpenMP AMDGPU offload capable compiler?
88^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
89A subset of the `ROCm <https://github.com/radeonopencompute>` toolchain is
90required to build the LLVM toolchain and to execute the openmp application.
91Either install ROCm somewhere that cmake's find_package can locate it, or
92build the required subcomponents ROCt and ROCr from source.
93
94The two components used are ROCT-Thunk-Interface, roct, and ROCR-Runtime,
95rocr. Roct is the userspace part of the linux driver. It calls into the
96driver which ships with the linux kernel. It is an implementation detail of
97Rocr from OpenMP's perspective. Rocr is an implementation of `HSA <http://www.hsafoundation.com>`.
98
99    SOURCE_DIR=same-as-llvm-source # e.g. the checkout of llvm-project, next to openmp
100    BUILD_DIR=somewhere
101    INSTALL_PREFIX=same-as-llvm-install
102
103    cd $SOURCE_DIR
104    git clone [email protected]:RadeonOpenCompute/ROCT-Thunk-Interface.git -b roc-4.1.x --single-branch
105    git clone [email protected]:RadeonOpenCompute/ROCR-Runtime.git -b rocm-4.1.x --single-branch
106
107    cd $BUILD_DIR && mkdir roct && cd roct
108    cmake $SOURCE_DIR/ROCT-Thunk-Interface/ -DCMAKE_INSTALL_PREFIX=$INSTALL_PREFIX -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF
109    make && make install
110
111    cd $BUILD_DIR && mkdir rocr && cd rocr
112    cmake $SOURCE_DIR/ROCR-Runtime/src -DIMAGE_SUPPORT=OFF -DCMAKE_INSTALL_PREFIX=$INSTALL_PREFIX -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=ON
113    make && make install
114
115IMAGE_SUPPORT requires building rocr with clang and is not used by openmp.
116
117Provided cmake's find_package can find the ROCR-Runtime package, LLVM will
118build a tool `bin/amdgpu-arch` which will print a string like 'gfx906' when
119run if it recognises a GPU on the local system. LLVM will also build a shared
120library, libomptarget.rtl.amdgpu.so, which is linked against rocr.
121
122With those libraries installed, then LLVM build and installed, try:
123
124    clang -O2 -fopenmp -fopenmp-targets=amdgcn-amd-amdhsa example.c -o example && ./example
125
126Q: What are the known limitations of OpenMP AMDGPU offload?
127^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
128LD_LIBRARY_PATH is presently required to find the openmp libraries.
129
130There is no libc. That is, malloc and printf do not exist. Also no libm, so
131functions like cos(double) will not work from target regions.
132
133Cards from the gfx10 line, 'navi', that use wave32 are not yet implemented.
134
135Some versions of the driver for the radeon vii (gfx906) will error unless the
136environment variable 'export HSA_IGNORE_SRAMECC_MISREPORT=1' is set.
137
138It is a recent addition to LLVM and the implementation differs from that which
139has been shipping in ROCm and AOMP for some time. Early adopters will encounter
140bugs.
141
142Q: Does OpenMP offloading support work in pre-packaged LLVM releases?
143^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
144For now, the answer is most likely *no*. Please see :ref:`build_offload_capable_compiler`.
145
146Q: Does OpenMP offloading support work in packages distributed as part of my OS?
147^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
148For now, the answer is most likely *no*. Please see :ref:`build_offload_capable_compiler`.
149
150
151.. _math_and_complex_in_target_regions:
152
153Q: Does Clang support `<math.h>` and `<complex.h>` operations in OpenMP target on GPUs?
154^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
155
156Yes, LLVM/Clang allows math functions and complex arithmetic inside of OpenMP target regions
157that are compiled for GPUs.
158
159Clang provides a set of wrapper headers that are found first when `math.h` and
160`complex.h`, for C, `cmath` and `complex`, for C++, or similar headers are
161included by the application. These wrappers will eventually include the system
162version of the corresponding header file after setting up a target device
163specific environment. The fact that the system header is included is important
164because they differ based on the architecture and operating system and may
165contain preprocessor, variable, and function definitions that need to be
166available in the target region regardless of the targeted device architecture.
167However, various functions may require specialized device versions, e.g.,
168`sin`, and others are only available on certain devices, e.g., `__umul64hi`. To
169provide "native" support for math and complex on the respective architecture,
170Clang will wrap the "native" math functions, e.g., as provided by the device
171vendor, in an OpenMP begin/end declare variant. These functions will then be
172picked up instead of the host versions while host only variables and function
173definitions are still available. Complex arithmetic and functions are support
174through a similar mechanism. It is worth noting that this support requires
175`extensions to the OpenMP begin/end declare variant context selector
176<https://clang.llvm.org/docs/AttributeReference.html#pragma-omp-declare-variant>`__
177that are exposed through LLVM/Clang to the user as well.
178
179Q: What is a way to debug errors from mapping memory to a target device?
180^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
181
182An experimental way to debug these errors is to use :ref:`remote process
183offloading <remote_offloading_plugin>`.
184By using ``libomptarget.rtl.rpc.so`` and ``openmp-offloading-server``, it is
185possible to explicitly perform memory transfers between processes on the host
186CPU and run sanitizers while doing so in order to catch these errors.
187
188Q: Why does my application say "Named symbol not found" and abort when I run it?
189^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
190
191This is most likely caused by trying to use OpenMP offloading with static
192libraries. Static libraries do not contain any device code, so when the runtime
193attempts to execute the target region it will not be found and you will get an
194an error like this.
195
196.. code-block:: text
197
198   CUDA error: Loading '__omp_offloading_fd02_3231c15__Z3foov_l2' Failed
199   CUDA error: named symbol not found
200   Libomptarget error: Unable to generate entries table for device id 0.
201
202Currently, the only solution is to change how the application is built and avoid
203the use of static libraries.
204
205Q: Can I use dynamically linked libraries with OpenMP offloading
206^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
207
208Dynamically linked libraries can be only used if there is no device code split
209between the library and application. Anything declared on the device inside the
210shared library will not be visible to the application when it's linked.
211
212Q: How to build an OpenMP offload capable compiler with an outdated host compiler?
213^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
214
215Enabling the OpenMP runtime will perform a two-stage build for you.
216If your host compiler is different from your system-wide compiler, you may need
217to set the CMake variable `GCC_INSTALL_PREFIX` so clang will be able to find the
218correct GCC toolchain in the second stage of the build.
219
220For example, if your system-wide GCC installation is too old to build LLVM and
221you would like to use a newer GCC, set the CMake variable `GCC_INSTALL_PREFIX`
222to inform clang of the GCC installation you would like to use in the second stage.
223