1Support, Getting Involved, and FAQ
2==================================
3
4Please do not hesitate to reach out to us via [email protected] or join
5one of our :ref:`regular calls <calls>`. Some common questions are answered in
6the :ref:`faq`.
7
8.. _calls:
9
10Calls
11-----
12
13OpenMP in LLVM Technical Call
14^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
15
16-   Development updates on OpenMP (and OpenACC) in the LLVM Project, including Clang, optimization, and runtime work.
17-   Join `OpenMP in LLVM Technical Call <https://bluejeans.com/544112769//webrtc>`__.
18-   Time: Weekly call on every Wednesday 7:00 AM Pacific time.
19-   Meeting minutes are `here <https://docs.google.com/document/d/1Tz8WFN13n7yJ-SCE0Qjqf9LmjGUw0dWO9Ts1ss4YOdg/edit>`__.
20-   Status tracking `page <https://openmp.llvm.org/docs>`__.
21
22
23OpenMP in Flang Technical Call
24^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
25-   Development updates on OpenMP and OpenACC in the Flang Project.
26-   Join `OpenMP in Flang Technical Call <https://bit.ly/39eQW3o>`_
27-   Time: Weekly call on every Thursdays 8:00 AM Pacific time.
28-   Meeting minutes are `here <https://docs.google.com/document/d/1yA-MeJf6RYY-ZXpdol0t7YoDoqtwAyBhFLr5thu5pFI>`__.
29-   Status tracking `page <https://docs.google.com/spreadsheets/d/1FvHPuSkGbl4mQZRAwCIndvQx9dQboffiD-xD0oqxgU0/edit#gid=0>`__.
30
31
32.. _faq:
33
34FAQ
35---
36
37.. note::
38   The FAQ is a work in progress and most of the expected content is not
39   yet available. While you can expect changes, we always welcome feedback and
40   additions. Please contact, e.g., through ``[email protected]``.
41
42
43Q: How to contribute a patch to the webpage or any other part?
44^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
45
46All patches go through the regular `LLVM review process
47<https://llvm.org/docs/Contributing.html#how-to-submit-a-patch>`_.
48
49
50.. _build_offload_capable_compiler:
51
52Q: How to build an OpenMP GPU offload capable compiler?
53^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
54To build an *effective* OpenMP offload capable compiler, only one extra CMake
55option, `LLVM_ENABLE_RUNTIMES="openmp"`, is needed when building LLVM (Generic
56information about building LLVM is available `here <https://llvm.org/docs/GettingStarted.html>`__.).
57Make sure all backends that are targeted by OpenMP to be enabled. By default,
58Clang will be built with all backends enabled.
59
60For Nvidia offload, please see :ref:`_build_nvidia_offload_capable_compiler`.
61For AMDGPU offload, please see :ref:`_build_amdgpu_offload_capable_compiler`.
62
63.. note::
64  The compiler that generates the offload code should be the same (version) as
65  the compiler that builds the OpenMP device runtimes. The OpenMP host runtime
66  can be built by a different compiler.
67
68.. _advanced_builds: https://llvm.org//docs/AdvancedBuilds.html
69
70.. _build_nvidia_offload_capable_compiler:
71
72Q: How to build an OpenMP NVidia offload capable compiler?
73^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
74The Cuda SDK is required on the machine that will execute the openmp application.
75
76If your build machine is not the target machine or automatic detection of the
77available GPUs failed, you should also set:
78
79- `CLANG_OPENMP_NVPTX_DEFAULT_ARCH=sm_XX` where `XX` is the architecture of your GPU, e.g, 80.
80- `LIBOMPTARGET_NVPTX_COMPUTE_CAPABILITIES=YY` where `YY` is the numeric compute capacity of your GPU, e.g., 75.
81
82
83.. _build_amdgpu_offload_capable_compiler:
84
85Q: How to build an OpenMP AMDGPU offload capable compiler?
86^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
87A subset of the `ROCm <https://github.com/radeonopencompute>` toolchain is
88required to build the LLVM toolchain and to execute the openmp application.
89Either install ROCm somewhere that cmake's find_package can locate it, or
90build the required subcomponents ROCt and ROCr from source.
91
92The two components used are ROCT-Thunk-Interface, roct, and ROCR-Runtime,
93rocr. Roct is the userspace part of the linux driver. It calls into the
94driver which ships with the linux kernel. It is an implementation detail of
95Rocr from OpenMP's perspective. Rocr is an implementation of `HSA <http://www.hsafoundation.com>`.
96
97    SOURCE_DIR=same-as-llvm-source # e.g. the checkout of llvm-project, next to openmp
98    BUILD_DIR=somewhere
99    INSTALL_PREFIX=same-as-llvm-install
100
101    cd $SOURCE_DIR
102    git clone [email protected]:RadeonOpenCompute/ROCT-Thunk-Interface.git -b roc-4.1.x --single-branch
103    git clone [email protected]:RadeonOpenCompute/ROCR-Runtime.git -b rocm-4.1.x --single-branch
104
105    cd $BUILD_DIR && mkdir roct && cd roct
106    cmake $SOURCE_DIR/ROCT-Thunk-Interface/ -DCMAKE_INSTALL_PREFIX=$INSTALL_PREFIX -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF
107    make && make install
108
109    cd $BUILD_DIR && mkdir rocr && cd rocr
110    cmake $SOURCE_DIR/ROCR-Runtime/src -DIMAGE_SUPPORT=OFF -DCMAKE_INSTALL_PREFIX=$INSTALL_PREFIX -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=ON
111    make && make install
112
113IMAGE_SUPPORT requires building rocr with clang and is not used by openmp.
114
115Provided cmake's find_package can find the ROCR-Runtime package, LLVM will
116build a tool `bin/amdgpu-arch` which will print a string like 'gfx906' when
117run if it recognises a GPU on the local system. LLVM will also build a shared
118library, libomptarget.rtl.amdgpu.so, which is linked against rocr.
119
120With those libraries installed, then LLVM build and installed, try:
121
122    clang -O2 -fopenmp -fopenmp-targets=amdgcn-amd-amdhsa example.c -o example && ./example
123
124Q: What are the known limitations of OpenMP AMDGPU offload?
125^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
126LD_LIBRARY_PATH is presently required to find the openmp libraries.
127
128There is no libc. That is, malloc and printf do not exist. Also no libm, so
129functions like cos(double) will not work from target regions.
130
131Cards from the gfx10 line, 'navi', that use wave32 are not yet implemented.
132
133Some versions of the driver for the radeon vii (gfx906) will error unless the
134environment variable 'export HSA_IGNORE_SRAMECC_MISREPORT=1' is set.
135
136It is a recent addition to LLVM and the implementation differs from that which
137has been shipping in ROCm and AOMP for some time. Early adopters will encounter
138bugs.
139
140Q: Does OpenMP offloading support work in pre-packaged LLVM releases?
141^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
142For now, the answer is most likely *no*. Please see :ref:`build_offload_capable_compiler`.
143
144Q: Does OpenMP offloading support work in packages distributed as part of my OS?
145^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
146For now, the answer is most likely *no*. Please see :ref:`build_offload_capable_compiler`.
147
148
149.. _math_and_complex_in_target_regions:
150
151Q: Does Clang support `<math.h>` and `<complex.h>` operations in OpenMP target on GPUs?
152^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
153
154Yes, LLVM/Clang allows math functions and complex arithmetic inside of OpenMP target regions
155that are compiled for GPUs.
156
157Clang provides a set of wrapper headers that are found first when `math.h` and
158`complex.h`, for C, `cmath` and `complex`, for C++, or similar headers are
159included by the application. These wrappers will eventually include the system
160version of the corresponding header file after setting up a target device
161specific environment. The fact that the system header is included is important
162because they differ based on the architecture and operating system and may
163contain preprocessor, variable, and function definitions that need to be
164available in the target region regardless of the targeted device architecture.
165However, various functions may require specialized device versions, e.g.,
166`sin`, and others are only available on certain devices, e.g., `__umul64hi`. To
167provide "native" support for math and complex on the respective architecture,
168Clang will wrap the "native" math functions, e.g., as provided by the device
169vendor, in an OpenMP begin/end declare variant. These functions will then be
170picked up instead of the host versions while host only variables and function
171definitions are still available. Complex arithmetic and functions are support
172through a similar mechanism. It is worth noting that this support requires
173`extensions to the OpenMP begin/end declare variant context selector
174<https://clang.llvm.org/docs/AttributeReference.html#pragma-omp-declare-variant>`__
175that are exposed through LLVM/Clang to the user as well.
176
177Q: What is a way to debug errors from mapping memory to a target device?
178^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
179
180An experimental way to debug these errors is to use :ref:`remote process
181offloading <remote_offloading_plugin>`.
182By using ``libomptarget.rtl.rpc.so`` and ``openmp-offloading-server``, it is
183possible to explicitly perform memory transfers between processes on the host
184CPU and run sanitizers while doing so in order to catch these errors.
185
186Q: Why does my application say "Named symbol not found" and abort when I run it?
187^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
188
189This is most likely caused by trying to use OpenMP offloading with static
190libraries. Static libraries do not contain any device code, so when the runtime
191attempts to execute the target region it will not be found and you will get an
192an error like this.
193
194.. code-block:: text
195
196   CUDA error: Loading '__omp_offloading_fd02_3231c15__Z3foov_l2' Failed
197   CUDA error: named symbol not found
198   Libomptarget error: Unable to generate entries table for device id 0.
199
200Currently, the only solution is to change how the application is built and avoid
201the use of static libraries.
202
203Q: Can I use dynamically linked libraries with OpenMP offloading
204^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
205
206Dynamically linked libraries can be only used if there is no device code split
207between the library and application. Anything declared on the device inside the
208shared library will not be visible to the application when it's linked.
209