1==================== 2XRay Instrumentation 3==================== 4 5:Version: 1 as of 2016-11-08 6 7.. contents:: 8 :local: 9 10 11Introduction 12============ 13 14XRay is a function call tracing system which combines compiler-inserted 15instrumentation points and a runtime library that can dynamically enable and 16disable the instrumentation. 17 18More high level information about XRay can be found in the `XRay whitepaper`_. 19 20This document describes how to use XRay as implemented in LLVM. 21 22XRay in LLVM 23============ 24 25XRay consists of three main parts: 26 27- Compiler-inserted instrumentation points. 28- A runtime library for enabling/disabling tracing at runtime. 29- A suite of tools for analysing the traces. 30 31 **NOTE:** As of the time of this writing, XRay is only available for x86_64 32 and arm7 32-bit (no-thumb) Linux. 33 34The compiler-inserted instrumentation points come in the form of nop-sleds in 35the final generated binary, and an ELF section named ``xray_instr_map`` which 36contains entries pointing to these instrumentation points. The runtime library 37relies on being able to access the entries of the ``xray_instr_map``, and 38overwrite the instrumentation points at runtime. 39 40Using XRay 41========== 42 43You can use XRay in a couple of ways: 44 45- Instrumenting your C/C++/Objective-C/Objective-C++ application. 46- Generating LLVM IR with the correct function attributes. 47 48The rest of this section covers these main ways and later on how to customise 49what XRay does in an XRay-instrumented binary. 50 51Instrumenting your C/C++/Objective-C Application 52------------------------------------------------ 53 54The easiest way of getting XRay instrumentation for your application is by 55enabling the ``-fxray-instrument`` flag in your clang invocation. 56 57For example: 58 59:: 60 61 clang -fxray-instrument .. 62 63By default, functions that have at least 200 instructions will get XRay 64instrumentation points. You can tweak that number through the 65``-fxray-instruction-threshold=`` flag: 66 67:: 68 69 clang -fxray-instrument -fxray-instruction-threshold=1 .. 70 71You can also specifically instrument functions in your binary to either always 72or never be instrumented using source-level attributes. You can do it using the 73GCC-style attributes or C++11-style attributes. 74 75.. code-block:: c++ 76 77 [[clang::xray_always_intrument]] void always_instrumented(); 78 79 [[clang::xray_never_instrument]] void never_instrumented(); 80 81 void alt_always_instrumented() __attribute__((xray_always_intrument)); 82 83 void alt_never_instrumented() __attribute__((xray_never_instrument)); 84 85When linking a binary, you can either manually link in the `XRay Runtime 86Library`_ or use ``clang`` to link it in automatically with the 87``-fxray-instrument`` flag. 88 89LLVM Function Attribute 90----------------------- 91 92If you're using LLVM IR directly, you can add the ``function-instrument`` 93string attribute to your functions, to get the similar effect that the 94C/C++/Objective-C source-level attributes would get: 95 96.. code-block:: llvm 97 98 define i32 @always_instrument() uwtable "function-instrument"="xray-always" { 99 ; ... 100 } 101 102 define i32 @never_instrument() uwtable "function-instrument"="xray-never" { 103 ; ... 104 } 105 106You can also set the ``xray-instruction-threshold`` attribute and provide a 107numeric string value for how many instructions should be in the function before 108it gets instrumented. 109 110.. code-block:: llvm 111 112 define i32 @maybe_instrument() uwtable "xray-instruction-threshold"="2" { 113 ; ... 114 } 115 116XRay Runtime Library 117-------------------- 118 119The XRay Runtime Library is part of the compiler-rt project, which implements 120the runtime components that perform the patching and unpatching of inserted 121instrumentation points. When you use ``clang`` to link your binaries and the 122``-fxray-instrument`` flag, it will automatically link in the XRay runtime. 123 124The default implementation of the XRay runtime will enable XRay instrumentation 125before ``main`` starts, which works for applications that have a short 126lifetime. This implementation also records all function entry and exit events 127which may result in a lot of records in the resulting trace. 128 129Also by default the filename of the XRay trace is ``xray-log.XXXXXX`` where the 130``XXXXXX`` part is randomly generated. 131 132These options can be controlled through the ``XRAY_OPTIONS`` environment 133variable, where we list down the options and their defaults below. 134 135+-------------------+-----------------+---------------+------------------------+ 136| Option | Type | Default | Description | 137+===================+=================+===============+========================+ 138| patch_premain | ``bool`` | ``true`` | Whether to patch | 139| | | | instrumentation points | 140| | | | before main. | 141+-------------------+-----------------+---------------+------------------------+ 142| xray_naive_log | ``bool`` | ``true`` | Whether to install | 143| | | | the naive log | 144| | | | implementation. | 145+-------------------+-----------------+---------------+------------------------+ 146| xray_logfile_base | ``const char*`` | ``xray-log.`` | Filename base for the | 147| | | | XRay logfile. | 148+-------------------+-----------------+---------------+------------------------+ 149 150If you choose to not use the default logging implementation that comes with the 151XRay runtime and/or control when/how the XRay instrumentation runs, you may use 152the XRay APIs directly for doing so. To do this, you'll need to include the 153``xray_interface.h`` from the compiler-rt ``xray`` directory. The important API 154functions we list below: 155 156- ``__xray_set_handler(void (*entry)(int32_t, XRayEntryType))``: Install your 157 own logging handler for when an event is encountered. See 158 ``xray/xray_interface.h`` for more details. 159- ``__xray_remove_handler()``: Removes whatever the installed handler is. 160- ``__xray_patch()``: Patch all the instrumentation points defined in the 161 binary. 162- ``__xray_unpatch()``: Unpatch the instrumentation points defined in the 163 binary. 164 165There are some requirements on the logging handler to be installed for the 166thread-safety of operations to be performed by the XRay runtime library: 167 168- The function should be thread-safe, as multiple threads may be invoking the 169 function at the same time. If the logging function needs to do 170 synchronisation, it must do so internally as XRay does not provide any 171 synchronisation guarantees outside from the atomicity of updates to the 172 pointer. 173- The pointer provided to ``__xray_set_handler(...)`` must be live even after 174 calls to ``__xray_remove_handler()`` and ``__xray_unpatch()`` have succeeded. 175 XRay cannot guarantee that all threads that have ever gotten a copy of the 176 pointer will not invoke the function. 177 178 179Trace Analysis Tools 180-------------------- 181 182We currently have the beginnings of a trace analysis tool in LLVM, which can be 183found in the ``tools/llvm-xray`` directory. The ``llvm-xray`` tool currently 184supports the following subcommands: 185 186- ``extract``: Extract the instrumentation map from a binary, and return it as 187 YAML. 188 189 190Future Work 191=========== 192 193There are a number of ongoing efforts for expanding the toolset building around 194the XRay instrumentation system. 195 196Flight Data Recorder Mode 197------------------------- 198 199The `XRay whitepaper`_ mentions a mode for when events are kept in memory, and 200have the traces be dumped on demand through a triggering API. This work is 201currently ongoing. 202 203Trace Analysis 204-------------- 205 206There are a few more subcommands making its way to the ``llvm-xray`` tool, that 207are currently under review: 208 209- ``convert``: Turns an XRay trace from one format to another. Currently 210 supporting conversion from the binary XRay log to YAML. 211- ``account``: Do function call accounting based on data in the XRay log. 212 213We have more subcommands and modes that we're thinking of developing, in the 214following forms: 215 216- ``stack``: Reconstruct the function call stacks in a timeline. 217- ``convert``: Converting from one version of the XRay log to another (higher) 218 version, and converting to other trace formats (i.e. Chrome Trace Viewer, 219 pprof, etc.). 220- ``graph``: Generate a function call graph with relative timings and distributions. 221 222More Platforms 223-------------- 224 225Since XRay is only currently available in x86_64 and arm7 32-bit (no-thumb) 226running Linux, we're looking to supporting more platforms (architectures and 227operating systems). 228 229.. References... 230 231.. _`XRay whitepaper`: http://research.google.com/pubs/pub45287.html 232 233