1==========================
2Source-based Code Coverage
3==========================
4
5.. contents::
6   :local:
7
8Introduction
9============
10
11This document explains how to use clang's source-based code coverage feature.
12It's called "source-based" because it operates on AST and preprocessor
13information directly. This allows it to generate very precise coverage data.
14
15Clang ships two other code coverage implementations:
16
17* :doc:`SanitizerCoverage` - A low-overhead tool meant for use alongside the
18  various sanitizers. It can provide up to edge-level coverage.
19
20* gcov - A GCC-compatible coverage implementation which operates on DebugInfo.
21
22From this point onwards "code coverage" will refer to the source-based kind.
23
24The code coverage workflow
25==========================
26
27The code coverage workflow consists of three main steps:
28
291. Compiling with coverage enabled.
30
312. Running the instrumented program.
32
333. Creating coverage reports.
34
35The next few sections work through a complete, copy-'n-paste friendly example
36based on this program:
37
38.. code-block:: cpp
39
40    % cat <<EOF > foo.cc
41    #define BAR(x) ((x) || (x))
42    template <typename T> void foo(T x) {
43      for (unsigned I = 0; I < 10; ++I) { BAR(I); }
44    }
45    int main() {
46      foo<int>(0);
47      foo<float>(0);
48      return 0;
49    }
50    EOF
51
52Compiling with coverage enabled
53===============================
54
55To compile code with coverage enabled pass ``-fprofile-instr-generate
56-fcoverage-mapping`` to the compiler:
57
58.. code-block:: console
59
60    # Step 1: Compile with coverage enabled.
61    % clang++ -fprofile-instr-generate -fcoverage-mapping foo.cc -o foo
62
63Note that linking together code with and without coverage instrumentation is
64supported: any uninstrumented code simply won't be accounted for.
65
66Running the instrumented program
67================================
68
69The next step is to run the instrumented program. When the program exits it
70will write a **raw profile** to the path specified by the ``LLVM_PROFILE_FILE``
71environment variable. If that variable does not exist the profile is written to
72``default.profraw`` in the current directory of the program.
73
74If ``LLVM_PROFILE_FILE`` contains a path to a non-existent directory the
75missing directory structure will be created.  Additionally, the following
76special **pattern strings** are replaced:
77
78* "%p" expands out to the process ID.
79
80* "%h" expands out to the hostname of the machine running the program.
81
82.. code-block:: console
83
84    # Step 2: Run the program.
85    % LLVM_PROFILE_FILE="foo.profraw" ./foo
86
87Creating coverage reports
88=========================
89
90Raw profiles have to be **indexed** before they can be used to generated
91coverage reports. This is done using the "merge" tool in ``llvm-profdata``, so
92named because it can combine and index profiles at the same time:
93
94.. code-block:: console
95
96    # Step 3(a): Index the raw profile.
97    % llvm-profdata merge -sparse foo.profraw -o foo.profdata
98
99There are multiple different ways to render coverage reports. One option is to
100generate a line-oriented report:
101
102.. code-block:: console
103
104    # Step 3(b): Create a line-oriented coverage report.
105    % llvm-cov show ./foo -instr-profile=foo.profdata
106
107To demangle any C++ identifiers in the ouput, use:
108
109.. code-block:: console
110
111    % llvm-cov show ./foo -instr-profile=foo.profdata | c++filt -n
112
113This report includes a summary view as well as dedicated sub-views for
114templated functions and their instantiations. For our example program, we get
115distinct views for ``foo<int>(...)`` and ``foo<float>(...)``.  If
116``-show-line-counts-or-regions`` is enabled, ``llvm-cov`` displays sub-line
117region counts (even in macro expansions):
118
119.. code-block:: cpp
120
121       20|    1|#define BAR(x) ((x) || (x))
122                               ^20     ^2
123        2|    2|template <typename T> void foo(T x) {
124       22|    3|  for (unsigned I = 0; I < 10; ++I) { BAR(I); }
125                                       ^22     ^20  ^20^20
126        2|    4|}
127    ------------------
128    | void foo<int>(int):
129    |      1|    2|template <typename T> void foo(T x) {
130    |     11|    3|  for (unsigned I = 0; I < 10; ++I) { BAR(I); }
131    |                                     ^11     ^10  ^10^10
132    |      1|    4|}
133    ------------------
134    | void foo<float>(int):
135    |      1|    2|template <typename T> void foo(T x) {
136    |     11|    3|  for (unsigned I = 0; I < 10; ++I) { BAR(I); }
137    |                                     ^11     ^10  ^10^10
138    |      1|    4|}
139    ------------------
140
141It's possible to generate a file-level summary of coverage statistics (instead
142of a line-oriented report) with:
143
144.. code-block:: console
145
146    # Step 3(c): Create a coverage summary.
147    % llvm-cov report ./foo -instr-profile=foo.profdata
148    Filename                    Regions    Miss   Cover Functions  Executed
149    -----------------------------------------------------------------------
150    /tmp/foo.cc                      13       0 100.00%         3   100.00%
151    -----------------------------------------------------------------------
152    TOTAL                            13       0 100.00%         3   100.00%
153
154A few final notes:
155
156* The ``-sparse`` flag is optional but can result in dramatically smaller
157  indexed profiles. This option should not be used if the indexed profile will
158  be reused for PGO.
159
160* Raw profiles can be discarded after they are indexed. Advanced use of the
161  profile runtime library allows an instrumented program to merge profiling
162  information directly into an existing raw profile on disk. The details are
163  out of scope.
164
165* The ``llvm-profdata`` tool can be used to merge together multiple raw or
166  indexed profiles. To combine profiling data from multiple runs of a program,
167  try e.g:
168
169.. code-block:: console
170
171    % llvm-profdata merge -sparse foo1.profraw foo2.profdata -o foo3.profdata
172
173Format compatibility guarantees
174===============================
175
176* There are no backwards or forwards compatibility guarantees for the raw
177  profile format. Raw profiles may be dependent on the specific compiler
178  revision used to generate them. It's inadvisable to store raw profiles for
179  long periods of time.
180
181* Tools must retain **backwards** compatibility with indexed profile formats.
182  These formats are not forwards-compatible: i.e, a tool which uses format
183  version X will not be able to understand format version (X+k).
184
185* There is a third format in play: the format of the coverage mappings emitted
186  into instrumented binaries. Tools must retain **backwards** compatibility
187  with these formats. These formats are not forwards-compatible.
188