[InstrProf] Add single byte coverage mode

Use the llvm flag `-pgo-function-entry-coverage` to create single byte "counters" to track functions coverage. This mode has significantly less size overhea

[InstrProf] Add single byte coverage mode

Use the llvm flag `-pgo-function-entry-coverage` to create single byte "counters" to track functions coverage. This mode has significantly less size overhead in both code and data because
* We mark a function as "covered" with a store instead of an increment which generally requires fewer assembly instructions
* We use a single byte per function rather than 8 bytes per block

The trade off of course is that this mode only tells you if a function has been covered. This is useful, for example, to detect dead code.

When combined with debug info correlation [0] we are able to create an instrumented Clang binary that is only 150M (the vanilla Clang binary is 143M). That is an overhead of 7M (4.9%) compared to the default instrumentation (without value profiling) which has an overhead of 31M (21.7%).

[0] https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4

Reviewed By: kyulee

Differential Revision: https://reviews.llvm.org/D116180

show more ...

[InstrProf][NFC] Do not assume size of counter type

Existing code tended to assume that counters had type `uint64_t` and
computed size from the number of counters. Fix this code to directly
compute

[InstrProf][NFC] Do not assume size of counter type

Existing code tended to assume that counters had type `uint64_t` and
computed size from the number of counters. Fix this code to directly
compute the counters size in number of bytes where possible. When the
number of counters is needed, use `__llvm_profile_counter_entry_size()`
or `getCounterTypeSize()`. In a later diff these functions will depend
on the profile mode.

Change the meaning of `DataSize` and `CountersSize` to make them more clear.
* `DataSize` (`CountersSize`) - the size of the data (counter) section in bytes.
* `NumData` (`NumCounters`) - the number of data (counter) entries.

Reviewed By: kyulee

Differential Revision: https://reviews.llvm.org/D116179

show more ...

[profile] Fix profile merging with binary IDs

This fixes support for merging profiles which broke as a consequence
of e50a38840dc3db5813f74b1cd2e10e6d984d0e67. The issue was missing
adjustment in me

[profile] Fix profile merging with binary IDs

This fixes support for merging profiles which broke as a consequence
of e50a38840dc3db5813f74b1cd2e10e6d984d0e67. The issue was missing
adjustment in merge logic to account for the binary IDs which are
now included in the raw profile just after header.

In addition, this change also:
* Includes the version in module signature that's used for merging
to avoid accidental attempts to merge incompatible profiles.
* Moves the binary IDs size field after version field in the header
as was suggested in the review.

Differential Revision: https://reviews.llvm.org/D107143

show more ...

Revert "[profile] Fix profile merging with binary IDs"

This reverts commit dcadd64986b8a84dc244d4e7faa848fb4c18cea6.

[profile] Fix profile merging with binary IDs

This fixes support for merging profiles which broke as a consequence
of e50a38840dc3db5813f74b1cd2e10e6d984d0e67. The issue was missing
adjustment in me

[profile] Fix profile merging with binary IDs

This fixes support for merging profiles which broke as a consequence
of e50a38840dc3db5813f74b1cd2e10e6d984d0e67. The issue was missing
adjustment in merge logic to account for the binary IDs which are
now included in the raw profile just after header.

In addition, this change also:
* Includes the version in module signature that's used for merging
to avoid accidental attempts to merge incompatible profiles.
* Moves the binary IDs size field after version field in the header
as was suggested in the review.

Differential Revision: https://reviews.llvm.org/D107143

show more ...

Revert "[profile] Fix profile merging with binary IDs"

This reverts commit 89d6eb6f8c5d94093f30a5f37b193a2422491642, this
seemed to have break a few builders.

[profile] Fix profile merging with binary IDs

This fixes support for merging profiles which broke as a consequence
of e50a38840dc3db5813f74b1cd2e10e6d984d0e67. The issue was missing
adjustment in me

[profile] Fix profile merging with binary IDs

This fixes support for merging profiles which broke as a consequence
of e50a38840dc3db5813f74b1cd2e10e6d984d0e67. The issue was missing
adjustment in merge logic to account for the binary IDs which are
now included in the raw profile just after header.

In addition, this change also:
* Includes the version in module signature that's used for merging
to avoid accidental attempts to merge incompatible profiles.
* Moves the binary IDs size field after version field in the header
as was suggested in the review.

Differential Revision: https://reviews.llvm.org/D107143

show more ...

[InstrProfiling] Use weak alias for bias variable

We need the compiler generated variable to override the weak symbol of
the same name inside the profile runtime, but using LinkOnceODRLinkage
result

[InstrProfiling] Use weak alias for bias variable

We need the compiler generated variable to override the weak symbol of
the same name inside the profile runtime, but using LinkOnceODRLinkage
results in weak symbol being emitted in which case the symbol selected
by the linker is going to depend on the order of inputs which can be
fragile.

This change replaces the use of weak definition inside the runtime with
a weak alias. We place the compiler generated symbol inside a COMDAT
group so dead definition can be garbage collected by the linker.

We also disable the use of runtime counter relocation on Darwin since
Mach-O doesn't support weak external references, but Darwin already uses
a different continous mode that relies on overmapping so runtime counter
relocation isn't needed there.

Differential Revision: https://reviews.llvm.org/D105176

show more ...

[profile] Remove dependence on getpagesize from InstrProfilingBuffer.c.o

InstrProfilingBuffer.c.o is generic code that must support compilation
into freestanding projects. This gets rid of its depen

[profile] Remove dependence on getpagesize from InstrProfilingBuffer.c.o

InstrProfilingBuffer.c.o is generic code that must support compilation
into freestanding projects. This gets rid of its dependence on the
_getpagesize symbol from libc, shifting it to InstrProfilingFile.c.o.

This fixes a build failure seen in a firmware project.

rdar://66249701

show more ...

[profile] Add InstrProfilingInternal.c.o to Darwin kext builtins

Fixes a build failure in the Darwin kernel. Tested with:

% nm -m ./lib/libclang_rt.cc_kext_x86_64_osx.a | grep lprofSetProfileDumped

[profile] Add InstrProfilingInternal.c.o to Darwin kext builtins

Fixes a build failure in the Darwin kernel. Tested with:

% nm -m ./lib/libclang_rt.cc_kext_x86_64_osx.a | grep lprofSetProfileDumped

rdar://66249602

show more ...

[profile] Make atexit hook a no-op on Fuchsia

On Fuchsia, we always use the continuous mode with runtime counter
relocation, so there's no need for atexit hook or support for dumping
the profile man

[profile] Make atexit hook a no-op on Fuchsia

On Fuchsia, we always use the continuous mode with runtime counter
relocation, so there's no need for atexit hook or support for dumping
the profile manually.

Differential Revision: https://reviews.llvm.org/D76556

show more ...

[profile] Support counter relocation at runtime

This is an alternative to the continous mode that was implemented in
D68351. This mode relies on padding and the ability to mmap a file over
the exist

[profile] Support counter relocation at runtime

This is an alternative to the continous mode that was implemented in
D68351. This mode relies on padding and the ability to mmap a file over
the existing mapping which is generally only available on POSIX systems
and isn't suitable for other platforms.

This change instead introduces the ability to relocate counters at
runtime using a level of indirection. On every counter access, we add a
bias to the counter address. This bias is stored in a symbol that's
provided by the profile runtime and is initially set to zero, meaning no
relocation. The runtime can mmap the profile into memory at abitrary
location, and set bias to the offset between the original and the new
counter location, at which point every subsequent counter access will be
to the new location, which allows updating profile directly akin to the
continous mode.

The advantage of this implementation is that doesn't require any special
OS support. The disadvantage is the extra overhead due to additional
instructions required for each counter access (overhead both in terms of
binary size and performance) plus duplication of counters (i.e. one copy
in the binary itself and another copy that's mmapped).

Differential Revision: https://reviews.llvm.org/D69740

show more ...

[profile] Support online merging with continuous sync mode

Make it possible to use online profile merging ("%m" mode) with
continuous sync ("%c" mode).

To implement this, the merged profile is lock

[profile] Support online merging with continuous sync mode

Make it possible to use online profile merging ("%m" mode) with
continuous sync ("%c" mode).

To implement this, the merged profile is locked in the runtime
initialization step and either a) filled out for the first time or b)
checked for compatibility. Then, the profile can simply be mmap()'d with
MAP_SHARED set. With the mmap() in place, counter updates from every
process which uses an image are mapped onto the same set of physical
pages assigned by the filesystem cache. After the mmap() is set up, the
profile is unlocked.

Differential Revision: https://reviews.llvm.org/D69586

show more ...

[profile] Add a mode to continuously sync counter updates to a file

Add support for continuously syncing profile counter updates to a file.

The motivation for this is that programs do not always ex

[profile] Add a mode to continuously sync counter updates to a file

Add support for continuously syncing profile counter updates to a file.

The motivation for this is that programs do not always exit cleanly. On
iOS, for example, programs are usually killed via a signal from the OS.
Running atexit() handlers after catching a signal is unreliable, so some
method for progressively writing out profile data is necessary.

The approach taken here is to mmap() the `__llvm_prf_cnts` section onto
a raw profile. To do this, the linker must page-align the counter and
data sections, and the runtime must ensure that counters are mapped to a
page-aligned offset within a raw profile.

Continuous mode is (for the moment) incompatible with the online merging
mode. This limitation is lifted in https://reviews.llvm.org/D69586.

Continuous mode is also (for the moment) incompatible with value
profiling, as I'm not sure whether there is interest in this and the
implementation may be tricky.

As I have not been able to test extensively on non-Darwin platforms,
only Darwin support is included for the moment. However, continuous mode
may "just work" without modification on Linux and some UNIX-likes. AIUI
the default value for the GNU linker's `--section-alignment` flag is set
to the page size on many systems. This appears to be true for LLD as
well, as its `no_nmagic` option is on by default. Continuous mode will
not "just work" on Fuchsia or Windows, as it's not possible to mmap() a
section on these platforms. There is a proposal to add a layer of
indirection to the profile instrumentation to support these platforms.

rdar://54210980

Differential Revision: https://reviews.llvm.org/D68351

show more ...

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the ne

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636

show more ...

[PGO] Reduce IO in profile dumping with merging

Differential Revision: http://reviews.llvm.org/D34709

llvm-svn: 306561

[PGO] Refactor file/buffer writer callback interfaces /NFC

Introduces a 'owner' struct to include the overridable write
method and the write context in C.

This allows easy introdution of new member

[PGO] Refactor file/buffer writer callback interfaces /NFC

Introduces a 'owner' struct to include the overridable write
method and the write context in C.

This allows easy introdution of new member API to help reduce
profile merge time in the follow up patch.

llvm-svn: 306432

show more ...

Reapply r268840: [profile] Simplify value profile writing

Revert r268864 that reverted 268840 after underlying problem
is fixed for arm bot.

llvm-svn: 268992

Revert "[profile] Simplify value profile writing"

This reverts commit r268840, as it breaks Thumb2 self-hosting. There is something
unstable in the profiling for Thumb2 that needs to be sorted out b

Revert "[profile] Simplify value profile writing"

This reverts commit r268840, as it breaks Thumb2 self-hosting. There is something
unstable in the profiling for Thumb2 that needs to be sorted out before we continue
implementing these changes to the profiler. See PR27667.

llvm-svn: 268864

show more ...

[profile] Simplify value profile writing

With this patch, value data are longer pre-collected
before writing. The code is simplified and requires
less heap space for dumping.

llvm-svn: 268840

[PGO] internal API name cleanups (for better consistency)

llvm-svn: 262788

[profile] Compute number of data entries correctly

Compiler-rt miscalculates the number of entries in the __llvm_prf_data section
on i386 Darwin. This results in a number of test failures (which we

[profile] Compute number of data entries correctly

Compiler-rt miscalculates the number of entries in the __llvm_prf_data section
on i386 Darwin. This results in a number of test failures (which we started
catching after r261344).

The fix we attempted earlier is insufficient (r261683). It caused some tests to
start passing again, but that hid the fact that we drop some data entries.

This patch should fix the real problem. It fixes the way we compute DataSize by
taking into account the way the Darwin linker lays out __llvm_prf_data.

Differential Revision: http://reviews.llvm.org/D17623

llvm-svn: 261957

show more ...

[PGO] Move buffer write callback to a common file

This is a NFC refactoring enabling code sharing by file writer.

llvm-svn: 256264

[PGO] cleanup: unify prefix for portability macros

llvm-svn: 255748

Fix a bug introduced in cleanup

llvm-svn: 253847