[BOLT] Fix concurrent hash table modification in the instrumentation runtime

`__bolt_instr_data_dump()` does not lock the hash tables when iterating
over them, so the iteration can happen concurrent

[BOLT] Fix concurrent hash table modification in the instrumentation runtime

`__bolt_instr_data_dump()` does not lock the hash tables when iterating
over them, so the iteration can happen concurrently with a modification
done in another thread, when the table is in an inconsistent state. This
also has been observed in practice, when it caused a segmentation fault.

We fix this by locking hash tables during iteration. This is done by taking
the lock in `forEachElement()`.
The only other site of iteration, `resetCounters()`, has been correctly
locking the table even before this patch. This patch removes its `Lock`
because the lock is now taken in the inner `forEachElement()`.

Reviewed By: maksfb, yota9

Differential Revision: https://reviews.llvm.org/D129089

show more ...

[BOLT] Add runtime functions required by freestanding environment

Compiler can generate calls to some functions implicitly, even under
constraints of freestanding environment. Make sure these functi

[BOLT] Add runtime functions required by freestanding environment

Compiler can generate calls to some functions implicitly, even under
constraints of freestanding environment. Make sure these functions are
available in our runtime objects.

Fixes test failures on some systems after https://reviews.llvm.org/D128960.

Reviewed By: yota9

Differential Revision: https://reviews.llvm.org/D129168

show more ...

[BOLT][NFC] Fix braces usage in the rest of the codebase

Summary:
Refactor remaining bolt sources to follow the braces rule for if/else/loop from
[LLVM Coding Standards](https://llvm.org/docs/Coding

[BOLT][NFC] Fix braces usage in the rest of the codebase

Summary:
Refactor remaining bolt sources to follow the braces rule for if/else/loop from
[LLVM Coding Standards](https://llvm.org/docs/CodingStandards.html).

(cherry picked from FBD33345885)

show more ...

[BOLT][NFC] Fix file-description comments

Summary: Fix comments at the start of source files.

(cherry picked from FBD33274597)

[PR] instr: change assert to allow FD 0 return by __open()

Summary: In some cases __open() is returning 0 for me. The open
syscall will return a negative number (-1) on error so 0 should be
valid.

[PR] instr: change assert to allow FD 0 return by __open()

Summary: In some cases __open() is returning 0 for me. The open
syscall will return a negative number (-1) on error so 0 should be
valid.

This happens when one compiles a BOLT instrumented executable of
pyston (python implementation) and afterwards pip installs a package
which needs to be compiled and sets CFLAGS=-pipe. I could not reduce
it down to a small testcase but I guess it one can trigger when
manually closing std{in, out, err}. Everything seems to work
normally when disabling the assert for 0 in getBinaryPath() - I
decided to also modify the second case in readDescriptions() even
though I did not run into that one yet.

(cherry picked from FBD32409548)

show more ...

[PR] bolt_rt: getBinaryPath() increase max file path

Summary:
Increase the hard limit from 256 to 4096.
This fixes the 'Assertion failed: failed to open binary path' error I'm seeing.

(cherry picke

[PR] bolt_rt: getBinaryPath() increase max file path

Summary:
Increase the hard limit from 256 to 4096.
This fixes the 'Assertion failed: failed to open binary path' error I'm seeing.

(cherry picked from FBD31911946)

show more ...

[PR] Disable instrumentation and hugify build for aarch64

Summary:
This patch temporarily disables instrumentation and higufy build not for
x86 platforms to be able to build llvm-bolt tool on aarch6

[PR] Disable instrumentation and hugify build for aarch64

Summary:
This patch temporarily disables instrumentation and higufy build not for
x86 platforms to be able to build llvm-bolt tool on aarch64.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

(cherry picked from FBD31738306)

show more ...

[PR] Instrumentation: Sync file on dump

Summary:
Sync the file with storage device on data dump to stabilize
instrumentation testing

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

[PR] Instrumentation: Sync file on dump

Summary:
Sync the file with storage device on data dump to stabilize
instrumentation testing

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

(cherry picked from FBD31738021)

show more ...

[PR] Instrumentation: use TryLock for SimpleHashTable getter

Summary:
This commit introduces TryLock usage for SimpleHashTable getter to
avoid deadlock and relax syscalls usage which causes signific

[PR] Instrumentation: use TryLock for SimpleHashTable getter

Summary:
This commit introduces TryLock usage for SimpleHashTable getter to
avoid deadlock and relax syscalls usage which causes significant
overhead in runtime.
The old behavior left under -conservative-instrumentation option passed
to instrumentation library.
Also, this commit includes a corresponding test case: instrumentation of
executable which performs indirect calls from common code and signal
handler.

Note: in case if TryLock was failed to acquire the lock - this indirect
call will not be accounted in the resulting profile.

Vasily Leonenko,
Advanced Software Technology Lab, Huawei

(cherry picked from FBD30821949)

show more ...

[PR] Instrumentation: Avoid generating GOT table in instrumentation library

Summary:
To avoid RELATIVE relocations avoid using of GOT table
by using hidden visibility for all symbols in library.

Vl

[PR] Instrumentation: Avoid generating GOT table in instrumentation library

Summary:
To avoid RELATIVE relocations avoid using of GOT table
by using hidden visibility for all symbols in library.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

(cherry picked from FBD30092712)

show more ...

[PR] Instrumentation: Fix start and fini trampoline pointers

Summary:
The trampolines are no loger pointers to the functions. For
propper name resolving by bolt use extern "C" for all external symb

[PR] Instrumentation: Fix start and fini trampoline pointers

Summary:
The trampolines are no loger pointers to the functions. For
propper name resolving by bolt use extern "C" for all external symbols
in instr.cpp

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

(cherry picked from FBD30092698)

show more ...

[PR] Instrumentation: Introduce instrumentation-binpath argument

Summary:
This commit introduces -instrumentation-binpath argument used
to point instuqmented binary in runtime in case if /proc/self/

[PR] Instrumentation: Introduce instrumentation-binpath argument

Summary:
This commit introduces -instrumentation-binpath argument used
to point instuqmented binary in runtime in case if /proc/self/map_files
path is not accessible due to access restriction issues.

Vasily Leonenko
Advanced Software Technology Lab, Huawei

(cherry picked from FBD30092681)

show more ...

[PR] Instrumentation: Fix runtime handlers for PIE files

Summary:
This commit fixes runtime instrumentation handlers for PIE
binaries case.

Vladislav Khmelevsky,
Advanced Software Technology Lab, H

[PR] Instrumentation: Fix runtime handlers for PIE files

Summary:
This commit fixes runtime instrumentation handlers for PIE
binaries case.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

(cherry picked from FBD30092522)

show more ...

[PR] Instrumentation: Add support for opening libs based on links /proc/self/map_files

Summary:
This commit adds support for opening libs based on links
/proc/self/map_files. For this we're getting

[PR] Instrumentation: Add support for opening libs based on links /proc/self/map_files

Summary:
This commit adds support for opening libs based on links
/proc/self/map_files. For this we're getting current virtual address
and searching the lib in the directory with such address range. After
that, we're getting full path to the binary by using readlink
function. Direct read from link in /proc/self/map_files entries is not
possible because of lack of permissions.

Elvina Yakubova,
Advanced Software Technology Lab, Huawei

(cherry picked from FBD30092422)

show more ...

[PR] Instrumentation: Generate and use _start and _fini trampolines

Summary:
This commit implements new method for _start & _fini functions hooking
which allows to use relative jumps for future PIE

[PR] Instrumentation: Generate and use _start and _fini trampolines

Summary:
This commit implements new method for _start & _fini functions hooking
which allows to use relative jumps for future PIE & .so library support.
Instead of using absolute address of _start & _fini functions known on
linking stage - we'll use dynamically created trampoline functions and
use corresponding symbols in instrumentation runtime library.

As we would like to use instrumentation for dynamically loaded binaries
(with PIE & .so), thus we need to compile instrumentation library with
"-fPIC" flag to support relative address resolution for functions and
data.

For shared libraries we need to handle initialization of instrumentation
library case by using DT_INIT section entry point.

Also this commit adds detection if the binary is executable or shared
library based on existence of PT_INTERP header. In case of shared
library we save information about real library init function address
for further usage for instrumentation library init trampoline function
creation and also update DT_INIT to point instrumentation library init
function.

Functions called from init/fini functions should be called with forced
stack alignment to avoid issues with instructions which relies on it.
E.g. optimized string operations.

Vasily Leonenko,
Advanced Software Technology Lab, Huawei

(cherry picked from FBD30092316)

show more ...

Rebase: [BOLT][NFC] Expand auto types

Summary:
Expanded auto types across BOLT semi-automatically with the aid
of clangd LSP

(cherry picked from FBD33289309)

[BOLT][PR] Instrumentation: Introduce -no-counters-clear and -wait-forks options

Summary:
This PR introduces 2 new instrumentation options:
1. instrumentation-no-counters-clear: Discussed at https:/

[BOLT][PR] Instrumentation: Introduce -no-counters-clear and -wait-forks options

Summary:
This PR introduces 2 new instrumentation options:
1. instrumentation-no-counters-clear: Discussed at https://github.com/facebookincubator/BOLT/issues/121
2. instrumentation-wait-forks: Since the instrumentation counters are mapped as MAP_SHARED it will be nice to add ability to wait until all forks of the parent process will die using tracking of process group.
The last patch is just emitBinary code refactor.
Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

Pull Request resolved: https://github.com/facebookincubator/BOLT/pull/125
GitHub Author: Vladislav Khmelevskyi <[email protected]>

(cherry picked from FBD26919011)

show more ...

Fix license for a few remaining files

Summary: As titled.

(cherry picked from FBD28112137)

[BOLT] Add support for dumping profile on MacOS

Summary: Add support for dumping profile on MacOS.

(cherry picked from FBD25751363)

[BOLT] Add support for dumping counters on MacOS

Summary: Add support for dumping counters on MacOS

(cherry picked from FBD25750516)

[BOLT] Fix operator new signature

Summary:
Use size_t for the first parameter of operator new.
https://en.cppreference.com/w/cpp/memory/new/operator_new

(cherry picked from FBD25750921)

Inject instrumentation's global dtor on MachO

Summary:
This diff is a preparation for dumping the profile generated by BOLT's instrumenation on MachO.

1/ Function "bolt_instr_fini" is placed into

Inject instrumentation's global dtor on MachO

Summary:
This diff is a preparation for dumping the profile generated by BOLT's instrumenation on MachO.

1/ Function "bolt_instr_fini" is placed into the predefined section "__fini"

2/ In the instrumentation pass we create a symbol "bolt_instr_fini" and
replace the last global destructor with it.

This is a temporary solution, in the future we need to register bolt_instr_fini in addition to the existing destructors without dropping the last one.

(cherry picked from FBD25071864)

show more ...

Link the instrumentation runtime on OSX

Summary: Link the instrumentation runtime on OSX.

(cherry picked from FBD24390019)

Add first bits to cross-compile the runtime for OSX

Summary: Add first bits to cross-compile the runtime for OSX.

(cherry picked from FBD24330977)

[BOLT] Fix stack alignment for runtime lib

Summary:
Right now, the SAVE_ALL sequence executed upon entry of both
of our runtime libs (hugify and instrumentation) will cause the stack to
not be align

[BOLT] Fix stack alignment for runtime lib

Summary:
Right now, the SAVE_ALL sequence executed upon entry of both
of our runtime libs (hugify and instrumentation) will cause the stack to
not be aligned at a 16B boundary because it saves 15 8-byte regs. Change
the code sequence to adjust for that. The compiler may generate code
that assumes the stack is aligned by using movaps instructions, which
will crash.

(cherry picked from FBD22744307)

show more ...