[NFC][sanitizer] Consolidate malloc hook invocations

[NFC][sanitizer] Remove unnececary HOOK macros

Reland "[memprof] Refactor out the MemInfoBlock into a macro based def."

This reverts commit 857ec0d01f8021ff0d9540fcbf6ff24e29868ba4.

Fixes -DLLVM_ENABLE_MODULES=On build by adding the new textual

Reland "[memprof] Refactor out the MemInfoBlock into a macro based def."

This reverts commit 857ec0d01f8021ff0d9540fcbf6ff24e29868ba4.

Fixes -DLLVM_ENABLE_MODULES=On build by adding the new textual
header to the modulemap file.

Reviewed in https://reviews.llvm.org/D117722

show more ...

Revert "[memprof] Refactor out the MemInfoBlock into a macro based def."

This reverts commit 9def83c6d02944b2931efd50cd2491953a772aab. [4/4]

[memprof] Refactor out the MemInfoBlock into a macro based def.

This patch refactors out the MemInfoBlock definition into a macro based
header which can be included to generate enums, structus and c

[memprof] Refactor out the MemInfoBlock into a macro based def.

This patch refactors out the MemInfoBlock definition into a macro based
header which can be included to generate enums, structus and code for
each field recorded by the memprof profiling runtime.

Differential Revision: https://reviews.llvm.org/D117722

show more ...

[memprof] Move the meminfo block struct to MemProfData.inc.

The definition of the MemInfoBlock is shared between the memprof
compiler-rt runtime and llvm/lib/ProfileData/. This change removes the
me

[memprof] Move the meminfo block struct to MemProfData.inc.

The definition of the MemInfoBlock is shared between the memprof
compiler-rt runtime and llvm/lib/ProfileData/. This change removes the
memprof_meminfoblock header and moves the struct to the shared include
file. To enable this sharing, the Print method is moved to the
memprof_allocator (the only place it is used) and the remaining uses are
updated to refer to the MemInfoBlock defined in the MemProfData.inc
file.

Also a couple of other minor changes which improve usability of the
types in MemProfData.inc.
* Update the PACKED macro to handle commas.
* Add constructors and equality operators.
* Don't initialize the buildid field.

Differential Revision: https://reviews.llvm.org/D116780

show more ...

sanitizer_common: prefix thread-safety macros with SANITIZER_

Currently we use very common names for macros like ACQUIRE/RELEASE,
which cause conflicts with system headers.
Prefix all macros with SA

sanitizer_common: prefix thread-safety macros with SANITIZER_

Currently we use very common names for macros like ACQUIRE/RELEASE,
which cause conflicts with system headers.
Prefix all macros with SANITIZER_ to avoid conflicts.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D116652

show more ...

[NFC][sanitizer] Remove SetSoftRssLimitExceededCallback

According comments on D44404, something like that was the goal.

Reviewed By: morehouse, kstoimenov

Differential Revision: https://reviews.ll

[NFC][sanitizer] Remove SetSoftRssLimitExceededCallback

According comments on D44404, something like that was the goal.

Reviewed By: morehouse, kstoimenov

Differential Revision: https://reviews.llvm.org/D114991

show more ...

[NFC][sanitizer] Rename RssLimitExceeded -> IsRssLimitExceeded

[memprof] Remove the "Live on exit:" print for text format.

We dropped the printing of live on exit blocks in rG1243cef245f6 -
the commit changed the insertOrMerge logic. Remove the message since it

[memprof] Remove the "Live on exit:" print for text format.

We dropped the printing of live on exit blocks in rG1243cef245f6 -
the commit changed the insertOrMerge logic. Remove the message since it
is no longer needed (all live blocks are inserted into the hashmap)
before serializing/printing the profile. Furthermore, the original
intent was to capture evicted blocks so it wasn't entirely correct.

Also update the binary format test invocation to remove the redundant
print_text directive now that it is the default.

Differential Revision: https://reviews.llvm.org/D114285

show more ...

[memprof] Add a raw binary format to serialize memprof profiles.

This change implements the raw binary format discussed in
https://lists.llvm.org/pipermail/llvm-dev/2021-September/153007.html

Summa

[memprof] Add a raw binary format to serialize memprof profiles.

This change implements the raw binary format discussed in
https://lists.llvm.org/pipermail/llvm-dev/2021-September/153007.html

Summary of changes
* Add a new memprof option to choose binary or text (default) format.
* Add a rawprofile library which serializes the MIB map to profile.
* Add a unit test for rawprofile.
* Mark sanitizer procmaps methods as virtual to be able to mock them.
* Extend memprof_profile_dump regression test.

Differential Revision: https://reviews.llvm.org/D113317

show more ...

[memprof] Replace the block cache with a hashmap.

The existing implementation uses a cache + eviction based scheme to
record heap profile information. This design was adopted to ensure a
constant me

[memprof] Replace the block cache with a hashmap.

The existing implementation uses a cache + eviction based scheme to
record heap profile information. This design was adopted to ensure a
constant memory overhead (due to fixed number of cache entries) along
with incremental write-to-disk for evictions. We find that since the
number to entries to track is O(unique-allocation-contexts) the overhead
of keeping all contexts in memory is not very high. On a clang workload,
the max number of unique allocation contexts was ~35K, median ~11K.
For each context, we (currently) store 64 bytes of data - this amounts
to 5.5MB (max). Given the low overheads for a complex workload, we can
simplify the implementation by using a hashmap without eviction.

Other changes:
* Memory map is dumped at the end rather than startup. The relative
order in the profile dump is unchanged since we no longer have evicted
entries at runtime.
* Added a test to check meminfoblocks are merged.

Differential Revision: https://reviews.llvm.org/D111676

show more ...

[memprof] Move the MemInfoBlock definition to a separate header.

Move the memprof MemInfoBlock struct to it's own header as requested
during the review of D111676.

Differential Revision: https://re

[memprof] Move the MemInfoBlock definition to a separate header.

Move the memprof MemInfoBlock struct to it's own header as requested
during the review of D111676.

Differential Revision: https://reviews.llvm.org/D113315

show more ...

[MemProf] Fix format warnings

Fix the warnings that show up with -Wformat in this file.

[MemProf] Avoid global lock when updating MIB cache

Previously we used a global Allocator-scope mutex to lock when adding a
deallocation to the MIB cache. This resulted in a lot of contention.
Inste

[MemProf] Avoid global lock when updating MIB cache

Previously we used a global Allocator-scope mutex to lock when adding a
deallocation to the MIB cache. This resulted in a lot of contention.
Instead add and use per-set mutexes.

Along with this, we now need to remove the global miss and access count
variables and instead utilize the per-set statistics to report the
overall miss rate.

Differential Revision: https://reviews.llvm.org/D109853

show more ...

sanitizer_common: add thread safety annotations

Enable clang Thread Safety Analysis for sanitizers:
https://clang.llvm.org/docs/ThreadSafetyAnalysis.html

Thread Safety Analysis can detect inconsist

sanitizer_common: add thread safety annotations

Enable clang Thread Safety Analysis for sanitizers:
https://clang.llvm.org/docs/ThreadSafetyAnalysis.html

Thread Safety Analysis can detect inconsistent locking,
deadlocks and data races. Without GUARDED_BY annotations
it has limited value. But this does all the heavy lifting
to enable analysis and allows to add GUARDED_BY incrementally.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D105716

show more ...

[compiler-rt santizer] Use clock_gettime instead of timespec_get

On RH66, timespec_get is not available. Use clock_gettime instead.

This problem was introduced with D87120

Reviewed By: tejohnson

[compiler-rt santizer] Use clock_gettime instead of timespec_get

On RH66, timespec_get is not available. Use clock_gettime instead.

This problem was introduced with D87120

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D91687

show more ...

[MemProf] Add interface to dump profile

Add an interface so that the profile can be dumped on demand.

Differential Revision: https://reviews.llvm.org/D91768

[CMake] Replace ctime with time.h in memprof

Part of D88922

[MemProf] Memory profiling runtime support

See RFC for background:
http://lists.llvm.org/pipermail/llvm-dev/2020-June/142744.html

Follow on companion to the clang/llvm instrumentation support in D8

[MemProf] Memory profiling runtime support

See RFC for background:
http://lists.llvm.org/pipermail/llvm-dev/2020-June/142744.html

Follow on companion to the clang/llvm instrumentation support in D85948
and committed earlier.

This patch adds the compiler-rt runtime support for the memory
profiling.

Note that much of this support was cloned from asan (and then greatly
simplified and renamed). For example the interactions with the
sanitizer_common allocators, error handling, interception, etc.

The bulk of the memory profiling specific code can be found in the
MemInfoBlock, MemInfoBlockCache, and related classes defined and used
in memprof_allocator.cpp.

For now, the memory profile is dumped to text (stderr by default, but
honors the sanitizer_common log_path flag). It is dumped in either a
default verbose format, or an optional terse format.

This patch also adds a set of tests for the core functionality.

Differential Revision: https://reviews.llvm.org/D87120

show more ...