[libFuzzer] Refactor GetNextInstructionPc/GetPreviousInstructionPc

Port the change to compiler-rt/lib/fuzzer/FuzzerTracePC.cpp .
Update RISCV to use PC-2: this is coarse (C extension may be disabled

[libFuzzer] Refactor GetNextInstructionPc/GetPreviousInstructionPc

Port the change to compiler-rt/lib/fuzzer/FuzzerTracePC.cpp .
Update RISCV to use PC-2: this is coarse (C extension may be disabled) but
sufficient for pure symbolization purpose.

The commit is separate from D120362 so that bisecting/reverting is easier.

show more ...

[libFuzzer] Disable Msan on InternalStrnlen

It's called from ATTRIBUTE_NO_SANITIZE_MEMORY code.
It worked as expected if inlined and complained otherwise.

Reviewed By: eugenis

Differential Revisio

[libFuzzer] Disable Msan on InternalStrnlen

It's called from ATTRIBUTE_NO_SANITIZE_MEMORY code.
It worked as expected if inlined and complained otherwise.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D113323

show more ...

[compiler-rt] NFC: Fix trivial typo

Reviewed By: xgupta

Differential Revision: https://reviews.llvm.org/D77457

[libFuzzer] replace Vector/Set with std::vector/std::set. The custom names are not required any more since we now build with a private version of libc++. Fix some of the 81+ character lines. Mechanic

[libFuzzer] replace Vector/Set with std::vector/std::set. The custom names are not required any more since we now build with a private version of libc++. Fix some of the 81+ character lines. Mechanical change, NFC expected.

[libFuzzer] replace Vector/Set with std::vector/std::set.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D107374

show more ...

Revert "Refactor mutation strategies into a standalone library"

This reverts commit 361f742f168de0f0f256802a329c19d081615d0d.

Refactor mutation strategies into a standalone library

This change introduces libMutagen/libclang_rt.mutagen.a as a subset of libFuzzer/libclang_rt.fuzzer.a. This library contains only the fuzzing s

Refactor mutation strategies into a standalone library

This change introduces libMutagen/libclang_rt.mutagen.a as a subset of libFuzzer/libclang_rt.fuzzer.a. This library contains only the fuzzing strategies used by libFuzzer to produce new test inputs from provided inputs, dictionaries, and SanitizerCoverage feedback.

Most of this change is simply moving sections of code to one side or the other of the library boundary. The only meaningful new code is:

* The Mutagen.h interface and its implementation in Mutagen.cpp.
* The following methods in MutagenDispatcher.cpp:
* UseCmp
* UseMemmem
* SetCustomMutator
* SetCustomCrossOver
* LateInitialize (similar to the MutationDispatcher's original constructor)
* Mutate_AddWordFromTORC (uses callbacks instead of accessing TPC directly)
* StartMutationSequence
* MutationSequence
* DictionaryEntrySequence
* RecommendDictionary
* RecommendDictionaryEntry
* FuzzerMutate.cpp (which now justs sets callbacks and handles printing)
* MutagenUnittest.cpp (which adds tests of Mutagen.h)

A note on performance: This change was tested with a 100 passes of test/fuzzer/LargeTest.cpp with 1000 runs per pass, both with and without the change. The running time distribution was qualitatively similar both with and without the change, and the average difference was within 30 microseconds (2.240 ms/run vs 2.212 ms/run, respectively). Both times were much higher than observed with the fully optimized system clang (~0.38 ms/run), most likely due to the combination of CMake "dev mode" settings (e.g. CMAKE_BUILD_TYPE="Debug", LLVM_ENABLE_LTO=OFF, etc.). The difference between the two versions built similarly seems to be "in the noise" and suggests no meaningful performance degradation.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D102447

show more ...

Revert "Refactor mutation strategies into a standalone library"

This reverts commit c4a41cd77c15c2905ac74beeec09f8343a65a549 due to
buildbot failure.

Refactor mutation strategies into a standalone library

This change introduces libMutagen/libclang_rt.mutagen.a as a subset of libFuzzer/libclang_rt.fuzzer.a. This library contains only the fuzzing s

Refactor mutation strategies into a standalone library

This change introduces libMutagen/libclang_rt.mutagen.a as a subset of libFuzzer/libclang_rt.fuzzer.a. This library contains only the fuzzing strategies used by libFuzzer to produce new test inputs from provided inputs, dictionaries, and SanitizerCoverage feedback.

Most of this change is simply moving sections of code to one side or the other of the library boundary. The only meaningful new code is:

* The Mutagen.h interface and its implementation in Mutagen.cpp.
* The following methods in MutagenDispatcher.cpp:
* UseCmp
* UseMemmem
* SetCustomMutator
* SetCustomCrossOver
* LateInitialize (similar to the MutationDispatcher's original constructor)
* Mutate_AddWordFromTORC (uses callbacks instead of accessing TPC directly)
* StartMutationSequence
* MutationSequence
* DictionaryEntrySequence
* RecommendDictionary
* RecommendDictionaryEntry
* FuzzerMutate.cpp (which now justs sets callbacks and handles printing)
* MutagenUnittest.cpp (which adds tests of Mutagen.h)

A note on performance: This change was tested with a 100 passes of test/fuzzer/LargeTest.cpp with 1000 runs per pass, both with and without the change. The running time distribution was qualitatively similar both with and without the change, and the average difference was within 30 microseconds (2.240 ms/run vs 2.212 ms/run, respectively). Both times were much higher than observed with the fully optimized system clang (~0.38 ms/run), most likely due to the combination of CMake "dev mode" settings (e.g. CMAKE_BUILD_TYPE="Debug", LLVM_ENABLE_LTO=OFF, etc.). The difference between the two versions built similarly seems to be "in the noise" and suggests no meaningful performance degradation.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D102447

show more ...

[crt][fuzzer] Fix up various numeric conversions

Attempting to build a standalone libFuzzer in Fuchsia's default toolchain for the purpose of cross-compiling the unit tests revealed a number of not

[crt][fuzzer] Fix up various numeric conversions

Attempting to build a standalone libFuzzer in Fuchsia's default toolchain for the purpose of cross-compiling the unit tests revealed a number of not-quite-proper type conversions. Fuchsia's toolchain include `-std=c++17` and `-Werror`, among others, leading to many errors like `-Wshorten-64-to-32`, `-Wimplicit-float-conversion`, etc.

Most of these have been addressed by simply making the conversion explicit with a `static_cast`. These typically fell into one of two categories: 1) conversions between types where high precision isn't critical, e.g. the "energy" calculations for `InputInfo`, and 2) conversions where the values will never reach the bits being truncated, e.g. `DftTimeInSeconds` is not going to exceed 136 years.

The major exception to this is the number of features: there are several places that treat features as `size_t`, and others as `uint32_t`. This change makes the decision to cap the features at 32 bits. The maximum value of a feature as produced by `TracePC::CollectFeatures` is roughly:
(NumPCsInPCTables + ValueBitMap::kMapSizeInBits + ExtraCountersBegin() - ExtraCountersEnd() + log2(SIZE_MAX)) * 8

It's conceivable for extremely large targets and/or extra counters that this limit could be reached. This shouldn't break fuzzing, but it will cause certain features to collide and lower the fuzzers overall precision. To address this, this change adds a warning to TracePC::PrintModuleInfo about excessive feature size if it is detected, and recommends refactoring the fuzzer into several smaller ones.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D97992

show more ...

[libFuzzer] Added -print_full_coverage flag.

-print_full_coverage=1 produces a detailed branch coverage dump when run on a single file.
Uses same infrastructure as -print_coverage flag, but prints a

[libFuzzer] Added -print_full_coverage flag.

-print_full_coverage=1 produces a detailed branch coverage dump when run on a single file.
Uses same infrastructure as -print_coverage flag, but prints all branches (regardless of coverage status) in an easy-to-parse format.
Usage: For internal use with machine learning fuzzing models which require detailed coverage information on seed files to generate mutations.

Differential Revision: https://reviews.llvm.org/D85928

show more ...

[libFuzzer] Separate platform related macros out from FuzzerDefs.h into FuzzerPlatform.h, and adjust includes in other files.

Summary: This patch separates platform related macros in lib/fuzzer/Fuzz

[libFuzzer] Separate platform related macros out from FuzzerDefs.h into FuzzerPlatform.h, and adjust includes in other files.

Summary: This patch separates platform related macros in lib/fuzzer/FuzzerDefs.h into lib/fuzzer/FuzzerPlatform.h, and use FuzzerPlatform.h where necessary. This separation helps when compiling libFuzzer's interceptor module (under review); an unnecessary include of standard headers (such as string.h) may produce conflicts/ambiguation with the interceptor's declarations/definitions of library functions, which complicates interceptor implementation.

Reviewers: morehouse, hctim

Reviewed By: morehouse

Subscribers: krytarowski, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D83805

show more ...

[libFuzzer] Make dataflow and focus functions more user friendly.

Summary:
- Fail loudly if SetFocusFunction failed when it should not. For more info see
- https://github.com/google/oss-fuzz/issue

[libFuzzer] Make dataflow and focus functions more user friendly.

Summary:
- Fail loudly if SetFocusFunction failed when it should not. For more info see
- https://github.com/google/oss-fuzz/issues/3311
- https://github.com/google/sanitizers/issues/1190
- Fail loudly if CollectDataFlow is called without seed corpus.

Reviewers: kcc, metzman

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D73813

show more ...

[libFuzzer] Remove lazy counters.

Summary: Lazy counters haven't improved performance for large fuzz targets.

Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits

Tags: #llvm

Differential

[libFuzzer] Remove lazy counters.

Summary: Lazy counters haven't improved performance for large fuzz targets.

Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D67476

llvm-svn: 373403

show more ...

compiler-rt: Rename .cc file in lib/sanitizer_common to .cpp

See https://reviews.llvm.org/D58620 for discussion, and for the commands
I ran. In addition I also ran

for f in $(svn diff | diffstat

compiler-rt: Rename .cc file in lib/sanitizer_common to .cpp

See https://reviews.llvm.org/D58620 for discussion, and for the commands
I ran. In addition I also ran

for f in $(svn diff | diffstat | grep .cc | cut -f 2 -d ' '); do rg $f . ; done

and manually updated (many) references to renamed files found by that.

llvm-svn: 367463

show more ...

[libFuzzer] perform more agressive value profiling in memcmp

llvm-svn: 360385

[libFuzzer] print new functions as they are discovered in the fork mode

llvm-svn: 354092

[libFuzzer] when doing the merge, keep track of the coveraged edges, not just features

llvm-svn: 354076

[libFuzzer] simplify the code for print_coverage=1 so that it doesn't fail on broken debug info

llvm-svn: 353781

[fuzzer] Use RawPrint instead of Printf for instrumentation warning

Summary:
Use RawPrint instead of Printf for instrumentation warning because
Printf doesn't work on Win when instrumentation is bei

[fuzzer] Use RawPrint instead of Printf for instrumentation warning

Summary:
Use RawPrint instead of Printf for instrumentation warning because
Printf doesn't work on Win when instrumentation is being
initialized (since OutputFile is not yet initialized).

Reviewers: kcc

Reviewed By: kcc

Differential Revision: https://reviews.llvm.org/D57531

llvm-svn: 352789

show more ...

[libFuzzer] experimental performance optimization -lazy_counters, off by default. Posix-only for now, tested on Linux

llvm-svn: 352700

[libFuzzer] refactor the handling of instrumentation counters so that they are grouped in regions one full page each. Needed for future optimization. NFC

llvm-svn: 352603

[libFuzzer] remove stale code

llvm-svn: 352571

[libFuzzer] revert an accidental commit

llvm-svn: 352567

[libFuzzer] remove deprecated support for -fsanitize-coverage=trace-pc[-guard]

llvm-svn: 352566

[libFuzzer] remove deprecated support for -fsanitize-coverage=trace-pc[-guard]

llvm-svn: 352564