Rename parallelForEachN to just parallelFor

Patch created by running:

rg -l parallelForEachN | xargs sed -i '' -c 's/parallelForEachN/parallelFor/'

No behavior change.

Differential Revision: ht

Rename parallelForEachN to just parallelFor

Patch created by running:

rg -l parallelForEachN | xargs sed -i '' -c 's/parallelForEachN/parallelFor/'

No behavior change.

Differential Revision: https://reviews.llvm.org/D128140

show more ...

Cleanup llvm/DebugInfo/PDB headers

accumulated preprocessed size:
before: 1065515095
after: 1065629059

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differenti

Cleanup llvm/DebugInfo/PDB headers

accumulated preprocessed size:
before: 1065515095
after: 1065629059

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D120195

show more ...

[llvm] Use = default (NFC)

[llvm] Rename StringRef _lower() method calls to _insensitive()

This is a mechanical change. This actually also renames the
similarly named methods in the SmallString class, however these
methods do

[llvm] Rename StringRef _lower() method calls to _insensitive()

This is a mechanical change. This actually also renames the
similarly named methods in the SmallString class, however these
methods don't seem to be used outside of the llvm subproject, so
this doesn't break building of the rest of the monorepo.

show more ...

[PDB] Fix unsigned integer overflow

When building with -fsanitize=unsigned-integer-overflow, this code
causes a diagnostic like:

../../llvm/lib/DebugInfo/PDB/Native/GSIStreamBuilder.cpp:159:15:

[PDB] Fix unsigned integer overflow

When building with -fsanitize=unsigned-integer-overflow, this code
causes a diagnostic like:

../../llvm/lib/DebugInfo/PDB/Native/GSIStreamBuilder.cpp:159:15:
runtime error: unsigned integer overflow:
90 - 229 cannot be represented in type 'unsigned long'

unsigned integer overflow is well defined and it isn't an issue in
practice, but in obscure scenarios (S1.size() small, S2.size over 2GB
on 32-bit systems) it could even be a bug.

So use the usual idiom for implementing cmp functions instead of the
gernally considered buggy idiom :)
See e.g. https://www.gnu.org/software/libc/manual/html_node/Comparison-Functions.html
or https://stackoverflow.com/questions/10996418/efficient-integer-compare-function/10997428#10997428

Differential Revision: https://reviews.llvm.org/D97557

show more ...

fix typos to cycle bots

[PDB] Fix out-of-bounds acces when sorting GSI buckets

When building in Debug on Windows-MSVC after b7402edce315, a lot of tests were failing because we were dereferencing an element past the end of

[PDB] Fix out-of-bounds acces when sorting GSI buckets

When building in Debug on Windows-MSVC after b7402edce315, a lot of tests were failing because we were dereferencing an element past the end of HashRecords. This happened towards the end of the table, in unused slots.

show more ...

[PDB] Defer public serialization until PDB writing

This reduces peak memory on my test case from 1960.14MB to 1700.63MB
(-260MB, -13.2%) with no measurable impact on CPU time. I'm currently
working

[PDB] Defer public serialization until PDB writing

This reduces peak memory on my test case from 1960.14MB to 1700.63MB
(-260MB, -13.2%) with no measurable impact on CPU time. I'm currently
working with a publics stream that is about 277MB. Before this change,
we would allocate 277MB of heap memory, serialize publics into them,
hold onto that heap memory, open the PDB, and commit into it. After
this change, we defer the serialization until commit time.

In the last change I made to public writing, I re-sorted the list of
publics multiple times in place to avoid allocating new temporary data
structures. Deferring serialization until later requires that we don't
reorder the publics. Instead of sorting the publics, I partially
construct the hash table data structures, store a publics index in them,
and then sort the hash table data structures. Later, I replace the index
with the symbol record offset.

This change also addresses a FIXME and moves the list of global and
public records from GSIHashStreamBuilder to GSIStreamBuilder. Now that
publics aren't being serialized, it makes even less sense to store them
as a list of CVSymbol records. The hash table used to deduplicate
globals is moved as well, since that is specific to globals, and not
publics.

Reviewed By: aganea, hans

Differential Revision: https://reviews.llvm.org/D81296

show more ...

[PDB] Switch from LLVM_PACKED to LLVM_PACKED_START/END

Reportedly using the pragma instead of the __attribute__ silences
warnings with some GCC versions.

[PDB] Optimize public symbol processing

Reduces time to link PGO instrumented net_unittets.exe by 11% (9.766s ->
8.672s, best of three). Reduces peak memory by 65.7MB (2142.71MB ->
2076.95MB).

Use

[PDB] Optimize public symbol processing

Reduces time to link PGO instrumented net_unittets.exe by 11% (9.766s ->
8.672s, best of three). Reduces peak memory by 65.7MB (2142.71MB ->
2076.95MB).

Use a more compact struct, BulkPublic, for faster sorting. Sort in
parallel. Construct the hash buckets in parallel. Try to use one vector
to hold all the publics instead of copying them from one to another.
Allocate all the memory needed to serialize publics up front, and then
serialize them in place in parallel.

Reviewed By: aganea, hans

Differential Revision: https://reviews.llvm.org/D79467

show more ...

[PDB] Move stream index tracking to GSIStreamBuilder

The GSIHashStreamBuilder doesn't need to know the stream index.
Standardize the naming (Idx -> Index in public APIs).

[PDB] Bypass generic deserialization code for publics sorting

The number of public symbols is very large, and each deserialization
does a few heap allocations. The public symbols are serialized by t

[PDB] Bypass generic deserialization code for publics sorting

The number of public symbols is very large, and each deserialization
does a few heap allocations. The public symbols are serialized by the
linker, so we can assume they have the expected layout and use it
directly.

Saves O(#publics) temporary heap allocations and shrinks some data
structures.

show more ...

[llvm] Migrate llvm::make_unique to std::make_unique

Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of

[llvm] Migrate llvm::make_unique to std::make_unique

Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.

llvm-svn: 369013

show more ...

Deduplicate S_CONSTANTs in LLD.

Summary: Deduplicate S_CONSTANTS when linking, if they have the same value.

Reviewers: rnk

Subscribers: hiraditya, llvm-commits

Tags: #llvm

Differential Revision:

Deduplicate S_CONSTANTs in LLD.

Summary: Deduplicate S_CONSTANTS when linking, if they have the same value.

Reviewers: rnk

Subscribers: hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D63151

llvm-svn: 363089

show more ...

Use llvm::stable_sort

While touching the code, simplify if feasible.

llvm-svn: 358996

[codeview] Remove Type member from CVRecord

Summary:
Now CVType and CVSymbol are effectively type-safe wrappers around
ArrayRef<uint8_t>. Make the kind() accessor load it from the
RecordPrefix, whic

[codeview] Remove Type member from CVRecord

Summary:
Now CVType and CVSymbol are effectively type-safe wrappers around
ArrayRef<uint8_t>. Make the kind() accessor load it from the
RecordPrefix, which is the same for types and symbols.

Reviewers: zturner, aganea

Subscribers: hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60018

llvm-svn: 357658

show more ...

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the ne

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636

show more ...

[PDB] Emit S_UDT records in LLD.

Previously these were dropped. We now understand them sufficiently
well to start emitting them. From the debugger's perspective, this
now enables us to have debug

[PDB] Emit S_UDT records in LLD.

Previously these were dropped. We now understand them sufficiently
well to start emitting them. From the debugger's perspective, this
now enables us to have debug info about typedefs (both global and
function-locally scoped)

Differential Revision: https://reviews.llvm.org/D55228

llvm-svn: 348306

show more ...

llvm::sort(C.begin(), C.end(), ...) -> llvm::sort(C, ...)

Summary: The convenience wrapper in STLExtras is available since rL342102.

Reviewers: dblaikie, javed.absar, JDevlieghere, andreadb

Subscr

llvm::sort(C.begin(), C.end(), ...) -> llvm::sort(C, ...)

Summary: The convenience wrapper in STLExtras is available since rL342102.

Reviewers: dblaikie, javed.absar, JDevlieghere, andreadb

Subscribers: MatzeB, sanjoy, arsenm, dschuff, mehdi_amini, sdardis, nemanjai, jvesely, nhaehnle, sbc100, jgravelle-google, eraman, aheejin, kbarton, JDevlieghere, javed.absar, gbedwell, jrtc27, mgrang, atanasyan, steven_wu, george.burgess.iv, dexonsmith, kristina, jsji, llvm-commits

Differential Revision: https://reviews.llvm.org/D52573

llvm-svn: 343163

show more ...

pdb output: Initialize padding in PublicsStreamHeader.

Makes the produced pdbs more deterministic; before they'd contain 2 arbitary
bytes where this padding was.

Also reorder initialization to matc

pdb output: Initialize padding in PublicsStreamHeader.

Makes the produced pdbs more deterministic; before they'd contain 2 arbitary
bytes where this padding was.

Also reorder initialization to match the order of the fields in the struct (nfc)

llvm-svn: 341945

show more ...

[llvm] Change 2 instances of std::sort to llvm::sort

llvm-svn: 337192

[PDB] memicmp only exists on Windows, use StringRef::compare_lower instead

llvm-svn: 336469

[PDB] One more fix for hasing GSI records.

The reference implementation uses a case-insensitive string
comparison for strings of equal length. This will cause the
string "tEo" to compare less than

[PDB] One more fix for hasing GSI records.

The reference implementation uses a case-insensitive string
comparison for strings of equal length. This will cause the
string "tEo" to compare less than "VUo". However we were using
a case sensitive comparison, which would generate the opposite
outcome. Switch to a case insensitive comparison. Also, when
one of the strings contains non-ascii characters, fallback to
a straight memcmp.

The only way to really test this is with a DIA test. Before this
patch, the test will fail (but succeed if link.exe is used instead
of lld-link). After the patch, it succeeds even with lld-link.

llvm-svn: 336464

show more ...

[PDB] Sort globals symbols by name in GSI hash buckets.

It seems like the debugger first computes a symbol's bucket,
and then does a binary search of entries in the bucket using the
symbol's name in

[PDB] Sort globals symbols by name in GSI hash buckets.

It seems like the debugger first computes a symbol's bucket,
and then does a binary search of entries in the bucket using the
symbol's name in order to find it. If the bucket entries are not
in sorted order, this obviously won't work. After this patch a
couple of simple test cases show that we generate an exactly
identical GSI hash stream, which is very nice.

llvm-svn: 336405

show more ...

[PDB] Make our PDBs look more like MS PDBs.

When investigating bugs in PDB generation, the first step is
often to do the same link with link.exe and then compare PDBs.

But comparing PDBs is hard be

[PDB] Make our PDBs look more like MS PDBs.

When investigating bugs in PDB generation, the first step is
often to do the same link with link.exe and then compare PDBs.

But comparing PDBs is hard because two completely different byte
sequences can both be correct, so it hampers the investigation when
you also have to spend time figuring out not just which bytes are
different, but also if the difference is meaningful.

This patch fixes a couple of cases related to string table emission,
hash table emission, and the order in which we emit strings that
makes more of our bytes the same as the bytes generated by MS PDBs.

Differential Revision: https://reviews.llvm.org/D44810

llvm-svn: 328348

show more ...