Convert for_each to range-based for loops (NFC)

Rename parallelForEachN to just parallelFor

Patch created by running:

rg -l parallelForEachN | xargs sed -i '' -c 's/parallelForEachN/parallelFor/'

No behavior change.

Differential Revision: ht

Rename parallelForEachN to just parallelFor

Patch created by running:

rg -l parallelForEachN | xargs sed -i '' -c 's/parallelForEachN/parallelFor/'

No behavior change.

Differential Revision: https://reviews.llvm.org/D128140

show more ...

Force GHashCell to be 8-byte-aligned.

Otherwise, with recent versions of libstdc++, clang can't tell that the
atomic operations are properly aligned, and generates calls to
libatomic. (Actually, be

Force GHashCell to be 8-byte-aligned.

Otherwise, with recent versions of libstdc++, clang can't tell that the
atomic operations are properly aligned, and generates calls to
libatomic. (Actually, because of the use of reinterpret_cast, it wasn't
guaranteed to be aligned, but I think it ended up being aligned in
practice.)

Fixes https://github.com/llvm/llvm-project/issues/54790 , the part where
LLVM failed to build.

Differential Revision: https://reviews.llvm.org/D123872

show more ...

[LLD][COFF] Fix TypeServerSource matcher with more than one collision

Follow-up from 98bc304e9faded44f1d8988ffa4c5d8b50c759ec - while that
commit fixed when you had two PDBs colliding on the same Gu

[LLD][COFF] Fix TypeServerSource matcher with more than one collision

Follow-up from 98bc304e9faded44f1d8988ffa4c5d8b50c759ec - while that
commit fixed when you had two PDBs colliding on the same Guid it didn't
fix the case where you had more than two PDBs using the same Guid.

This commit fixes that and also tests much more carefully that all
the types are correct no matter the order.

Reviewed By: aganea, saudi

Differential Revision: https://reviews.llvm.org/D123185

show more ...

[lld][COFF] Fix TypeServerSource lookup on GUID collisions

Microsoft shipped a bunch of PDB files with broken/invalid GUIDs
which lead lld to use 0xFF as the key for these files in an internal
cache

[lld][COFF] Fix TypeServerSource lookup on GUID collisions

Microsoft shipped a bunch of PDB files with broken/invalid GUIDs
which lead lld to use 0xFF as the key for these files in an internal
cache. When multiple files have this key it will lead to collisions
and confused symbol lookup.

Several approaches to fix this was considered. Including making the key
the path to the PDB file, but this requires some filesystem operations
in order to normalize the file path.

Since this only happens with malformatted PDB files and we haven't
seen this before they malformatted files where shipped with visual
studio we probably shouldn't optimize for this use-case.

Instead we now just don't insert files with Guid == 0xFF into the
cache map and warn if we get collisions so similar problems can be
found in the future instead of being silent.

Discussion about the root issue and the approach to this fix can be found on Github: https://github.com/llvm/llvm-project/issues/54487

Reviewed By: aganea

Differential Revision: https://reviews.llvm.org/D122372

show more ...

lld: const-qualify iterations through VarStreamArray, NFC

No functionality change here; just unblocking a patch to LLVM.

Reland "[LLD] Remove global state in lld/COFF" after fixing asan and msan test failures

Original commit description:

[LLD] Remove global state in lld/COFF

This patch removes globals from the l

Reland "[LLD] Remove global state in lld/COFF" after fixing asan and msan test failures

Original commit description:

[LLD] Remove global state in lld/COFF

This patch removes globals from the lldCOFF library, by moving globals
into a context class (COFFLinkingContext) and passing it around wherever
it's needed.

See https://lists.llvm.org/pipermail/llvm-dev/2021-June/151184.html for
context about removing globals from LLD.

I also haven't moved the `driver` or `config` variables yet.

Differential Revision: https://reviews.llvm.org/D109634

This reverts commit a2fd05ada9030eab2258fff25e77a05adccae128.

Original commits were b4fa71eed34d967195514fe9b0a5211fca2bc5bc
and e03c7e367adb8f228332e3c2ef8f45484597b719.

show more ...

Temporarily revert "[LLD] Remove global state in lld/COFF" and "[lld] Add test to
check for timer output"

Seems to be causing a number of asan test failures.

This reverts commit b4fa71eed34d9671955

Temporarily revert "[LLD] Remove global state in lld/COFF" and "[lld] Add test to
check for timer output"

Seems to be causing a number of asan test failures.

This reverts commit b4fa71eed34d967195514fe9b0a5211fca2bc5bc
and e03c7e367adb8f228332e3c2ef8f45484597b719.

show more ...

[LLD] Remove global state in lld/COFF

This patch removes globals from the lldCOFF library, by moving globals
into a context class (COFFLinkingContext) and passing it around wherever
it's needed.

Se

[LLD] Remove global state in lld/COFF

This patch removes globals from the lldCOFF library, by moving globals
into a context class (COFFLinkingContext) and passing it around wherever
it's needed.

See https://lists.llvm.org/pipermail/llvm-dev/2021-June/151184.html for
context about removing globals from LLD.

I also haven't moved the `driver` or `config` variables yet.

Differential Revision: https://reviews.llvm.org/D109634

show more ...

[lld] Rename StringRef _lower() method calls to _insensitive()

[LLD][COFF] Reduce the maximum size of the GHASH table

Before this patch, the maximum size of the GHASH table was 2^31 buckets. However we were storing the bucket index into a TypeIndex which has an

[LLD][COFF] Reduce the maximum size of the GHASH table

Before this patch, the maximum size of the GHASH table was 2^31 buckets. However we were storing the bucket index into a TypeIndex which has an input limit of (2^31)-4095 indices, see this link. Any value above that limit will improperly set the TypeIndex's high bit, which is interpreted as DecoratedItemIdMask. This used to cause bad indices on extraction when calling TypeIndex::toArrayIndex().

Differential Revision: https://reviews.llvm.org/D103297

show more ...

[PDB] Fix ubsan complaint about memcpy from null pointer

[PDB] Check the type server guid when ghashing

Previously we simply didn't check this. Prereq to make the test suite
pass with ghash enabled by default.

Differential Revision: https://reviews.llvm.

[PDB] Check the type server guid when ghashing

Previously we simply didn't check this. Prereq to make the test suite
pass with ghash enabled by default.

Differential Revision: https://reviews.llvm.org/D102885

show more ...

[LLD][COFF] Avoid std::vector resizes during type merging

Consistently saves approx. 0.6 sec (out of 18 sec) on a large output (400 MB EXE, 2 GB PDB).

Differential Revision: https://reviews.llvm.or

[LLD][COFF] Avoid std::vector resizes during type merging

Consistently saves approx. 0.6 sec (out of 18 sec) on a large output (400 MB EXE, 2 GB PDB).

Differential Revision: https://reviews.llvm.org/D94555

show more ...

[LLD][COFF] Simplify function. NFC.

[LLD][COFF] When using PCH.OBJ, ensure func_id records indices are remapped under /DEBUG:GHASH

Before this patch, when using LLD with /DEBUG:GHASH and MSVC precomp.OBJ files, we had a bunch of:

lld

[LLD][COFF] When using PCH.OBJ, ensure func_id records indices are remapped under /DEBUG:GHASH

Before this patch, when using LLD with /DEBUG:GHASH and MSVC precomp.OBJ files, we had a bunch of:

lld-link: warning: S_[GL]PROC32ID record in blabla.obj refers to PDB item index 0x206ED1 which is not a LF[M]FUNC_ID record

This was caused by LF_FUNC_ID and LF_MFUNC_ID which didn't have correct mapping to the corresponding TPI records. The root issue was that the indexMapStorage was improperly re-assembled in UsePrecompSource::remapTpiWithGHashes.

After this patch, /DEBUG and /DEBUG:GHASH produce exactly the same debug infos in the PDB.

Differential Revision: https://reviews.llvm.org/D93732

show more ...

fix typo to cycle bots

fix typos to cycle bots

[LLD] [COFF] Fix a ubsan error in pdb-type-server-missing.yaml

This error has been present since 5519e4da83d1abc666203.

Differential Revision: https://reviews.llvm.org/D89027

[LLD][COFF] Add more type record information to /summary

This adds the following two new lines to /summary:

21351 Input OBJ files (expanded from all cmd-line inputs)
61 PDB type serv

[LLD][COFF] Add more type record information to /summary

This adds the following two new lines to /summary:

21351 Input OBJ files (expanded from all cmd-line inputs)
61 PDB type server dependencies
38 Precomp OBJ dependencies
1420669231 Input type records <<<<
78665073382 Input type records bytes <<<<
8801393 Merged TPI records
3177158 Merged IPI records
59194 Output PDB strings
71576766 Global symbol records
25416935 Module symbol records
2103431 Public symbol records

Differential Revision: https://reviews.llvm.org/D88703

show more ...

[LLD][COFF] Fix crash with /summary and PCH input files

Before this patch /summary was crashing with some .PCH.OBJ files, because tpiMap[srcIdx++] was reading at the wrong location. When the TpiSour

[LLD][COFF] Fix crash with /summary and PCH input files

Before this patch /summary was crashing with some .PCH.OBJ files, because tpiMap[srcIdx++] was reading at the wrong location. When the TpiSource depends on a .PCH.OBJ file, the types should be offset by the previously merged PCH.OBJ set of indices.

Differential Revision: https://reviews.llvm.org/D88678

show more ...

[PDB] Use one func id DenseMap instead of per-source maps, NFC

This avoids some DenseMap copies when /Zi is in use, and results in
fewer data structures.

Differential Revision: https://reviews.llvm

[PDB] Use one func id DenseMap instead of per-source maps, NFC

This avoids some DenseMap copies when /Zi is in use, and results in
fewer data structures.

Differential Revision: https://reviews.llvm.org/D88617

show more ...

Re-land "[PDB] Merge types in parallel when using ghashing"

Stored Error objects have to be checked, even if they are success
values.

This reverts commit 8d250ac3cd48d0f17f9314685a85e77895c05351.
R

Re-land "[PDB] Merge types in parallel when using ghashing"

Stored Error objects have to be checked, even if they are success
values.

This reverts commit 8d250ac3cd48d0f17f9314685a85e77895c05351.
Relands commit 49b3459930655d879b2dc190ff8fe11c38a8be5f..

Original commit message:
-----------------------------------------

This makes type merging much faster (-24% on chrome.dll) when multiple
threads are available, but it slightly increases the time to link (+10%)
when /threads:1 is passed. With only one more thread, the new type
merging is faster (-11%). The output PDB should be identical to what it
was before this change.

To give an idea, here is the /time output placed side by side:
BEFORE | AFTER
Input File Reading: 956 ms | 968 ms
Code Layout: 258 ms | 190 ms
Commit Output File: 6 ms | 7 ms
PDB Emission (Cumulative): 6691 ms | 4253 ms
Add Objects: 4341 ms | 2927 ms
Type Merging: 2814 ms | 1269 ms -55%!
Symbol Merging: 1509 ms | 1645 ms
Publics Stream Layout: 111 ms | 112 ms
TPI Stream Layout: 764 ms | 26 ms trivial
Commit to Disk: 1322 ms | 1036 ms -300ms
----------------------------------------- --------
Total Link Time: 8416 ms 5882 ms -30% overall

The main source of the additional overhead in the single-threaded case
is the need to iterate all .debug$T sections up front to check which
type records should go in the IPI stream. See fillIsItemIndexFromDebugT.
With changes to the .debug$H section, we could pre-calculate this info
and eliminate the need to do this walk up front. That should restore
single-threaded performance back to what it was before this change.

This change will cause LLD to be much more parallel than it used to, and
for users who do multiple links in parallel, it could regress
performance. However, when the user is only doing one link, it's a huge
improvement. In the future, we can use NT worker threads to avoid
oversaturating the machine with work, but for now, this is such an
improvement for the single-link use case that I think we should land
this as is.

Algorithm
----------

Before this change, we essentially used a
DenseMap<GloballyHashedType, TypeIndex> to check if a type has already
been seen, and if it hasn't been seen, insert it now and use the next
available type index for it in the destination type stream. DenseMap
does not support concurrent insertion, and even if it did, the linker
must be deterministic: it cannot produce different PDBs by using
different numbers of threads. The output type stream must be in the same
order regardless of the order of hash table insertions.

In order to create a hash table that supports concurrent insertion, the
table cells must be small enough that they can be updated atomically.
The algorithm I used for updating the table using linear probing is
described in this paper, "Concurrent Hash Tables: Fast and General(?)!":
https://dl.acm.org/doi/10.1145/3309206

The GHashCell in this change is essentially a pair of 32-bit integer
indices: <sourceIndex, typeIndex>. The sourceIndex is the index of the
TpiSource object, and it represents an input type stream. The typeIndex
is the index of the type in the stream. Together, we have something like
a ragged 2D array of ghashes, which can be looked up as:
tpiSources[tpiSrcIndex]->ghashes[typeIndex]

By using these side tables, we can omit the key data from the hash
table, and keep the table cell small. There is a cost to this: resolving
hash table collisions requires many more loads than simply looking at
the key in the same cache line as the insertion position. However, most
supported platforms should have a 64-bit CAS operation to update the
cell atomically.

To make the result of concurrent insertion deterministic, the cell
payloads must have a priority function. Defining one is pretty
straightforward: compare the two 32-bit numbers as a combined 64-bit
number. This means that types coming from inputs earlier on the command
line have a higher priority and are more likely to appear earlier in the
final PDB type stream than types from an input appearing later on the
link line.

After table insertion, the non-empty cells in the table can be copied
out of the main table and sorted by priority to determine the ordering
of the final type index stream. At this point, item and type records
must be separated, either by sorting or by splitting into two arrays,
and I chose sorting. This is why the GHashCell must contain the isItem
bit.

Once the final PDB TPI stream ordering is known, we need to compute a
mapping from source type index to PDB type index. To avoid starting over
from scratch and looking up every type again by its ghash, we save the
insertion position of every hash table insertion during the first
insertion phase. Because the table does not support rehashing, the
insertion position is stable. Using the array of insertion positions
indexed by source type index, we can replace the source type indices in
the ghash table cells with the PDB type indices.

Once the table cells have been updated to contain PDB type indices, the
mapping for each type source can be computed in parallel. Simply iterate
the list of cell positions and replace them with the PDB type index,
since the insertion positions are no longer needed.

Once we have a source to destination type index mapping for every type
source, there are no more data dependencies. We know which type records
are "unique" (not duplicates), and what their final type indices will
be. We can do the remapping in parallel, and accumulate type sizes and
type hashes in parallel by type source.

Lastly, TPI stream layout must be done serially. Accumulate all the type
records, sizes, and hashes, and add them to the PDB.

Differential Revision: https://reviews.llvm.org/D87805

show more ...

Revert "[PDB] Merge types in parallel when using ghashing"

This reverts commit 49b3459930655d879b2dc190ff8fe11c38a8be5f.

[PDB] Merge types in parallel when using ghashing

This makes type merging much faster (-24% on chrome.dll) when multiple
threads are available, but it slightly increases the time to link (+10%)
when

[PDB] Merge types in parallel when using ghashing

This makes type merging much faster (-24% on chrome.dll) when multiple
threads are available, but it slightly increases the time to link (+10%)
when /threads:1 is passed. With only one more thread, the new type
merging is faster (-11%). The output PDB should be identical to what it
was before this change.

To give an idea, here is the /time output placed side by side:
BEFORE | AFTER
Input File Reading: 956 ms | 968 ms
Code Layout: 258 ms | 190 ms
Commit Output File: 6 ms | 7 ms
PDB Emission (Cumulative): 6691 ms | 4253 ms
Add Objects: 4341 ms | 2927 ms
Type Merging: 2814 ms | 1269 ms -55%!
Symbol Merging: 1509 ms | 1645 ms
Publics Stream Layout: 111 ms | 112 ms
TPI Stream Layout: 764 ms | 26 ms trivial
Commit to Disk: 1322 ms | 1036 ms -300ms
----------------------------------------- --------
Total Link Time: 8416 ms 5882 ms -30% overall

The main source of the additional overhead in the single-threaded case
is the need to iterate all .debug$T sections up front to check which
type records should go in the IPI stream. See fillIsItemIndexFromDebugT.
With changes to the .debug$H section, we could pre-calculate this info
and eliminate the need to do this walk up front. That should restore
single-threaded performance back to what it was before this change.

This change will cause LLD to be much more parallel than it used to, and
for users who do multiple links in parallel, it could regress
performance. However, when the user is only doing one link, it's a huge
improvement. In the future, we can use NT worker threads to avoid
oversaturating the machine with work, but for now, this is such an
improvement for the single-link use case that I think we should land
this as is.

Algorithm
----------

Before this change, we essentially used a
DenseMap<GloballyHashedType, TypeIndex> to check if a type has already
been seen, and if it hasn't been seen, insert it now and use the next
available type index for it in the destination type stream. DenseMap
does not support concurrent insertion, and even if it did, the linker
must be deterministic: it cannot produce different PDBs by using
different numbers of threads. The output type stream must be in the same
order regardless of the order of hash table insertions.

In order to create a hash table that supports concurrent insertion, the
table cells must be small enough that they can be updated atomically.
The algorithm I used for updating the table using linear probing is
described in this paper, "Concurrent Hash Tables: Fast and General(?)!":
https://dl.acm.org/doi/10.1145/3309206

The GHashCell in this change is essentially a pair of 32-bit integer
indices: <sourceIndex, typeIndex>. The sourceIndex is the index of the
TpiSource object, and it represents an input type stream. The typeIndex
is the index of the type in the stream. Together, we have something like
a ragged 2D array of ghashes, which can be looked up as:
tpiSources[tpiSrcIndex]->ghashes[typeIndex]

By using these side tables, we can omit the key data from the hash
table, and keep the table cell small. There is a cost to this: resolving
hash table collisions requires many more loads than simply looking at
the key in the same cache line as the insertion position. However, most
supported platforms should have a 64-bit CAS operation to update the
cell atomically.

To make the result of concurrent insertion deterministic, the cell
payloads must have a priority function. Defining one is pretty
straightforward: compare the two 32-bit numbers as a combined 64-bit
number. This means that types coming from inputs earlier on the command
line have a higher priority and are more likely to appear earlier in the
final PDB type stream than types from an input appearing later on the
link line.

After table insertion, the non-empty cells in the table can be copied
out of the main table and sorted by priority to determine the ordering
of the final type index stream. At this point, item and type records
must be separated, either by sorting or by splitting into two arrays,
and I chose sorting. This is why the GHashCell must contain the isItem
bit.

Once the final PDB TPI stream ordering is known, we need to compute a
mapping from source type index to PDB type index. To avoid starting over
from scratch and looking up every type again by its ghash, we save the
insertion position of every hash table insertion during the first
insertion phase. Because the table does not support rehashing, the
insertion position is stable. Using the array of insertion positions
indexed by source type index, we can replace the source type indices in
the ghash table cells with the PDB type indices.

Once the table cells have been updated to contain PDB type indices, the
mapping for each type source can be computed in parallel. Simply iterate
the list of cell positions and replace them with the PDB type index,
since the insertion positions are no longer needed.

Once we have a source to destination type index mapping for every type
source, there are no more data dependencies. We know which type records
are "unique" (not duplicates), and what their final type indices will
be. We can do the remapping in parallel, and accumulate type sizes and
type hashes in parallel by type source.

Lastly, TPI stream layout must be done serially. Accumulate all the type
records, sizes, and hashes, and add them to the PDB.

Differential Revision: https://reviews.llvm.org/D87805

show more ...