Re-land [LLD] Remove global state in lldCommon

Move all variables at file-scope or function-static-scope into a hosting structure (lld::CommonLinkerContext) that lives at lldMain()-scope. Drivers wi

Re-land [LLD] Remove global state in lldCommon

Move all variables at file-scope or function-static-scope into a hosting structure (lld::CommonLinkerContext) that lives at lldMain()-scope. Drivers will inherit from this structure and add their own global state, in the same way as for the existing COFFLinkerContext.

See discussion in https://lists.llvm.org/pipermail/llvm-dev/2021-June/151184.html

The previous land f860fe362282ed69b9d4503a20e5d20b9a041189 caused issues in https://lab.llvm.org/buildbot/#/builders/123/builds/8383, fixed by 22ee510dac9440a74b2e5b3fe3ff13ccdbf55af3.

Differential Revision: https://reviews.llvm.org/D108850

show more ...

Revert [LLD] Remove global state in lldCommon

It seems to be causing issues on https://lab.llvm.org/buildbot/#/builders/123/builds/8383

[LLD] Remove global state in lldCommon

Move all variables at file-scope or function-static-scope into a hosting structure (lld::CommonLinkerContext) that lives at lldMain()-scope. Drivers will inher

[LLD] Remove global state in lldCommon

Move all variables at file-scope or function-static-scope into a hosting structure (lld::CommonLinkerContext) that lives at lldMain()-scope. Drivers will inherit from this structure and add their own global state, in the same way as for the existing COFFLinkerContext.

See discussion in https://lists.llvm.org/pipermail/llvm-dev/2021-June/151184.html

Differential Revision: https://reviews.llvm.org/D108850

show more ...

[lld-link] Remove unneeded lto::InputFile::create after D116434

[lld-link] Replace LazyObjFile with lazy ObjFile/BitcodeFile

Similar to ELF 3a5fb57393c3bc77be9e7afc2ec9d4ec3c9bbf70.

* previously when a LazyObjFile was extracted, a new ObjFile/BitcodeFile was cr

[lld-link] Replace LazyObjFile with lazy ObjFile/BitcodeFile

Similar to ELF 3a5fb57393c3bc77be9e7afc2ec9d4ec3c9bbf70.

* previously when a LazyObjFile was extracted, a new ObjFile/BitcodeFile was created; now the file is reused, just with `lazy` cleared
* avoid the confusing transfer of `symbols` from LazyObjFile to the new file
* simpler code, smaller executable (5200+ bytes smaller on x86-64)
* make eager parsing feasible (for parallel section/symbol table initialization)

Reviewed By: aganea, rnk

Differential Revision: https://reviews.llvm.org/D116434

show more ...

Reland "[LLD] Remove global state in lld/COFF" after fixing asan and msan test failures

Original commit description:

[LLD] Remove global state in lld/COFF

This patch removes globals from the l

Reland "[LLD] Remove global state in lld/COFF" after fixing asan and msan test failures

Original commit description:

[LLD] Remove global state in lld/COFF

This patch removes globals from the lldCOFF library, by moving globals
into a context class (COFFLinkingContext) and passing it around wherever
it's needed.

See https://lists.llvm.org/pipermail/llvm-dev/2021-June/151184.html for
context about removing globals from LLD.

I also haven't moved the `driver` or `config` variables yet.

Differential Revision: https://reviews.llvm.org/D109634

This reverts commit a2fd05ada9030eab2258fff25e77a05adccae128.

Original commits were b4fa71eed34d967195514fe9b0a5211fca2bc5bc
and e03c7e367adb8f228332e3c2ef8f45484597b719.

show more ...

Temporarily revert "[LLD] Remove global state in lld/COFF" and "[lld] Add test to
check for timer output"

Seems to be causing a number of asan test failures.

This reverts commit b4fa71eed34d9671955

Temporarily revert "[LLD] Remove global state in lld/COFF" and "[lld] Add test to
check for timer output"

Seems to be causing a number of asan test failures.

This reverts commit b4fa71eed34d967195514fe9b0a5211fca2bc5bc
and e03c7e367adb8f228332e3c2ef8f45484597b719.

show more ...

[LLD] Remove global state in lld/COFF

This patch removes globals from the lldCOFF library, by moving globals
into a context class (COFFLinkingContext) and passing it around wherever
it's needed.

Se

[LLD] Remove global state in lld/COFF

This patch removes globals from the lldCOFF library, by moving globals
into a context class (COFFLinkingContext) and passing it around wherever
it's needed.

See https://lists.llvm.org/pipermail/llvm-dev/2021-June/151184.html for
context about removing globals from LLD.

I also haven't moved the `driver` or `config` variables yet.

Differential Revision: https://reviews.llvm.org/D109634

show more ...

[LTO] Add SelectionKind to IRSymtab and use it in ld.lld/LLVMgold

In PGO, a C++ external linkage function `foo` has a private counter
`__profc_foo` and a private `__profd_foo` in a `comdat nodedupli

[LTO] Add SelectionKind to IRSymtab and use it in ld.lld/LLVMgold

In PGO, a C++ external linkage function `foo` has a private counter
`__profc_foo` and a private `__profd_foo` in a `comdat nodeduplicate`.

A `__attribute__((weak))` function `foo` has a weak hidden counter `__profc_foo`
and a private `__profd_foo` in a `comdat nodeduplicate`.

In `ld.lld a.o b.o`, say a.o defines an external linkage `foo` and b.o
defines a weak `foo`. Currently we treat `comdat nodeduplicate` as `comdat any`,
ld.lld will incorrectly consider `b.o:__profc_foo` non-prevailing. In the worst
case when `b.o:__profd_foo` is retained and `b.o:__profc_foo` isn't, there will
be dangling reference causing an `undefined hidden symbol` error.

Add SelectionKind to `Comdat` in IRSymtab and let linkers ignore nodeduplicate comdat.

Differential Revision: https://reviews.llvm.org/D106228

show more ...

[LLD] [COFF] Fix up missing stdcall decorations in MinGW mode

If linking directly against a DLL without an import library, the
DLL export symbols might not contain stdcall decorations.

If we have a

[LLD] [COFF] Fix up missing stdcall decorations in MinGW mode

If linking directly against a DLL without an import library, the
DLL export symbols might not contain stdcall decorations.

If we have an undefined symbol with decoration, and we happen to have
a matching undecorated symbol (which either is lazy and can be loaded,
or already defined), then alias it against that instead.

This matches what's done in reverse, when we have a def file
declaring to export a symbol without decoration, but we only have
a defined decorated symbol. In that case we do a fuzzy match
(SymbolTable::findMangle). This case is more straightforward; if we
have a decorated undefined symbol, just strip the decoration and look
for the corresponding undecorated symbol name.

Add warnings and options for either silencing the warning or disabling
the whole feature, corresponding to how ld.bfd does it.

(This feature works for any symbol decoration mismatch, not only when
linking against a DLL directly; ld.bfd also tolerates it anywhere,
and also fixes up mismatches in the other direction, like
SymbolTable::findMangle, for any symbol, not only exports. But in
practice, at least for lld, it would primarily end up used for linking
against DLLs.)

Differential Revision: https://reviews.llvm.org/D104532

show more ...

[LLD] [COFF] Support linking directly against DLLs in MinGW mode

GNU ld.bfd supports linking directly against DLLs without using an
import library, and some projects have picked up on this habit.
(T

[LLD] [COFF] Support linking directly against DLLs in MinGW mode

GNU ld.bfd supports linking directly against DLLs without using an
import library, and some projects have picked up on this habit.
(There's no one single unsurmountable issue with using import
libraries, but this is a regularly surfacing missing feature.)

As long as one is linking by name (instead of by ordinal), the DLL
export table contains most of the information needed. (One can
inspect what section a symbol points at, to see if it's a function
or data symbol. The practical implementation of this loops over all
sections for each symbol, but as long as they're not very many, that
should hopefully be tolerable performance wise.)

One exception where the information in the DLL isn't entirely enough
is on i386 with stdcall functions; depending on how they're done,
the exported function name can be a plain undecorated name, while
the import library would contain the full decorated symbol name. This
issue is addressed separately in a different patch.

This is implemented mimicing the structure of a regular import library,
with one InputFile corresponding to the static archive that just adds
lazy symbols, which then are fetched when they are needed. When such
a symbol is fetched, we synthesize a coff_import_header structure
in memory and create a regular ImportFile out of it.

The implementation could be even smaller by just creating ImportFiles
for every symbol available immediately, but that would have the
drawback of actually ending up importing all symbols unless running
with GC enabled (and mingw mode defaults to having it disabled for
historical reasons).

Differential Revision: https://reviews.llvm.org/D104530

show more ...

[LLD] [COFF] Fix handling of LTO comdats with nontrivial selection types after 728cc0075e5dfdb454eb

Commit 728cc0075e5dfdb454ebe6418b63bdffae71ec14 made comdat symbols
from LTO objects be treated as

[LLD] [COFF] Fix handling of LTO comdats with nontrivial selection types after 728cc0075e5dfdb454eb

Commit 728cc0075e5dfdb454ebe6418b63bdffae71ec14 made comdat symbols
from LTO objects be treated as any regular comdat symbol. This works
great for symbols that actually are IMAGE_COMDAT_SELECT_ANY, but
if the symbols have a less trivial selection type that require comparing
either the section chunk size or contents, we can't check that before
actually doing the LTO compilation.

Therefore bring back one aspect of handling from before; that comdat
resolution with a leader from an LTO symbol is essentially skipped,
like it was before 728cc0075e5dfdb454ebe6418b63bdffae71ec14.

Differential Revision: https://reviews.llvm.org/D104605

show more ...

[LLD] [COFF] Fix autoexport from LTO objects with comdat symbols

Make sure that comdat symbols also have a non-null dummy
SectionChunk associated.

This requires moving around an existing FIXME rega

[LLD] [COFF] Fix autoexport from LTO objects with comdat symbols

Make sure that comdat symbols also have a non-null dummy
SectionChunk associated.

This requires moving around an existing FIXME regarding comdats in
LTO.

Differential Revision: https://reviews.llvm.org/D103012

show more ...

[LLD] [COFF] Fix automatic export of symbols from LTO objects

Differential Revision: https://reviews.llvm.org/D101569

[LLD] Implement /guard:[no]ehcont

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D99078

[CFGuard] Add address-taken IAT tables and delay-load support

This patch adds support for creating Guard Address-Taken IAT Entry Tables (.giats$y sections) in object files, matching the behavior of

[CFGuard] Add address-taken IAT tables and delay-load support

This patch adds support for creating Guard Address-Taken IAT Entry Tables (.giats$y sections) in object files, matching the behavior of MSVC. These contain lists of address-taken imported functions, which are used by the linker to create the final GIATS table.
Additionally, if any DLLs are delay-loaded, the linker must look through the .giats tables and add the respective load thunks of address-taken imports to the GFIDS table, as these are also valid call targets.

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D87544

show more ...

Revert "Reland [CFGuard] Add address-taken IAT tables and delay-load support"

This broke both Firefox and Chromium (PR47905) due to what seems like dllimport
function not being handled correctly.

>

Revert "Reland [CFGuard] Add address-taken IAT tables and delay-load support"

This broke both Firefox and Chromium (PR47905) due to what seems like dllimport
function not being handled correctly.

> This patch adds support for creating Guard Address-Taken IAT Entry Tables (.giats$y sections) in object files, matching the behavior of MSVC. These contain lists of address-taken imported functions, which are used by the linker to create the final GIATS table.
> Additionally, if any DLLs are delay-loaded, the linker must look through the .giats tables and add the respective load thunks of address-taken imports to the GFIDS table, as these are also valid call targets.
>
> Reviewed By: rnk
>
> Differential Revision: https://reviews.llvm.org/D87544

This reverts commit cfd8481da1adba1952e0f6ecd00440986e49a946.

show more ...

Reland [CFGuard] Add address-taken IAT tables and delay-load support

This patch adds support for creating Guard Address-Taken IAT Entry Tables (.giats$y sections) in object files, matching the behav

Reland [CFGuard] Add address-taken IAT tables and delay-load support

This patch adds support for creating Guard Address-Taken IAT Entry Tables (.giats$y sections) in object files, matching the behavior of MSVC. These contain lists of address-taken imported functions, which are used by the linker to create the final GIATS table.
Additionally, if any DLLs are delay-loaded, the linker must look through the .giats tables and add the respective load thunks of address-taken imports to the GFIDS table, as these are also valid call targets.

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D87544

show more ...

Revert "[CFGuard] Add address-taken IAT tables and delay-load support"

This reverts commit ef4e971e5e18ae796466623df8f26265ba6bdfb5.

[CFGuard] Add address-taken IAT tables and delay-load support

This patch adds support for creating Guard Address-Taken IAT Entry Tables (.giats$y sections) in object files, matching the behavior of

[CFGuard] Add address-taken IAT tables and delay-load support

This patch adds support for creating Guard Address-Taken IAT Entry Tables (.giats$y sections) in object files, matching the behavior of MSVC. These contain lists of address-taken imported functions, which are used by the linker to create the final GIATS table.
Additionally, if any DLLs are delay-loaded, the linker must look through the .giats tables and add the respective load thunks of address-taken imports to the GFIDS table, as these are also valid call targets.

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D87544

show more ...

[PDB] Split TypeServerSource and extend type index map lifetime

Extending the lifetime of these type index mappings does increase memory
usage (+2% in my case), but it decouples type merging from sy

[PDB] Split TypeServerSource and extend type index map lifetime

Extending the lifetime of these type index mappings does increase memory
usage (+2% in my case), but it decouples type merging from symbol
merging. This is a pre-requisite for two changes that I have in mind:
- parallel type merging: speeds up slow type merging
- defered symbol merging: avoid heap allocating (relocating) all symbols

This eliminates CVIndexMap and moves its data into TpiSource. The maps
are also split into a SmallVector and ArrayRef component, so that the
ipiMap can alias the tpiMap for /Z7 object files, and so that both maps
can simply alias the PDB type server maps for /Zi files.

Splitting TypeServerSource establishes that all input types to be merged
can be identified with two 32-bit indices:
- The index of the TpiSource object
- The type index of the record
This is useful, because this information can be stored in a single
64-bit atomic word to enable concurrent hashtable insertion.

One last change is that now all object files with debugChunks get a
TpiSource, even if they have no type info. This avoids some null checks
and special cases.

Differential Revision: https://reviews.llvm.org/D87736

show more ...

[PDB] Drop LF_PRECOMP from debugTypes earlier

This is a minor simplification to avoid firing up a BinaryStreamReader
and CVType parser.

[LLD] [COFF] Check the aux section definition size for IMAGE_COMDAT_SELECT_SAME_SIZE

Binutils generated sections seem to be padded to a multiple of 16 bytes,
but the aux section definition contains

[LLD] [COFF] Check the aux section definition size for IMAGE_COMDAT_SELECT_SAME_SIZE

Binutils generated sections seem to be padded to a multiple of 16 bytes,
but the aux section definition contains the original, unpadded section
length.

The size check used for IMAGE_COMDAT_SELECT_SAME_SIZE previously
only checked the size of the section itself. When checking the
currently processed object file against the previously chosen
comdat section, we easily have access to the aux section definition
of the currently processed section, but we have to iterate over the
symbols of the previously selected object file to find the section
definition of the previously picked section. (We don't want to
inflate SectionChunk to carry more data, for something that is only
needed in corner cases.) Only do this when the mingw flag is set.

This fixes statically linking clang-built C++ object files against
libstdc++ built with GCC, if the object files contain e.g. typeinfo.

Differential Revision: https://reviews.llvm.org/D86659

show more ...

[COFF] Port CallGraphSort to COFF from ELF

[LLD] [COFF] Fix mingw comdat associativity for leader symbols with a different name

For a weak symbol func in a comdat, the actual leader symbol ends up
named like .weak.func.default*. Likewise, fo

[LLD] [COFF] Fix mingw comdat associativity for leader symbols with a different name

For a weak symbol func in a comdat, the actual leader symbol ends up
named like .weak.func.default*. Likewise, for stdcall on i386, the symbol
may be named _func@4, while the section suffix only is "func", which the
previous implementation didn't handle.

This fixes unwinding through weak functions when using
-ffunction-sections in mingw environments.

Differential Revision: https://reviews.llvm.org/D84607

show more ...