[ELF] Simplify createObjectFile/createLazyFile. NFC

And avoid redundant identify_magic test.

[LTO][ELF] Add selective --save-temps= option

Allows specific “temps” to be saved, instead of the current all-or-nothing nature of --save-temps. Multiple of these “temps” can be saved by specifying

[LTO][ELF] Add selective --save-temps= option

Allows specific “temps” to be saved, instead of the current all-or-nothing nature of --save-temps. Multiple of these “temps” can be saved by specifying the argument multiple times.

Differential Revision: https://reviews.llvm.org/D127778

show more ...

[ELF] Move InputFiles global variables (memoryBuffers, objectFiles, etc) into Ctx. NFC

[ThinLTO][ELF] Add --thinlto-emit-index-files option

Allows ThinLTO indices to be written to disk on-the-fly/as-part-of “normal” linker execution. Previously ThinLTO indices could be written via --t

[ThinLTO][ELF] Add --thinlto-emit-index-files option

Allows ThinLTO indices to be written to disk on-the-fly/as-part-of “normal” linker execution. Previously ThinLTO indices could be written via --thinlto-index-only but that would cause the linker to exit early. For MLGO specifically, this enables saving the ThinLTO index files without having to restart the linker to collect data only available at later stages (i.e. output of --save-temps) of the linker's execution.

Note, this option does not currently work with:
--thinlto-object-suffix-replace, as this is intended to be used to consume minimized IR bitcode files while --thinlto-emit-index-files is intended to be run together with InProcessThinLTO (which cannot parse minimized IR).
--thinlto-prefix-replace support is left unimplemented but can be implemented if needed

Differential Revision: https://reviews.llvm.org/D127777

show more ...

LTO: Decide upfront whether to use opaque/non-opaque pointer types

LTO code may end up mixing bitcode files from various sources varying in
their use of opaque pointer types. The current strategy to

LTO: Decide upfront whether to use opaque/non-opaque pointer types

LTO code may end up mixing bitcode files from various sources varying in
their use of opaque pointer types. The current strategy to decide
between opaque / typed pointers upon the first bitcode file loaded does
not work here, since we could be loading a non-opaque bitcode file first
and would then be unable to load any files with opaque pointer types
later.

So for LTO this:
- Adds an `lto::Config::OpaquePointer` option and enforces an upfront
decision between the two modes.
- Adds `-opaque-pointers`/`-no-opaque-pointers` options to the gold
plugin; disabled by default.
- `--opaque-pointers`/`--no-opaque-pointers` options with
`-plugin-opt=-opaque-pointers`/`-plugin-opt=-no-opaque-pointers`
aliases to lld; disabled by default.
- Adds an `-lto-opaque-pointers` option to the `llvm-lto2` tool.
- Changes the clang driver to pass `-plugin-opt=-opaque-pointers` to
the linker in LTO modes when clang was configured with opaque
pointers enabled by default.

This fixes https://github.com/llvm/llvm-project/issues/55377

Differential Revision: https://reviews.llvm.org/D125847

show more ...

[ELF] Fix wrapping symbols produced during LTO codegen

We were previously not correctly wrapping symbols that were only
produced during LTO codegen and unreferenced before then, or symbols
only refe

[ELF] Fix wrapping symbols produced during LTO codegen

We were previously not correctly wrapping symbols that were only
produced during LTO codegen and unreferenced before then, or symbols
only referenced from such symbols. The root cause was that we weren't
marking the wrapped symbol as used if we only saw the use after LTO
codegen, leading to the failed wrapping.

Fix this by explicitly tracking whether a symbol will become referenced
after wrapping is done. We can use this property to tell LTO to preserve
such symbols, instead of overload isUsedInRegularObj for this purpose.
Since we're no longer setting isUsedInRegularObj for all symbols which
will be wrapped, its value at the time of performing the wrapping in the
symbol table will accurately reflect whether the symbol was actually
used in an object (including in an LTO-generated object), and we can
propagate that value to the wrapped symbol and thereby ensure we wrap
correctly.

This incorrect wrapping was the only scenario I was aware of where we
produced an invalid PLT relocation, which D123985 started diagnosing,
and with it fixed, we lose the test for that diagnosis. I think it's
worth keeping the diagnosis though, in case we run into other issues in
the future which would be caught by it.

Fixes PR50675.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D124056

show more ...

[lld] Remove support for legacy pass manager

This removes options for performing LTO with the legacy pass
manager in LLD. Options that explicitly enable the new pass manager
are retained as no-ops.

[lld] Remove support for legacy pass manager

This removes options for performing LTO with the legacy pass
manager in LLD. Options that explicitly enable the new pass manager
are retained as no-ops.

Differential Revision: https://reviews.llvm.org/D123219

show more ...

Reland "[ELF] Enable new passmanager plugin support for LTO"

This is the orignal patch + a check that LLVM_BUILD_EXAMPLES is enabled before
adding a dependency on the 'Bye' example pass.

Original s

Reland "[ELF] Enable new passmanager plugin support for LTO"

This is the orignal patch + a check that LLVM_BUILD_EXAMPLES is enabled before
adding a dependency on the 'Bye' example pass.

Original summary:

Add cli options for new passmanager plugin support to lld.

Currently it is not possible to load dynamic NewPM plugins with lld. This is an
incremental update to D76866. While that patch only added cli options for
llvm-lto2, this adds them for lld as well. This is especially useful for running
dynamic plugins on the linux kernel with LTO.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D120490

show more ...

Revert "[ELF] Enable new passmanager plugin support for LTO"

This reverts commit 32012eb11b235e1560a253664095676ea8ebd027.

Broke CMake configuration.

[ELF] Enable new passmanager plugin support for LTO

Add cli options for new passmanager plugin support to lld.

Currently it is not possible to load dynamic NewPM plugins with lld. This is an
increm

[ELF] Enable new passmanager plugin support for LTO

Add cli options for new passmanager plugin support to lld.

Currently it is not possible to load dynamic NewPM plugins with lld. This is an
incremental update to D76866. While that patch only added cli options for
llvm-lto2, this adds them for lld as well. This is especially useful for running
dynamic plugins on the linux kernel with LTO.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D120490

show more ...

[LTO][ELF] Add --stats-file= option.

This patch adds a StatsFile option supported by gold to lld, related patch https://reviews.llvm.org/D45531.

Reviewed By: tejohnson, MaskRay

Differential Revisi

[LTO][ELF] Add --stats-file= option.

This patch adds a StatsFile option supported by gold to lld, related patch https://reviews.llvm.org/D45531.

Reviewed By: tejohnson, MaskRay

Differential Revision: https://reviews.llvm.org/D121809

show more ...

[ELF] Set config->exportDynamic to true if config->shared. NFC

[ELF] Clean up headers. NFC

[ELF] Symbol::replace: use the old nameData/nameSize. NFC

Currently `this->getName() == newSym.getName()`.
By keeping the old nameData/nameSize, newSym's nameData/nameSize will be
ignored. The call

[ELF] Symbol::replace: use the old nameData/nameSize. NFC

Currently `this->getName() == newSym.getName()`.
By keeping the old nameData/nameSize, newSym's nameData/nameSize will be
ignored. The call sites can avoid calling getName().

printTraceSymbol needs to take the symbol name since `other`'s name is empty.

show more ...

[ELF] Merge canInline into scriptDefined

They perform similar tasks and are essentially the same after
d28c26bbdd9e6e55cc0a6156e9879f7e0ca36329.

[ELF] Refactor how exportDynamic is set. NFC

[ELF] Allow STV_PROTECTED shared definition to set exportDynamic

A STV_PROTECTED shared definition does not set exportDynamic of a defined
symbol. This is on the basis that a protected definition ca

[ELF] Allow STV_PROTECTED shared definition to set exportDynamic

A STV_PROTECTED shared definition does not set exportDynamic of a defined
symbol. This is on the basis that a protected definition cannot be preempted so
the export is unnecessary. However, the condition is imperfect because we don't
know whether the shared object was built with a symbolic option. Since dropping
the condition simplifies code and matches GNU ld, let's do it.

show more ...

[BitcodeWriter] Fix cases of some functions

`WriteIndexToFile` is used by external projects so I do not touch it.

[ELF] SymbolTable::symbols: don't filter out PlaceholderKind

Placeholders (-y and redirectSymbols removed versioned symbols) are very rare and
the check just makes symbol table iteration slower. Mos

[ELF] SymbolTable::symbols: don't filter out PlaceholderKind

Placeholders (-y and redirectSymbols removed versioned symbols) are very rare and
the check just makes symbol table iteration slower. Most iterations filter out
placeholders anyway, so this change just drops the filter behavior.

For "Add symbols to symtabs", we need to ensure that redirectSymbols sets
isUsedInRegularObj to false when making a symbol placeholder, to avoid an
assertion failure in SymbolTableSection<ELFT>::writeTo.

My .text is 2KiB smaller. The speed-up linking chrome is 0.x%.

show more ...

[ELF] LTO: skip expensive usedStartStop initialization if bitcodeFiles.empty()

This may cost 1.3+% of total link time.

[ELF] Replace LazyObjFile with lazy ObjFile/BitcodeFile

The new `lazy` state is the inverse of the previous `LazyObjFile::extracted`.
There are many advantages:

* previously when a LazyObjFile was

[ELF] Replace LazyObjFile with lazy ObjFile/BitcodeFile

The new `lazy` state is the inverse of the previous `LazyObjFile::extracted`.
There are many advantages:

* previously when a LazyObjFile was extracted, a new ObjFile/BitcodeFile was created; now the file is reused, just with `lazy` cleared
* avoid the confusing transfer of `symbols` from LazyObjFile to the new file
* the `incompatible file:` diagnostic is unified with `is incompatible with`
* simpler code, smaller executable (6200+ bytes smaller on x86-64)
* make eager parsing feasible (for parallel section/symbol table initialization)

show more ...

[ELF] Rename fetch to extract

The canonical term is "extract" (GNU ld documentation, Solaris's `-z *extract`
options). Avoid inventing a term and match --why-extract. (ld64 prefers "load"
but the wo

[ELF] Rename fetch to extract

The canonical term is "extract" (GNU ld documentation, Solaris's `-z *extract`
options). Avoid inventing a term and match --why-extract. (ld64 prefers "load"
but the word is overloaded too much)

Mostly MFC, except for --help messages and the header row in
--print-archive-stats output.

show more ...

[Support] Improve Caching conformance with Support library behavior

This diff makes several amendments to the local file caching mechanism
which was migrated from ThinLTO to Support in
rGe678c511771

[Support] Improve Caching conformance with Support library behavior

This diff makes several amendments to the local file caching mechanism
which was migrated from ThinLTO to Support in
rGe678c51177102845c93529d457b020f969125373 in response to follow-up
discussion on that commit.

Patch By: noajshu

Differential Revision: https://reviews.llvm.org/D113080

show more ...

[Support][ThinLTO] Move ThinLTO caching to LLVM Support library

We would like to move ThinLTO’s battle-tested file caching mechanism to
the LLVM Support library so that we can use it elsewhere in LL

[Support][ThinLTO] Move ThinLTO caching to LLVM Support library

We would like to move ThinLTO’s battle-tested file caching mechanism to
the LLVM Support library so that we can use it elsewhere in LLVM.

Patch By: noajshu

Differential Revision: https://reviews.llvm.org/D111371

show more ...

Revert "[Support][ThinLTO] Move ThinLTO caching to LLVM Support library"

This reverts commit 92b8cc52bbc8194f2cd6a5f742b874969421afca since
it broke the gold plugin.