[ELF] Move InputFiles global variables (memoryBuffers, objectFiles, etc) into Ctx. NFC

Rename parallelForEachN to just parallelFor

Patch created by running:

rg -l parallelForEachN | xargs sed -i '' -c 's/parallelForEachN/parallelFor/'

No behavior change.

Differential Revision: ht

Rename parallelForEachN to just parallelFor

Patch created by running:

rg -l parallelForEachN | xargs sed -i '' -c 's/parallelForEachN/parallelFor/'

No behavior change.

Differential Revision: https://reviews.llvm.org/D128140

show more ...

[ELF] Don't use multiple inheritance for OutputSection. NFC

Add an OutputDesc class inheriting from SectionCommand. An OutputDesc wraps an
OutputSection. This change allows InputSection::getParent t

[ELF] Don't use multiple inheritance for OutputSection. NFC

Add an OutputDesc class inheriting from SectionCommand. An OutputDesc wraps an
OutputSection. This change allows InputSection::getParent to be inlined.

Differential Revision: https://reviews.llvm.org/D120650

show more ...

[ELF] Move --print-archive-stats= and --why-extract= beside --warn-backrefs report

So that early errors don't suppress their output.

[ELF] Parse archives as --start-lib object files

https://maskray.me/blog/2022-01-16-archives-and-start-lib

For every definition in an extracted archive member, we intern the symbol twice,
once for

[ELF] Parse archives as --start-lib object files

https://maskray.me/blog/2022-01-16-archives-and-start-lib

For every definition in an extracted archive member, we intern the symbol twice,
once for the archive index entry, once for the .o symbol table after extraction.
This is inefficient.

Symbols in a --start-lib ObjFile/BitcodeFile are only interned once because the
result is cached in symbols[i].

Just handle an archive using the --start-lib code path. We can therefore remove
ArchiveFile and LazyArchive. For many projects, archive member extraction ratio
is high and it is a net performance win. Linking a Release build of clang is
1.01x as fast.

Note: --start-lib scans symbols in the same order that llvm-ar adds them to the
index, so in the common case the semantics should be identical. If the archive
symbol table was created in a different order, or is incomplete, this strategy
may have different semantics. Such cases are considered user error.

The `is neither ET_REL nor LLVM bitcode` error is changed to a warning.
Previously an archive may have such members without a diagnostic. Using a
warning prevents breakage.

* For some tests, the diagnostics get improved where we did not consider
the archive member name: `b.a:` => `b.a(b.o):`.
* `no-obj.s`: the link is now allowed, matching GNU ld
* `archive-no-index.s`: the `is neither ET_REL nor LLVM bitcode` diagnostic is
demoted to a warning.
* `incompatible.s`: even when an archive is unextracted, we may report an
"incompatible with" error.

---

I recently decreased sizeof(SymbolUnion) by 8 and decreased memory usage quite a
bit, so retaining `symbols` for un-extracted archive members should not cause a
memory usage problem.

Reviewed By: peter.smith

Differential Revision: https://reviews.llvm.org/D119074

show more ...

[ELF] Clean up headers. NFC

[ELF] Optimize two vector. NFC

[ELF] Optimize -Map. NFC

getVA is slow. Avoid calling it in the llvm::sort comparator.

[ELF] Change objectFiles to ELFFileBase *

This can sometimes avoid `cast<ObjFile<...>>`.

I intentionally do not touch postScanRelocations to wait for its stabilization.

Reland D114783/D115603 [ELF] Split scanRelocations into scanRelocations/postScanRelocations

(Fixed an issue about GOT on a copy relocated alias.)
(Fixed an issue about not creating r_addend=0 IRELAT

Reland D114783/D115603 [ELF] Split scanRelocations into scanRelocations/postScanRelocations

(Fixed an issue about GOT on a copy relocated alias.)
(Fixed an issue about not creating r_addend=0 IRELATIVE for unreferenced non-preemptible ifunc.)

The idea is to make scanRelocations mark some actions are needed (GOT/PLT/etc)
and postpone the real work to postScanRelocations. It gives some flexibility:

* Make it feasible to support .plt.got (PR32938): we need to know whether GLOB_DAT and JUMP_SLOT are both needed.
* Make non-preemptible IFUNC handling slightly cleaner: avoid setting/clearing sym.gotInIgot
* -z nocopyrel: report all copy relocation places for one symbol
* Make GOT deduplication feasible
* Make parallel relocation scanning feasible (if we can avoid all stateful operations and make Symbol attributes atomic), but parallelism may not be the appealing choice

Since this patch moves a large chunk of code out of ELFT templates. My x86-64
executable is actually a few hundred bytes smaller.

For ppc32-ifunc-nonpreemptible-pic.s: I remove absolute relocation references to non-preemptible ifunc
because absolute relocation references are incorrect in -fpie mode.

Reviewed By: peter.smith, ikudrin

Differential Revision: https://reviews.llvm.org/D114783

show more ...

Revert D114783 [ELF] Split scanRelocations into scanRelocations/postScanRelocations

May cause a failure for non-preemptible `bcmp` in a glibc -static link.

[ELF] -Map: Print symbols which needs canonical PLT entry/copy relocation just once

If a copy related symbol (say `copy`) is referenced in two .o
files, this change removes a duplicated line from th

[ELF] -Map: Print symbols which needs canonical PLT entry/copy relocation just once

If a copy related symbol (say `copy`) is referenced in two .o
files, this change removes a duplicated line from the -Map output:

```
202470 202470 1 1 .bss.rel.ro
202470 202470 1 1 <internal>:(.bss.rel.ro)
202470 202470 1 1 copy
removed 202470 202470 1 1 copy
```

Differential Revision: https://reviews.llvm.org/D115697

show more ...

[ELF] Remove needsPltAddr in favor of needsCopy

needsPltAddr is equivalent to `needsCopy && isFunc`. In many places, it is
equivalent to `needsCopy` because the non-STT_FUNC cases are ruled out.

Re

[ELF] Remove needsPltAddr in favor of needsCopy

needsPltAddr is equivalent to `needsCopy && isFunc`. In many places, it is
equivalent to `needsCopy` because the non-STT_FUNC cases are ruled out.

Reviewed By: ikudrin, peter.smith

Differential Revision: https://reviews.llvm.org/D115603

show more ...

[ELF] --cref: If -Map is specified, print to the map file

PR48282: This behavior matches GNU ld and gold.

Reviewed By: markj

Differential Revision: https://reviews.llvm.org/D114663

[ELF] Inline InputSection::getOffset into callers and remove it. NFC

This is an unneeded abstraction which may cause confusion:
SectionBase::getOffset has the same name but hard codes -1 as the size

[ELF] Inline InputSection::getOffset into callers and remove it. NFC

This is an unneeded abstraction which may cause confusion:
SectionBase::getOffset has the same name but hard codes -1 as the size of OutputSection.

show more ...

[ELF] Rename fetch to extract

The canonical term is "extract" (GNU ld documentation, Solaris's `-z *extract`
options). Avoid inventing a term and match --why-extract. (ld64 prefers "load"
but the wo

[ELF] Rename fetch to extract

The canonical term is "extract" (GNU ld documentation, Solaris's `-z *extract`
options). Avoid inventing a term and match --why-extract. (ld64 prefers "load"
but the word is overloaded too much)

Mostly MFC, except for --help messages and the header row in
--print-archive-stats output.

show more ...

[ELF] Rename BaseCommand to SectionCommand. NFC

BaseCommand was picked when PHDRS/INSERT/etc were not implemented. Rename it to
SectionCommand to match `sectionCommands` and make it clear that the c

[ELF] Rename BaseCommand to SectionCommand. NFC

BaseCommand was picked when PHDRS/INSERT/etc were not implemented. Rename it to
SectionCommand to match `sectionCommands` and make it clear that the commands
are used in SECTIONS (except a special case for SymbolAssignment).

Also, improve naming of some BaseCommand variables (base -> cmd).

show more ...

[ELF] Rename OutputSection::sectionCommands to commands. NFC

This partially reverts r315409: the description applies to LinkerScript, but not
to OutputSection.

The name "sectionCommands" is used in

[ELF] Rename OutputSection::sectionCommands to commands. NFC

This partially reverts r315409: the description applies to LinkerScript, but not
to OutputSection.

The name "sectionCommands" is used in both LinkerScript::sectionCommands and
OutputSection::sectionCommands, which may lead to confusion.
"commands" in OutputSection has no ambiguity because there are no other types
of commands.

show more ...

[ELF] Add --why-extract= to query why archive members/lazy object files are extracted

Similar to D69607 but for archive member extraction unrelated to GC. This patch adds --why-extract=.

Prior art:

[ELF] Add --why-extract= to query why archive members/lazy object files are extracted

Similar to D69607 but for archive member extraction unrelated to GC. This patch adds --why-extract=.

Prior art:

GNU ld -M prints
```
Archive member included to satisfy reference by file (symbol)

a.a(a.o) main.o (a)
b.a(b.o) (b())
```

-M is mainly for input section/symbol assignment <-> output section mapping
(often huge output) and the information may appear ad-hoc.

Apple ld64
```
__Z1bv forced load of b.a(b.o)
_a forced load of a.a(a.o)
```

It doesn't say the reference file.

Arm's proprietary linker
```
Selecting member vsnprintf.o(c_wfu.l) to define vsnprintf.
...
Loading member vsnprintf.o from c_wfu.l.
definition: vsnprintf
reference : _printf_a
```

---

--why-extract= gives the user the full data (which is much shorter than GNU ld
-Map). It is easy to track a chain of references to one archive member with a
one-liner, e.g.

```
% ld.lld main.o a_b.a b_c.a c.a -o /dev/null --why-extract=- | tee stdout
reference extracted symbol
main.o a_b.a(a_b.o) a
a_b.a(a_b.o) b_c.a(b_c.o) b()
b_c.a(b_c.o) c.a(c.o) c()

% ruby -ane 'BEGIN{p={}}; p[$F[1]]=[$F[0],$F[2]] if $.>1; END{x="c.a(c.o)"; while y=p[x]; puts "#{y[0]} extracts #{x} to resolve #{y[1]}"; x=y[0] end}' stdout
b_c.a(b_c.o) extracts c.a(c.o) to resolve c()
a_b.a(a_b.o) extracts b_c.a(b_c.o) to resolve b()
main.o extracts a_b.a(a_b.o) to resolve a
```

Archive member extraction happens before --gc-sections, so this may not be a live path
under --gc-sections, but I think it is a good approximation in practice.

* Specifying a file avoids output interleaving with --verbose.
* Required `=` prevents accidental overwrite of an input if the user forgets `=`. (Most of compiler drivers' long options accept `=` but not ` `)

Differential Revision: https://reviews.llvm.org/D109572

show more ...

[lld][ELF] Add additional time trace categories

I noticed when running a large link with the --time-trace option that
there were several areas which were missing any specific time trace
categories (

[lld][ELF] Add additional time trace categories

I noticed when running a large link with the --time-trace option that
there were several areas which were missing any specific time trace
categories (aside from the generic link/ExecuteLinker categories). This
patch adds new categories to fill most of the "gaps", or to provide more
detail than was previously provided.

Reviewed by: MaskRay, grimar, russell.gallop

Differential Revision: https://reviews.llvm.org/D90686

show more ...

[ELF] Use namespace qualifiers (lld:: or elf::) instead of `namespace lld { namespace elf {`

Similar to D74882. This reverts much code from commit
bd8cfe65f5fee4ad573adc2172359c9552e8cdc0 (D68323) a

[ELF] Use namespace qualifiers (lld:: or elf::) instead of `namespace lld { namespace elf {`

Similar to D74882. This reverts much code from commit
bd8cfe65f5fee4ad573adc2172359c9552e8cdc0 (D68323) and fixes some
problems before D68323.

Sorry for the churn but D68323 was a mistake. Namespace qualifiers avoid
bugs where the definition does not match the declaration from the
header. See
https://llvm.org/docs/CodingStandards.html#use-namespace-qualifiers-to-implement-previously-declared-functions (D74515)

Differential Revision: https://reviews.llvm.org/D79982

show more ...

[Support] Move LLD's parallel algorithm wrappers to support

Essentially takes the lld/Common/Threads.h wrappers and moves them to
the llvm/Support/Paralle.h algorithm header.

The changes are:
- Rem

[Support] Move LLD's parallel algorithm wrappers to support

Essentially takes the lld/Common/Threads.h wrappers and moves them to
the llvm/Support/Paralle.h algorithm header.

The changes are:
- Remove policy parameter, since all clients use `par`.
- Rename the methods to `parallelSort` etc to match LLVM style, since
they are no longer C++17 pstl compatible.
- Move algorithms from llvm::parallel:: to llvm::, since they have
"parallel" in the name and are no longer overloads of the regular
algorithms.
- Add range overloads
- Use the sequential algorithm directly when 1 thread is requested
(skips task grouping)
- Fix the index type of parallelForEachN to size_t. Nobody in LLVM was
using any other parameter, and it made overload resolution hard for
for_each_n(par, 0, foo.size(), ...) because 0 is int, not size_t.

Remove Threads.h and update LLD for that.

This is a prerequisite for parallel public symbol processing in the PDB
library, which is in LLVM.

Reviewed By: MaskRay, aganea

Differential Revision: https://reviews.llvm.org/D79390

show more ...

[ELF] Add --print-archive-stats=

gold has an option --print-symbol-counts= which prints:

// For each archive
archive $archive $members $fetched_members
// For each object file
symbols $obje

[ELF] Add --print-archive-stats=

gold has an option --print-symbol-counts= which prints:

// For each archive
archive $archive $members $fetched_members
// For each object file
symbols $object $defined_symbols $used_defined_symbols

In most cases, `$defined_symbols = $used_defined_symbols` unless weak
symbols are present. Strangely `$used_defined_symbols` includes symbols defined relative to --gc-sections discarded sections.
The `symbols` lines do not appear to be useful.

`archive` lines are useful: `$fetched_members=0` lines correspond to
unused archives. The information can be used to trim dependencies.

This patch implements --print-archive-stats= which prints the number of
members and the number of fetched members for each archive.

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D78983

show more ...

Make it possible to redirect not only errs() but also outs()

This change is for those who use lld as a library. Context:
https://reviews.llvm.org/D70287

This patch adds a new parmeter to lld::*::li

Make it possible to redirect not only errs() but also outs()

This change is for those who use lld as a library. Context:
https://reviews.llvm.org/D70287

This patch adds a new parmeter to lld::*::link() so that we can pass
an raw_ostream object representing stdout. Previously, lld::*::link()
took only an stderr object.

Justification for making stdoutOS and stderrOS mandatory: I wanted to
make link() functions to take stdout and stderr in that order.
However, if we change the function signature from

bool link(ArrayRef<const char *> args, bool canExitEarly,
raw_ostream &stderrOS = llvm::errs());

to

bool link(ArrayRef<const char *> args, bool canExitEarly,
raw_ostream &stdoutOS = llvm::outs(),
raw_ostream &stderrOS = llvm::errs());

, then the meaning of existing code that passes stderrOS silently
changes (stderrOS would be interpreted as stdoutOS). So, I chose to
make existing code not to compile, so that developers can fix their
code.

Differential Revision: https://reviews.llvm.org/D70292

show more ...

[ELF] Wrap things in `namespace lld { namespace elf {`, NFC

This makes it clear `ELF/**/*.cpp` files define things in the `lld::elf`
namespace and simplifies `elf::foo` to `foo`.

Reviewed By: atana

[ELF] Wrap things in `namespace lld { namespace elf {`, NFC

This makes it clear `ELF/**/*.cpp` files define things in the `lld::elf`
namespace and simplifies `elf::foo` to `foo`.

Reviewed By: atanasyan, grimar, ruiu

Differential Revision: https://reviews.llvm.org/D68323

llvm-svn: 373885

show more ...