Replace to_hexString by touhexstr [NFC]

LLVM had 2 methods to convert a number to an hexa string,
this remove one of them.

Differential Revision: https://reviews.llvm.org/D127958

[lld-macho][nfc] Run clang-format on lld/MachO/*.{h,cpp}

- fixed inconsistent indents and spaces
- prevent extraneous formatting changes in other patches

Differential Revision: https://reviews.llvm

[lld-macho][nfc] Run clang-format on lld/MachO/*.{h,cpp}

- fixed inconsistent indents and spaces
- prevent extraneous formatting changes in other patches

Differential Revision: https://reviews.llvm.org/D126262

show more ...

[lld-macho] Remove stray debug printf

Accidentally committed as part of b440c25742.

[lld-macho][nfc] Use includeInSymtab for all symtab-skipping logic

{D123302} got me looking deeper at `includeInSymtab`. I thought it was a
little odd that there were excluded (live) symbols for whi

[lld-macho][nfc] Use includeInSymtab for all symtab-skipping logic

{D123302} got me looking deeper at `includeInSymtab`. I thought it was a
little odd that there were excluded (live) symbols for which
`includeInSymtab` was false; we shouldn't have so many different ways to
exclude a symbol. As such, this diff makes the `L`-prefixed-symbol
exclusion code use `includeInSymtab` too. (Note that as part of our
support for `__eh_frame`, we will also be excluding all `__eh_frame`
symbols from the symtab in a future diff.)

Another thing I noticed is that the `emitStabs` code never has to deal
with excluded symbols because `SymtabSection::finalize()` already
filters them out. As such, I've updated the comments and asserts from
{D123302} to reflect this.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D123433

show more ...

Fix format specifier. NFCI.

Using a portable format specifier avoids a "format specifies type
'unsigned long long' but the argument has type 'uint64_t' (aka 'unsigned
long') [-Werror,-Wformat]" erro

Fix format specifier. NFCI.

Using a portable format specifier avoids a "format specifies type
'unsigned long long' but the argument has type 'uint64_t' (aka 'unsigned
long') [-Werror,-Wformat]" error depending on the exact definition of
`uint64_t`.

show more ...

[lld-macho][nfc] Give non-text ConcatOutputSections order-independent finalization

This diff is motivated by my work to add proper DWARF unwind support. As
detailed in PR50956 functions that need DW

[lld-macho][nfc] Give non-text ConcatOutputSections order-independent finalization

This diff is motivated by my work to add proper DWARF unwind support. As
detailed in PR50956 functions that need DWARF unwind need to have
compact unwind entries synthesized for them. These CU entries encode an
offset within `__eh_frame` that points to the corresponding DWARF FDE.

In order to encode this offset during
`UnwindInfoSectionImpl::finalize()`, we need to first assign values to
`InputSection::outSecOff` for each `__eh_frame` subsection. But
`__eh_frame` is ordered after `__unwind_info` (according to ld64 at
least), which puts us in a bit of a bind: `outSecOff` gets assigned
during finalization, but `__eh_frame` is being finalized after
`__unwind_info`.

But it occurred to me that there's no real need for most
ConcatOutputSections to be finalized sequentially. It's only necessary
for text-containing ConcatOutputSections that may contain branch relocs
which may need thunks. ConcatOutputSections containing other types of
data can be finalized in any order.

This diff moves the finalization logic for non-text sections into a
separate `finalizeContents()` method. This method is called before
section address assignment & unwind info finalization takes place. In
theory we could call these `finalizeContents()` methods in parallel, but
in practice it seems to be faster to do it all on the main thread.

Reviewed By: #lld-macho, oontvoo

Differential Revision: https://reviews.llvm.org/D123279

show more ...

[lld/mac] Tweak a few comments

Addresses review feedback I had missed on https://reviews.llvm.org/D122624

No behavior change.

Differential Revision: https://reviews.llvm.org/D122904

[lld/mac] Give range extension thunks for local symbols local visibility

When two local symbols (think: file-scope static functions, or functions in
unnamed namespaces) with the same name in two dif

[lld/mac] Give range extension thunks for local symbols local visibility

When two local symbols (think: file-scope static functions, or functions in
unnamed namespaces) with the same name in two different translation units
both needed thunks, ld64.lld previously created external thunks for both
of them. These thunks ended up with the same name, leading to a duplicate
symbol error for the thunk symbols.

Instead, give thunks for local symbols local visibility.

(Hitting this requires a jump to a local symbol from over 128 MiB away.
It's unlikely that a single .o file is 128 MiB large, but with ICF
you can end up with a situation where the local symbol is ICF'd with
a symbol in a separate translation unit. And that can introduce a
large enough jump to require a thunk.)

Fixes PR54599.

Differential Revision: https://reviews.llvm.org/D122624

show more ...

[lld-macho][nfc] Eliminate InputSection::Shared

Earlier in LLD's evolution, I tried to create the illusion that
subsections were indistinguishable from "top-level" sections. Thus, even
though the su

[lld-macho][nfc] Eliminate InputSection::Shared

Earlier in LLD's evolution, I tried to create the illusion that
subsections were indistinguishable from "top-level" sections. Thus, even
though the subsections shared many common field values, I hid those
common values away in a private Shared struct (see D105305). More
recently, however, @gkm added a public `Section` struct in D113241 that
served as an explicit way to store values that are common to an entire
set of subsections (aka InputSections). Now that we have another "common
value" struct, `Shared` has been rendered redundant. All its fields can
be moved into `Section` instead, and the pointer to `Shared` can be replaced
with a pointer to `Section`.

This `Section` pointer also has the advantage of letting us inspect other
subsections easily, simplifying the implementation of {D118798}.

P.S. I do think that having both `Section` and `InputSection` makes for
a slightly confusing naming scheme. I considered renaming `InputSection`
to `Subsection`, but that would break the symmetry with `OutputSection`.
It would also make us deviate from LLD-ELF's naming scheme.

This change is perf-neutral on my 3.2 GHz 16-Core Intel Xeon W machine:

base diff difference (95% CI)
sys_time 1.258 ± 0.031 1.248 ± 0.023 [ -1.6% .. +0.1%]
user_time 3.659 ± 0.047 3.658 ± 0.041 [ -0.5% .. +0.4%]
wall_time 4.640 ± 0.085 4.625 ± 0.063 [ -1.0% .. +0.3%]
samples 49 61

There's also no stat sig change in RSS (as measured by `time -l`):

base diff difference (95% CI)
time 998038627.097 ± 13567305.958 1003327715.556 ± 15210451.236 [ -0.2% .. +1.2%]
samples 31 36

Reviewed By: #lld-macho, oontvoo

Differential Revision: https://reviews.llvm.org/D118797

show more ...

Re-land [LLD] Remove global state in lldCommon

Move all variables at file-scope or function-static-scope into a hosting structure (lld::CommonLinkerContext) that lives at lldMain()-scope. Drivers wi

Re-land [LLD] Remove global state in lldCommon

Move all variables at file-scope or function-static-scope into a hosting structure (lld::CommonLinkerContext) that lives at lldMain()-scope. Drivers will inherit from this structure and add their own global state, in the same way as for the existing COFFLinkerContext.

See discussion in https://lists.llvm.org/pipermail/llvm-dev/2021-June/151184.html

The previous land f860fe362282ed69b9d4503a20e5d20b9a041189 caused issues in https://lab.llvm.org/buildbot/#/builders/123/builds/8383, fixed by 22ee510dac9440a74b2e5b3fe3ff13ccdbf55af3.

Differential Revision: https://reviews.llvm.org/D108850

show more ...

Revert [LLD] Remove global state in lldCommon

It seems to be causing issues on https://lab.llvm.org/buildbot/#/builders/123/builds/8383

[LLD] Remove global state in lldCommon

Move all variables at file-scope or function-static-scope into a hosting structure (lld::CommonLinkerContext) that lives at lldMain()-scope. Drivers will inher

[LLD] Remove global state in lldCommon

Move all variables at file-scope or function-static-scope into a hosting structure (lld::CommonLinkerContext) that lives at lldMain()-scope. Drivers will inherit from this structure and add their own global state, in the same way as for the existing COFFLinkerContext.

See discussion in https://lists.llvm.org/pipermail/llvm-dev/2021-June/151184.html

Differential Revision: https://reviews.llvm.org/D108850

show more ...

[lld-macho] Increase slops to prevent thunk out of range

One of our internal arm64 apps hit a thunk out of range error when building
with LLD. Per the comment, I'm arbitrarily increasing slop size t

[lld-macho] Increase slops to prevent thunk out of range

One of our internal arm64 apps hit a thunk out of range error when building
with LLD. Per the comment, I'm arbitrarily increasing slop size to 256.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D116705

show more ...

[lld-macho][nfc] InputSections don't need to track their total # of callsites

... only whether they have more than zero. This simplifies the code slightly.

I've also moved the field into the Concat

[lld-macho][nfc] InputSections don't need to track their total # of callsites

... only whether they have more than zero. This simplifies the code slightly.

I've also moved the field into the ConcatInputSection subclass since it doesn't
actually get used by the other InputSections.

Reviewed By: #lld-macho, oontvoo

Differential Revision: https://reviews.llvm.org/D115539

show more ...

[lld-macho] Allow exporting weak_def_can_be_hidden(AKA "autohide") symbols

autohide symbols behaves similarly to private_extern symbols.
However, LD64 allows exporting autohide symbols. LLD currentl

[lld-macho] Allow exporting weak_def_can_be_hidden(AKA "autohide") symbols

autohide symbols behaves similarly to private_extern symbols.
However, LD64 allows exporting autohide symbols. LLD currently does not.
This patch allows LLD to export them.

Differential Revision: https://reviews.llvm.org/D113167

show more ...

[lld-macho][nfc][cleanup] Fix a few code style lints and clang-tidy findings

- Use .empty() instead of `size() == 0` when possible.
- Use const-ref to avoid copying

Differential Revision: https://r

[lld-macho][nfc][cleanup] Fix a few code style lints and clang-tidy findings

- Use .empty() instead of `size() == 0` when possible.
- Use const-ref to avoid copying

Differential Revision: https://reviews.llvm.org/D112978

show more ...

[lld-macho] Associate compact unwind entries with function symbols

Compact unwind entries (CUEs) contain pointers to their respective
function symbols. However, during the link process, it's far mor

[lld-macho] Associate compact unwind entries with function symbols

Compact unwind entries (CUEs) contain pointers to their respective
function symbols. However, during the link process, it's far more useful
to have pointers from the function symbol to the CUE than vice versa.
This diff adds that pointer in the form of `Defined::compactUnwind`.

In particular, when doing dead-stripping, we want to mark CUEs live when
their function symbol is live; and when doing ICF, we want to dedup
sections iff the symbols in that section have identical CUEs. In both
cases, we want to be able to locate the symbols within a given section,
as well as locate the CUEs belonging to those symbols. So this diff also
adds `InputSection::symbols`.

The ultimate goal of this refactor is to have ICF support dedup'ing
functions with unwind info, but that will be handled in subsequent
diffs. This diff focuses on simplifying `-dead_strip` --
`findFunctionsWithUnwindInfo` is no longer necessary, and
`Defined::isLive()` is now a lot simpler. Moreover, UnwindInfoSection no
longer has to check for dead CUEs -- we simply avoid adding them in the
first place.

Additionally, we now support stripping of dead LSDAs, which follows
quite naturally since `markLive()` can now reach them via the CUEs.

Reviewed By: #lld-macho, gkm

Differential Revision: https://reviews.llvm.org/D109944

show more ...

[lld/mac] Fix comment typo to cycle bots

[lld/mac] Leave more room for thunks in thunk placement code

Fixes PR51578 in practice.

Currently there's only enough room for a single thunk, which for real-life code
isn't enough. The error case

[lld/mac] Leave more room for thunks in thunk placement code

Fixes PR51578 in practice.

Currently there's only enough room for a single thunk, which for real-life code
isn't enough. The error case only happens when there are many branch statements
very close to each other (0 or 1 instructions apart), with the function at the
finalization barrier small.

There's a FIXME on what to do if we hit this case, but that suggestion sounds
complicated to me (see end of PR51578 comment 5 for why).

Instead, just leave more room for thunks. Chromium's unit_tests links fine with
room for 3 thunks. Leave room for 100, which should fix this for most cases in
practice.

There's little cost for leaving lots of room: This slop value only determines
when we finalize sections, and we insert thunks for forward jumps into
unfinalized sections. So leaving room means we'll need a few more thunks, but
the thunk jump range is 128 MiB while a single thunk is just 12 bytes.

For Chromium's unit_tests:
With a slop of 3: thunk calls = 355418, thunks = 10903
With a slop of 100: thunk calls = 355426, thunks = 10904

Chances are 100 is enough for all use cases we'll hit in practice, but even
bumping it to 1000 would probably be fine.

Differential Revision: https://reviews.llvm.org/D108930

show more ...

[lld/mac] Tweak estimateStubsInRangeVA a bit

- Move a few variables closer to their uses, remove some completely
(no behavior change)
- Add some comments
- Make maxPotentialThunks include calls to

[lld/mac] Tweak estimateStubsInRangeVA a bit

- Move a few variables closer to their uses, remove some completely
(no behavior change)
- Add some comments
- Make maxPotentialThunks include calls to stubs. It's possible that
an earlier call to a stub late in the stub table will need a thunk,
and that inserted thunk could push a stub earlier in the stub table
out of range. This is unlikely to happen, but usually there are
way fewer stub calls than non-stub calls, so if we're doing a
conservative approximation here we might as well do it correctly.
(For chromium's unit_tests target, 134421/242639 stub calls are
direct calls without this change, compared to 134408/242639 with
this change)

No real, meaningful behavior difference.

Differential Revision: https://reviews.llvm.org/D108924

show more ...

[lld/mac] Set branchRange a bit more carefully

- Don't subtract thunkSize from branchRange. Most places care about
the actual maximal branch range. Subtract thunkSize in the one place
that wants

[lld/mac] Set branchRange a bit more carefully

- Don't subtract thunkSize from branchRange. Most places care about
the actual maximal branch range. Subtract thunkSize in the one place
that wants to leave room for a thunk.
- Set it to 0x800_0000 instead of 0xFF_FFFF
- Subtract 4 for the positive branch direction since it's a
two's complement 24bit number sign-extended mutiplied by 4,
so its range is -0x800_0000..+0x7FF_FFFC
- Make boundary checks include the boundary values

This doesn't make a huge difference in practice. It's preparation
for a "real" fix for PR51578 -- but it also lets the repro in comment 0
in that bug place one more thunk before hitting the TODO.

Differential Revision: https://reviews.llvm.org/D108897

show more ...

[lld/mac] Don't assert on -dead_strip + arm64 range extension thunks

The assert is harmless and thinks worked fine in builds with asserts enabled,
but it's still nice to fix the assert.

Differentia

[lld/mac] Don't assert on -dead_strip + arm64 range extension thunks

The assert is harmless and thinks worked fine in builds with asserts enabled,
but it's still nice to fix the assert.

Differential Revision: https://reviews.llvm.org/D108853

show more ...

[lld/mac] Move output segment rename logic into OutputSegment

Fixes the output segment name if both -rename_section and
-rename_segment are used and the post-section-rename segment
name is the same

[lld/mac] Move output segment rename logic into OutputSegment

Fixes the output segment name if both -rename_section and
-rename_segment are used and the post-section-rename segment
name is the same as the pre-segment-rename segment name to
match ld64's behavior.

The motivation is that segment$start$ can create section-less segments,
and this makes a corner case in the interaction between segment$start and
-rename_segment in the upcoming segment$start patch.

Differential Revision: https://reviews.llvm.org/D106766

show more ...

[lld-macho] Have ICF operate on all sections at once

ICF previously operated only within a given OutputSection. We would
merge all CFStrings first, then merge all regular code sections in a
second p

[lld-macho] Have ICF operate on all sections at once

ICF previously operated only within a given OutputSection. We would
merge all CFStrings first, then merge all regular code sections in a
second phase. This worked fine since CFStrings would never reference
regular `__text` sections. However, I would like to expand ICF to merge
functions that reference unwind info. Unwind info references the LSDA
section, which can in turn reference the `__text` section, so we cannot
perform ICF in phases.

In order to have ICF operate on InputSections spanning multiple
OutputSections, we need a way to distinguish InputSections that are
destined for different OutputSections, so that we don't fold across
section boundaries. We achieve this by creating OutputSections early,
and setting `InputSection::parent` to point to them. This is what
LLD-ELF does. (This change should also make it easier to implement the
`section$start$` symbols.)

This diff also folds InputSections w/o checking their flags, which I
think is the right behavior -- if they are destined for the same
OutputSection, they will have the same flags in the output (even if
their input flags differ). I.e. the `parent` pointer check subsumes the
`flags` check. In practice this has nearly no effect (ICF did not become
any more effective on chromium_framework).

I've also updated ICF.cpp's block comment to better reflect its current
status.

Reviewed By: #lld-macho, smeenai

Differential Revision: https://reviews.llvm.org/D105641

show more ...

[lld-macho] Parse relocations quickly by assuming sorted order

clang and gcc both seem to emit relocations in reverse order of
address. That means we can match relocations to their containing
subsec

[lld-macho] Parse relocations quickly by assuming sorted order

clang and gcc both seem to emit relocations in reverse order of
address. That means we can match relocations to their containing
subsections in `O(relocs + subsections)` rather than the `O(relocs *
log(subsections))` that our previous binary search implementation
required.

Unfortunately, `ld -r` can still emit unsorted relocations, so we have a
fallback code path for that (less common) case.

Numbers for linking chromium_framework on my 3.2 GHz 16-Core Intel Xeon W:

N Min Max Median Avg Stddev
x 20 4.04 4.11 4.075 4.0775 0.018027756
+ 20 3.95 4.02 3.98 3.985 0.020900768
Difference at 95.0% confidence
-0.0925 +/- 0.0124919
-2.26855% +/- 0.306361%
(Student's t, pooled s = 0.0195172)

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D105410

show more ...