[lld-macho] Demangle location name in undefined symbol diagnostics

If the `-demangle` flag is passed to lld, symbol names will now be
demangled in the "referenced by:" message in addition to the ref

[lld-macho] Demangle location name in undefined symbol diagnostics

If the `-demangle` flag is passed to lld, symbol names will now be
demangled in the "referenced by:" message in addition to the referenced
symbol's name, which was already demangled before this change.

Differential Revision: https://reviews.llvm.org/D130490

show more ...

[lld-macho] Support folding of functions with identical LSDAs

To do this, we need to slice away the LSDA pointer, just like we are
slicing away the functionAddress pointer.

No observable difference

[lld-macho] Support folding of functions with identical LSDAs

To do this, we need to slice away the LSDA pointer, just like we are
slicing away the functionAddress pointer.

No observable difference in perf on chromium_framework:

base diff difference (95% CI)
sys_time 1.769 ± 0.068 1.761 ± 0.065 [ -2.7% .. +1.8%]
user_time 9.517 ± 0.110 9.528 ± 0.116 [ -0.6% .. +0.8%]
wall_time 8.291 ± 0.174 8.307 ± 0.183 [ -1.1% .. +1.5%]
samples 21 25

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D129830

show more ...

[lld-macho] Handle user-provided dtrace symbols to avoid linking failure

This fixes https://github.com/llvm/llvm-project/issues/56238. ld64.lld currently does not generate __dof section in Mach-O, a

[lld-macho] Handle user-provided dtrace symbols to avoid linking failure

This fixes https://github.com/llvm/llvm-project/issues/56238. ld64.lld currently does not generate __dof section in Mach-O, and -no_dtrace_dof option is on by default. However when there are user-defined dtrace symbols, ld64.lld will treat them as undefined symbols, which causes the linking to fail because lld cannot find their definitions. This patch allows ld64.lld to rewrite the instructions calling dtrace symbols to instructions like nop as what ld64 does; therefore, when encountered with user-provided dtrace probes, the linking can still succeed.

I'm not sure whether support for dtrace is expected in lld, so for now I didn't add codes to make lld emit __dof section like ld64, and only made it possible to link with dtrace symbols provided. If this feature is needed, I can add that part in Dtrace.cpp & Dtrace.h.

Reviewed By: int3, #lld-macho

Differential Revision: https://reviews.llvm.org/D129062

show more ...

[lld-macho] Initial support for Linker Optimization Hints

Linker optimization hints mark a sequence of instructions used for
synthesizing an address, like ADRP+ADD. If the referenced symbol ends up

[lld-macho] Initial support for Linker Optimization Hints

Linker optimization hints mark a sequence of instructions used for
synthesizing an address, like ADRP+ADD. If the referenced symbol ends up
close enough, it can be replaced by a faster sequence of instructions
like ADR+NOP.

This commit adds support for 2 of the 7 defined ARM64 optimization
hints:
- LOH_ARM64_ADRP_ADD, which transforms a pair of ADRP+ADD into ADR+NOP
if the referenced address is within +/- 1 MiB
- LOH_ARM64_ADRP_ADRP, which transforms two ADRP instructions into
ADR+NOP if they reference the same page

These two kinds already cover more than 50% of all LOHs in
chromium_framework.

Differential Review: https://reviews.llvm.org/D128093

show more ...

[lld-macho] Use source information in duplicate symbol errors

Similarly to how undefined symbol diagnostics were changed in D128184,
we now show where in the source file duplicate symbols are define

[lld-macho] Use source information in duplicate symbol errors

Similarly to how undefined symbol diagnostics were changed in D128184,
we now show where in the source file duplicate symbols are defined at:

ld64.lld: error: duplicate symbol: _foo
>> defined in bar.c:42
>> /path/to/bar.o
>> defined in baz.c:1
>> /path/to/libbaz.a(baz.o)

For objects that don't contain DWARF data, the format is unchanged.

A slight difference to undefined symbol diagnostics is that we don't
print the name of the symbol on the third line, as it's already
contained on the first line.

Differential Revision: https://reviews.llvm.org/D128425

show more ...

Reland "[lld-macho] Show source information for undefined references"

The error used to look like this:

ld64.lld: error: undefined symbol: _foo
>>> referenced by /path/to/bar.o:(symbol _baz+0x4

Reland "[lld-macho] Show source information for undefined references"

The error used to look like this:

ld64.lld: error: undefined symbol: _foo
>>> referenced by /path/to/bar.o:(symbol _baz+0x4)

If DWARF line information is available, we now show where in the source
the references are coming from:

ld64.lld: error: unreferenced symbol: _foo
>>> referenced by: bar.cpp:42 (/path/to/bar.cpp:42)
>>> /path/to/bar.o:(symbol _baz+0x4)

The reland is identical to the first time this landed. The fix was in D128294.
This reverts commit 0cc7ad417585b3185c32e395cc5e6cf082a347af.

Differential Revision: https://reviews.llvm.org/D128184

show more ...

Revert "[lld-macho] Show source information for undefined references"

This reverts commit cd7624f15369f0d395c1edee1a0b9592083d2fe0.
See https://reviews.llvm.org/D128184#3597534

[lld-macho] Show source information for undefined references

The error used to look like this:

ld64.lld: error: undefined symbol: _foo
>>> referenced by /path/to/bar.o:(symbol _baz+0x4)

If DWA

[lld-macho] Show source information for undefined references

The error used to look like this:

ld64.lld: error: undefined symbol: _foo
>>> referenced by /path/to/bar.o:(symbol _baz+0x4)

If DWARF line information is available, we now show where in the source
the references are coming from:

ld64.lld: error: unreferenced symbol: _foo
>>> referenced by: bar.cpp:42 (/path/to/bar.cpp:42)
>>> /path/to/bar.o:(symbol _baz+0x4)

Differential Revision: https://reviews.llvm.org/D128184

show more ...

[lld-macho] Fix symbol name returned from InputSection::getLocation

This commit fixes the issue that getLocation always printed the name of
the first symbol in the section.

For clarity, upper_bound

[lld-macho] Fix symbol name returned from InputSection::getLocation

This commit fixes the issue that getLocation always printed the name of
the first symbol in the section.

For clarity, upper_bound is used instead of a linear search for finding
the closest symbol name. Note that this change does not affect
performance: this function is only called when printing errors and
`symbols` typically contains a single symbol because of
.subsections_via_symbols.

Differential Revision: https://reviews.llvm.org/D127670

show more ...

[lld-macho][reland] Initial support for EH Frames

This reverts commit 942f4e3a7cc9a9f8b2654817cff12907d1276031.

The additional change required to avoid the assertion errors seen
previously is:

-

[lld-macho][reland] Initial support for EH Frames

This reverts commit 942f4e3a7cc9a9f8b2654817cff12907d1276031.

The additional change required to avoid the assertion errors seen
previously is:

--- a/lld/MachO/ICF.cpp
+++ b/lld/MachO/ICF.cpp
@@ -443,7 +443,9 @@ void macho::foldIdenticalSections() {
/*relocVA=*/0);
isec->data = copy;
}
- } else {
+ } else if (!isEhFrameSection(isec)) {
+ // EH frames are gathered as hashables from unwindEntry above; give a
+ // unique ID to everything else.
isec->icfEqClass[0] = ++icfUniqueID;
}
}

Differential Revision: https://reviews.llvm.org/D123435

show more ...

Revert "[lld-macho] Initial support for EH Frames"

This reverts commit 826be330af9c0a8553a5b32718ecd2d97e10438e.

This was causing a test failure on build bots:
- https://lab.llvm.org/buildbot/#/b

Revert "[lld-macho] Initial support for EH Frames"

This reverts commit 826be330af9c0a8553a5b32718ecd2d97e10438e.

This was causing a test failure on build bots:
- https://lab.llvm.org/buildbot/#/builders/36/builds/21770
- https://lab.llvm.org/buildbot/#/builders/58/builds/23913

show more ...

[lld-macho] Initial support for EH Frames

== Background ==

`llvm-mc` generates unwind info in both compact unwind and DWARF
formats. LLD already handles the compact unwind format; this diff gets
us

[lld-macho] Initial support for EH Frames

== Background ==

`llvm-mc` generates unwind info in both compact unwind and DWARF
formats. LLD already handles the compact unwind format; this diff gets
us close to handling the DWARF format properly.

== Caveats ==

It's not quite done yet, but I figure it's worth getting this reviewed
and landed first as it's shaping up to be a fairly large code change.

**Known limitations of the current code:**

* Only works for x86_64, for which `llvm-mc` emits "abs-ified"
relocations as described in https://github.com/llvm/llvm-project/commit/618def651b59bd42c05bbd91d825af2fb2145683.
`llvm-mc` emits regular relocations for ARM EH frames, which we do not
yet handle correctly.

Since the feature is not ready for real use yet, I've gated it behind a
flag that only gets toggled on during test suite runs. With most of the
new code disabled, we see just a hint of perf regression, so I don't
think it'd be remiss to land this as-is:

base diff difference (95% CI)
sys_time 1.926 ± 0.168 1.979 ± 0.117 [ -1.2% .. +6.6%]
user_time 3.590 ± 0.033 3.606 ± 0.028 [ +0.0% .. +0.9%]
wall_time 7.104 ± 0.184 7.179 ± 0.151 [ -0.2% .. +2.3%]
samples 30 31

== Design ==

Like compact unwind entries, EH frames are also represented as regular
ConcatInputSections that get pointed to via `Defined::unwindEntry`. This
allows them to be handled generically by e.g. the MarkLive and ICF
code. (But note that unlike compact unwind subsections, EH frame
subsections do end up in the final binary.)

In order to make EH frames "look like" a regular ConcatInputSection,
some processing is required. First, we need to split the `__eh_frame`
section along EH frame boundaries rather than along symbol boundaries.
We do this by decoding the length field of each EH frame. Second, the
abs-ified relocations need to be turned into regular Relocs.

== Next Steps ==

In order to support EH frames on ARM targets, we will either have to
teach LLD how to handle EH frames with explicit relocs, or we can try to
make `llvm-mc` emit abs-ified relocs for ARM as well. I'm hoping to do
the latter as I think it will make the LLD implementation both simpler
and faster to execute.

== Misc ==

The `obj-file-with-stabs.s` test had to be updated as the previous
version would trip assertion errors in the code. It appears that in our
attempt to produce a minimal YAML test input, we created a file with
invalid EH frame data. I've fixed this by re-generating the YAML and not
doing any hand-pruning of it.

Reviewed By: #lld-macho, Roger

Differential Revision: https://reviews.llvm.org/D123435

show more ...

[lld/mac] Don't emit stabs entries for functions folded during ICF

This matches ld64, and makes dsymutil work better with lld's output.
Fixes PR54783, see there for details.

Reduces time needed to

[lld/mac] Don't emit stabs entries for functions folded during ICF

This matches ld64, and makes dsymutil work better with lld's output.
Fixes PR54783, see there for details.

Reduces time needed to run dsymutil on Chromium Framework from 8m30s
(which is already down from 26 min with D123218) to 6m30s and removes
many lines of "could not find object file symbol for symbol" from dsymutil output
(previously: several MB of those messages, now dsymutil is completely silent).

Differential Revision: https://reviews.llvm.org/D123252

show more ...

[lld-macho] Deduplicate the `__objc_classrefs` section contents

ld64 breaks down `__objc_classrefs` on a per-word level and deduplicates
them. This greatly reduces the number of bind entries emitted

[lld-macho] Deduplicate the `__objc_classrefs` section contents

ld64 breaks down `__objc_classrefs` on a per-word level and deduplicates
them. This greatly reduces the number of bind entries emitted (and
therefore the amount of work `dyld` has to do at runtime). For
chromium_framework, this change to LLD cuts the number of (non-lazy)
binds from 912 to 190, getting us to parity with ld64 in this aspect.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D121053

show more ...

[lld-macho][nfc] Reduce size of icfEqClass hash

... from a `uint64_t` to a `uint32_t`. (LLD-ELF uses a `uint32_t` too.)

About a 1.7% reduction in peak RSS when linking chromium_framework on my
3.2

[lld-macho][nfc] Reduce size of icfEqClass hash

... from a `uint64_t` to a `uint32_t`. (LLD-ELF uses a `uint32_t` too.)

About a 1.7% reduction in peak RSS when linking chromium_framework on my
3.2 GHz 16-Core Intel Xeon W Mac Pro, and no stat sig change in wall
time.

</Users/jezng/test2.sh ["before"]> </Users/jezng/test2.sh ["after"]> difference (95% CI)
RSS 1003036672.000 ± 9891065.259 985539505.231 ± 10272748.749 [ -2.3% .. -1.2%]
samples 27 26

base diff difference (95% CI)
sys_time 1.277 ± 0.023 1.277 ± 0.024 [ -0.9% .. +0.9%]
user_time 6.682 ± 0.046 6.598 ± 0.043 [ -1.6% .. -0.9%]
wall_time 5.904 ± 0.062 5.895 ± 0.063 [ -0.7% .. +0.4%]
samples 46 28

No appreciable change (~0.01%) in number of `equals` comparisons either:

Before:

ld64.lld: ICF needed 8 iterations
ld64.lld: equalsConstant() called 701643 times
ld64.lld: equalsVariable() called 3438526 times

After:

ld64.lld: ICF needed 8 iterations
ld64.lld: equalsConstant() called 701729 times
ld64.lld: equalsVariable() called 3438526 times

Reviewed By: #lld-macho, MaskRay, thakis

Differential Revision: https://reviews.llvm.org/D121052

show more ...

[lld-macho][nfc] Move ICF-specific logic into ICF.cpp

This mirrors the code organization in `lld/ELF`.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D120378

[lld-macho] Include address offsets in error messages

This makes it easier to pinpoint the source of the problem.

TODO: Have more relocation error messages make use of this
functionality.

Reviewed

[lld-macho] Include address offsets in error messages

This makes it easier to pinpoint the source of the problem.

TODO: Have more relocation error messages make use of this
functionality.

Reviewed By: #lld-macho, oontvoo

Differential Revision: https://reviews.llvm.org/D118798

show more ...

[lld-macho][nfc] Eliminate InputSection::Shared

Earlier in LLD's evolution, I tried to create the illusion that
subsections were indistinguishable from "top-level" sections. Thus, even
though the su

[lld-macho][nfc] Eliminate InputSection::Shared

Earlier in LLD's evolution, I tried to create the illusion that
subsections were indistinguishable from "top-level" sections. Thus, even
though the subsections shared many common field values, I hid those
common values away in a private Shared struct (see D105305). More
recently, however, @gkm added a public `Section` struct in D113241 that
served as an explicit way to store values that are common to an entire
set of subsections (aka InputSections). Now that we have another "common
value" struct, `Shared` has been rendered redundant. All its fields can
be moved into `Section` instead, and the pointer to `Shared` can be replaced
with a pointer to `Section`.

This `Section` pointer also has the advantage of letting us inspect other
subsections easily, simplifying the implementation of {D118798}.

P.S. I do think that having both `Section` and `InputSection` makes for
a slightly confusing naming scheme. I considered renaming `InputSection`
to `Subsection`, but that would break the symmetry with `OutputSection`.
It would also make us deviate from LLD-ELF's naming scheme.

This change is perf-neutral on my 3.2 GHz 16-Core Intel Xeon W machine:

base diff difference (95% CI)
sys_time 1.258 ± 0.031 1.248 ± 0.023 [ -1.6% .. +0.1%]
user_time 3.659 ± 0.047 3.658 ± 0.041 [ -0.5% .. +0.4%]
wall_time 4.640 ± 0.085 4.625 ± 0.063 [ -1.0% .. +0.3%]
samples 49 61

There's also no stat sig change in RSS (as measured by `time -l`):

base diff difference (95% CI)
time 998038627.097 ± 13567305.958 1003327715.556 ± 15210451.236 [ -0.2% .. +1.2%]
samples 31 36

Reviewed By: #lld-macho, oontvoo

Differential Revision: https://reviews.llvm.org/D118797

show more ...

[MachO] Fix struct size assertion

std::vector can have different sizes depending on the STL's debug level,
so account for its size separately. (You could argue that we should be
accounting for all t

[MachO] Fix struct size assertion

std::vector can have different sizes depending on the STL's debug level,
so account for its size separately. (You could argue that we should be
accounting for all the other members separately as well, but that would
be very unergonomic, and std::vector is the only one that's caused
problems so far.)

show more ...

[lld-macho][nfc] Factor-out NFC changes from main __eh_frame diff

In order to keep signal:noise high for the `__eh_frame` diff, I have teased-out the NFC changes and put them here.

Differential Rev

[lld-macho][nfc] Factor-out NFC changes from main __eh_frame diff

In order to keep signal:noise high for the `__eh_frame` diff, I have teased-out the NFC changes and put them here.

Differential Revision: https://reviews.llvm.org/D114017

show more ...

[MachO] Move type size asserts to source files. NFC

As discussed in https://reviews.llvm.org/D113809#3128636. It's a bit
unfortunate to move the asserts away from the structs whose sizes
they're che

[MachO] Move type size asserts to source files. NFC

As discussed in https://reviews.llvm.org/D113809#3128636. It's a bit
unfortunate to move the asserts away from the structs whose sizes
they're checking, but it's a far better developer experience when one of
the asserts is violated, because you get a single error instead of every
single source file including the header erroring out.

show more ...

[lld-macho][nfc][cleanup] Fix a few code style lints and clang-tidy findings

- Use .empty() instead of `size() == 0` when possible.
- Use const-ref to avoid copying

Differential Revision: https://r

[lld-macho][nfc][cleanup] Fix a few code style lints and clang-tidy findings

- Use .empty() instead of `size() == 0` when possible.
- Use const-ref to avoid copying

Differential Revision: https://reviews.llvm.org/D112978

show more ...

[lld/mac] Don't assert when ICFing arm64 code

WordLiteralSection dedupes literals by content.
WordLiteralInputSection::getOffset() used to read a literal at the passed-in
offset and look up this val

[lld/mac] Don't assert when ICFing arm64 code

WordLiteralSection dedupes literals by content.
WordLiteralInputSection::getOffset() used to read a literal at the passed-in
offset and look up this value in the deduping map to find the offset of the
deduped value.

But it's possible that (e.g.) a 16-byte literal's value is accessed 4 bytes in.
To get the offset at that address, we have to get the deduped value at offset 0
and then apply the offset 4 to the result.

(See also WordLiteralSection::finalizeContents() which fills in those maps.)

Only a problem on arm64 because in x86_64 the offset is part of the instruction
instead of a separate ARM64_RELOC_ADDEND relocation. (See bug for more details.)

Fixes PR51999.

Differential Revision: https://reviews.llvm.org/D112584

show more ...

[lld-macho] Associate compact unwind entries with function symbols

Compact unwind entries (CUEs) contain pointers to their respective
function symbols. However, during the link process, it's far mor

[lld-macho] Associate compact unwind entries with function symbols

Compact unwind entries (CUEs) contain pointers to their respective
function symbols. However, during the link process, it's far more useful
to have pointers from the function symbol to the CUE than vice versa.
This diff adds that pointer in the form of `Defined::compactUnwind`.

In particular, when doing dead-stripping, we want to mark CUEs live when
their function symbol is live; and when doing ICF, we want to dedup
sections iff the symbols in that section have identical CUEs. In both
cases, we want to be able to locate the symbols within a given section,
as well as locate the CUEs belonging to those symbols. So this diff also
adds `InputSection::symbols`.

The ultimate goal of this refactor is to have ICF support dedup'ing
functions with unwind info, but that will be handled in subsequent
diffs. This diff focuses on simplifying `-dead_strip` --
`findFunctionsWithUnwindInfo` is no longer necessary, and
`Defined::isLive()` is now a lot simpler. Moreover, UnwindInfoSection no
longer has to check for dead CUEs -- we simply avoid adding them in the
first place.

Additionally, we now support stripping of dead LSDAs, which follows
quite naturally since `markLive()` can now reach them via the CUEs.

Reviewed By: #lld-macho, gkm

Differential Revision: https://reviews.llvm.org/D109944

show more ...

[lld/mac] Remove "else" after return

No behavior change