[lld-macho] Canonicalize personality pointers in EH frames

We already do this for personality pointers referenced from compact
unwind entries; this patch extends that behavior to personalities
refer

[lld-macho] Canonicalize personality pointers in EH frames

We already do this for personality pointers referenced from compact
unwind entries; this patch extends that behavior to personalities
referenced via EH frames as well.

This reduces the number of distinct personalities we need in the final
binary, and helps us avoid hitting the "too many personalities" error.

I renamed `UnwindInfoSection::prepareRelocations()` to simply `prepare`
since we now do some non-reloc-specific stuff within.

Fixes #58277.

Reviewed By: #lld-macho, oontvoo

Differential Revision: https://reviews.llvm.org/D135728

(cherry picked from commit 7b45dfc6811a52ff4e9a6054dc276d70d77fddaf)

show more ...

[lld/mac] Add support for $ld$previous symbols with explicit symbol name

A symbol `$ld$previous$/Another$1.2.3$1$3.0$14.0$_xxx$` means
"pretend symbol `_xxx` is in dylib `/Another` with version `1.2

[lld/mac] Add support for $ld$previous symbols with explicit symbol name

A symbol `$ld$previous$/Another$1.2.3$1$3.0$14.0$_xxx$` means
"pretend symbol `_xxx` is in dylib `/Another` with version `1.2.3`
if the deployment target is between `3.0` and `14.0` and we're
targeting platform `1` (ie macOS)".

This means dylibs can now inject synthetic dylibs into the link, so
DylibFile needs to grow a 3rd constructor.

The only other interesting thing is that such an injected dylib
counts as a use of the original dylib. This patch gets this mostly
right (if _only_ `$ld$previous` symbols are used from a dylib,
we don't add a dep on the dylib itself, matching ld64), but one case
where we don't match ld64 yet is that ld64 even omits the original
dylib when linking it with `-needed-l`. Lld currently still adds a load
command for the original dylib in that case. (That's for a future
patch.)

Fixes #56074.

Differential Revision: https://reviews.llvm.org/D130725

(cherry picked from commit 241f0e8b76d544a4a07a7f775b8421632539be19)

show more ...

[lld-macho] Fold __objc_imageinfo sections

Previously, we treated it as a regular ConcatInputSection. However, ld64
actually parses its contents and uses that to synthesize a single image
info struc

[lld-macho] Fold __objc_imageinfo sections

Previously, we treated it as a regular ConcatInputSection. However, ld64
actually parses its contents and uses that to synthesize a single image
info struct, generating one 8-byte section instead of `8 * number of
object files with ObjC code`.

I'm not entirely sure what impact this section has on the runtime, so I
just tried to follow ld64's semantics as closely as possible in this
diff. My main motivation though was to reduce binary size.

No significant perf change on chromium_framework on my 16-core Mac Pro:

base diff difference (95% CI)
sys_time 1.764 ± 0.062 1.748 ± 0.032 [ -2.4% .. +0.5%]
user_time 5.112 ± 0.104 5.106 ± 0.046 [ -0.9% .. +0.7%]
wall_time 6.111 ± 0.184 6.085 ± 0.076 [ -1.6% .. +0.8%]
samples 30 32

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D130125

show more ...

[lld/mac] Add a few TimeTraceScopes

Identical literal folding takes ~1.4% of the time, and was missing
from the trace.

Signature computation still needs ~2.2% of the time, so probably worth
explici

[lld/mac] Add a few TimeTraceScopes

Identical literal folding takes ~1.4% of the time, and was missing
from the trace.

Signature computation still needs ~2.2% of the time, so probably worth
explicitly marking its contribution to "Write output file" (9.1%)

Differential Revision: https://reviews.llvm.org/D128343

show more ...

Reland "[lld-macho] Group undefined symbol diagnostics by symbol".

This reverts commit 36e7c9a450db5e22af1ec21412d918ceb2313942.

This relands d61341768cf0cff7c with the fix described in
https://rev

Reland "[lld-macho] Group undefined symbol diagnostics by symbol".

This reverts commit 36e7c9a450db5e22af1ec21412d918ceb2313942.

This relands d61341768cf0cff7c with the fix described in
https://reviews.llvm.org/D127753#3587390

show more ...

Revert "[lld-macho] Group undefined symbol diagnostics by symbol"

This reverts commit d61341768cf0cff7ceeaddecc2f769b5c1b901c4.

This change broke multiple lld tests, including some sanitizer builds

Revert "[lld-macho] Group undefined symbol diagnostics by symbol"

This reverts commit d61341768cf0cff7ceeaddecc2f769b5c1b901c4.

This change broke multiple lld tests, including some sanitizer builds: https://lab.llvm.org/buildbot/#/builders/5/builds/24787/steps/19/logs/stdio

show more ...

[lld-macho] Group undefined symbol diagnostics by symbol

ld64.lld used to print the "undefined symbol" line for each reference to
an undefined symbol previously:

ld64.lld: error: undefined symbol

[lld-macho] Group undefined symbol diagnostics by symbol

ld64.lld used to print the "undefined symbol" line for each reference to
an undefined symbol previously:

ld64.lld: error: undefined symbol: _foo
>>> referenced by /path/to/bar.o:(symbol _baz+0x0)

ld64.lld: error: undefined symbol: _foo
>>> referenced by /path/to/bar.o:(symbol _quux+0x1)

Now they are deduplicated:

ld64.lld: error: undefined symbol: _foo
>>> referenced by /path/to/bar.o:(symbol _baz+0x0)
>>> referenced by /path/to/bar.o:(symbol _quux+0x1)

As with the other lld ports, only the first 3 references are printed.

Differential Revision: https://reviews.llvm.org/D127753

show more ...

[lld-macho] Print the name of functions containing undefined references

The error used to look like this:

ld64.lld: error: undefined symbol: _foo
>>> referenced by /path/to/bar.o

Now it displa

[lld-macho] Print the name of functions containing undefined references

The error used to look like this:

ld64.lld: error: undefined symbol: _foo
>>> referenced by /path/to/bar.o

Now it displays the name of the function that contains the undefined
reference as well:

ld64.lld: error: undefined symbol: _foo
>>> referenced by /path/to/bar.o:(symbol _baz+0x4)

Differential Revision: https://reviews.llvm.org/D127696

show more ...

[lld-macho][reland] Initial support for EH Frames

This reverts commit 942f4e3a7cc9a9f8b2654817cff12907d1276031.

The additional change required to avoid the assertion errors seen
previously is:

-

[lld-macho][reland] Initial support for EH Frames

This reverts commit 942f4e3a7cc9a9f8b2654817cff12907d1276031.

The additional change required to avoid the assertion errors seen
previously is:

--- a/lld/MachO/ICF.cpp
+++ b/lld/MachO/ICF.cpp
@@ -443,7 +443,9 @@ void macho::foldIdenticalSections() {
/*relocVA=*/0);
isec->data = copy;
}
- } else {
+ } else if (!isEhFrameSection(isec)) {
+ // EH frames are gathered as hashables from unwindEntry above; give a
+ // unique ID to everything else.
isec->icfEqClass[0] = ++icfUniqueID;
}
}

Differential Revision: https://reviews.llvm.org/D123435

show more ...

Revert "[lld-macho] Initial support for EH Frames"

This reverts commit 826be330af9c0a8553a5b32718ecd2d97e10438e.

This was causing a test failure on build bots:
- https://lab.llvm.org/buildbot/#/b

Revert "[lld-macho] Initial support for EH Frames"

This reverts commit 826be330af9c0a8553a5b32718ecd2d97e10438e.

This was causing a test failure on build bots:
- https://lab.llvm.org/buildbot/#/builders/36/builds/21770
- https://lab.llvm.org/buildbot/#/builders/58/builds/23913

show more ...

[lld-macho] Initial support for EH Frames

== Background ==

`llvm-mc` generates unwind info in both compact unwind and DWARF
formats. LLD already handles the compact unwind format; this diff gets
us

[lld-macho] Initial support for EH Frames

== Background ==

`llvm-mc` generates unwind info in both compact unwind and DWARF
formats. LLD already handles the compact unwind format; this diff gets
us close to handling the DWARF format properly.

== Caveats ==

It's not quite done yet, but I figure it's worth getting this reviewed
and landed first as it's shaping up to be a fairly large code change.

**Known limitations of the current code:**

* Only works for x86_64, for which `llvm-mc` emits "abs-ified"
relocations as described in https://github.com/llvm/llvm-project/commit/618def651b59bd42c05bbd91d825af2fb2145683.
`llvm-mc` emits regular relocations for ARM EH frames, which we do not
yet handle correctly.

Since the feature is not ready for real use yet, I've gated it behind a
flag that only gets toggled on during test suite runs. With most of the
new code disabled, we see just a hint of perf regression, so I don't
think it'd be remiss to land this as-is:

base diff difference (95% CI)
sys_time 1.926 ± 0.168 1.979 ± 0.117 [ -1.2% .. +6.6%]
user_time 3.590 ± 0.033 3.606 ± 0.028 [ +0.0% .. +0.9%]
wall_time 7.104 ± 0.184 7.179 ± 0.151 [ -0.2% .. +2.3%]
samples 30 31

== Design ==

Like compact unwind entries, EH frames are also represented as regular
ConcatInputSections that get pointed to via `Defined::unwindEntry`. This
allows them to be handled generically by e.g. the MarkLive and ICF
code. (But note that unlike compact unwind subsections, EH frame
subsections do end up in the final binary.)

In order to make EH frames "look like" a regular ConcatInputSection,
some processing is required. First, we need to split the `__eh_frame`
section along EH frame boundaries rather than along symbol boundaries.
We do this by decoding the length field of each EH frame. Second, the
abs-ified relocations need to be turned into regular Relocs.

== Next Steps ==

In order to support EH frames on ARM targets, we will either have to
teach LLD how to handle EH frames with explicit relocs, or we can try to
make `llvm-mc` emit abs-ified relocs for ARM as well. I'm hoping to do
the latter as I think it will make the LLD implementation both simpler
and faster to execute.

== Misc ==

The `obj-file-with-stabs.s` test had to be updated as the previous
version would trip assertion errors in the code. It appears that in our
attempt to produce a minimal YAML test input, we created a file with
invalid EH frame data. I've fixed this by re-generating the YAML and not
doing any hand-pruning of it.

Reviewed By: #lld-macho, Roger

Differential Revision: https://reviews.llvm.org/D123435

show more ...

[lld/mac] Write output sections in parallel

This reduces linking time by ~8% for my project (1.19s -> 0.53s for
writeSections()). writeTo is const, which bodes well for it being
parallelizable, and

[lld/mac] Write output sections in parallel

This reduces linking time by ~8% for my project (1.19s -> 0.53s for
writeSections()). writeTo is const, which bodes well for it being
parallelizable, and I've looked through the different overridden versions and
can't see any race conditions. It produces the same byte-for-byte output for my
project.

Differential Revision: https://reviews.llvm.org/D126800

show more ...

[lld-macho][nfc] Run clang-format on lld/MachO/*.{h,cpp}

- fixed inconsistent indents and spaces
- prevent extraneous formatting changes in other patches

Differential Revision: https://reviews.llvm

[lld-macho][nfc] Run clang-format on lld/MachO/*.{h,cpp}

- fixed inconsistent indents and spaces
- prevent extraneous formatting changes in other patches

Differential Revision: https://reviews.llvm.org/D126262

show more ...

[lld/mac] Support writing zippered dylibs and bundles

With -platform_version flags for two distinct platforms,
this writes a LC_BUILD_VERSION header for each.

The motivation is that this is needed

[lld/mac] Support writing zippered dylibs and bundles

With -platform_version flags for two distinct platforms,
this writes a LC_BUILD_VERSION header for each.

The motivation is that this is needed for self-hosting with lld as linker
after D124059.

To create a zippered output at the clang driver level, pass

-target arm64-apple-macos -darwin-target-variant arm64-apple-ios-macabi

to create a zippered dylib.

(In Xcode's clang, `-darwin-target-variant` is spelled just `-target-variant`.)

(If you pass `-target arm64-apple-ios-macabi -target-variant arm64-apple-macos`
instead, ld64 crashes!)

This results in two -platform_version flags being passed to the linker.

ld64 also verifies that the iOS SDK version is at least 13.1. We don't do that
yet. But ld64 also does that for other platforms and we don't. So we need to
do that at some point, but not in this patch.

Only dylib and bundle outputs can be zippered.

I verified that a Catalyst app linked against a dylib created with

clang -shared foo.cc -o libfoo.dylib \
-target arm64-apple-macos \
-target-variant arm64-apple-ios-macabi \
-Wl,-install_name,@rpath/libfoo.dylib \
-fuse-ld=$PWD/out/gn/bin/ld64.lld

runs successfully. (The app calls a function `f()` in libfoo.dylib
that returns a const char* "foo", and NSLog(@"%s")s it.)

ld64 is a bit more permissive when writing zippered outputs,
see references to "unzippered twins". That's not implemented yet.
(If anybody wants to implement that, D124275 is a good start.)

Differential Revision: https://reviews.llvm.org/D124887

show more ...

[lld-macho][nfc] Give non-text ConcatOutputSections order-independent finalization

This diff is motivated by my work to add proper DWARF unwind support. As
detailed in PR50956 functions that need DW

[lld-macho][nfc] Give non-text ConcatOutputSections order-independent finalization

This diff is motivated by my work to add proper DWARF unwind support. As
detailed in PR50956 functions that need DWARF unwind need to have
compact unwind entries synthesized for them. These CU entries encode an
offset within `__eh_frame` that points to the corresponding DWARF FDE.

In order to encode this offset during
`UnwindInfoSectionImpl::finalize()`, we need to first assign values to
`InputSection::outSecOff` for each `__eh_frame` subsection. But
`__eh_frame` is ordered after `__unwind_info` (according to ld64 at
least), which puts us in a bit of a bind: `outSecOff` gets assigned
during finalization, but `__eh_frame` is being finalized after
`__unwind_info`.

But it occurred to me that there's no real need for most
ConcatOutputSections to be finalized sequentially. It's only necessary
for text-containing ConcatOutputSections that may contain branch relocs
which may need thunks. ConcatOutputSections containing other types of
data can be finalized in any order.

This diff moves the finalization logic for non-text sections into a
separate `finalizeContents()` method. This method is called before
section address assignment & unwind info finalization takes place. In
theory we could call these `finalizeContents()` methods in parallel, but
in practice it seems to be faster to do it all on the main thread.

Reviewed By: #lld-macho, oontvoo

Differential Revision: https://reviews.llvm.org/D123279

show more ...

Add output filename to UUID hash

Differential Revision: https://reviews.llvm.org/D122843

[lld][Macho][NFC] Encapsulate priorities map in a priority class

`config->priorities` has been used to hold the intermediate state during the construction of the order in which sections should be la

[lld][Macho][NFC] Encapsulate priorities map in a priority class

`config->priorities` has been used to hold the intermediate state during the construction of the order in which sections should be laid out. This is not a good place to hold this state since the intermediate state is not a "configuration" for LLD. It should be encapsulated in a class for building a mapping from section to priority (which I created in this diff as the `PriorityBuilder` class).

The same thing is being done for `config->callGraphProfile`.

Reviewed By: #lld-macho, int3

Differential Revision: https://reviews.llvm.org/D122156

show more ...

[lld-macho] -flat_namespace for dylibs should make all externs interposable

All references to interposable symbols can be redirected at runtime to
point to a different symbol definition (with the sa

[lld-macho] -flat_namespace for dylibs should make all externs interposable

All references to interposable symbols can be redirected at runtime to
point to a different symbol definition (with the same name). For
example, if both dylib A and B define symbol _foo, and we load A before
B at runtime, then all references to _foo within dylib B will point to
the definition in dylib A.

ld64 makes all extern symbols interposable when linking with
`-flat_namespace`.

TODO 1: Support `-interposable` and `-interposable_list`, which should
just be a matter of parsing those CLI flags and setting the
`Defined::interposable` bit.

TODO 2: Set Reloc::FinalDefinitionInLinkageUnit correctly with this info
(we are currently not setting it at all, so we're erring on the
conservative side, but we should help the LTO backend generate more
optimal code.)

Reviewed By: modimo, MaskRay

Differential Revision: https://reviews.llvm.org/D119294

show more ...

[lld-macho] Have relocation address included in range-check error message

This makes it easier to debug those errors. See e.g. https://github.com/llvm/llvm-project/issues/52767#issuecomment-10287139

[lld-macho] Have relocation address included in range-check error message

This makes it easier to debug those errors. See e.g. https://github.com/llvm/llvm-project/issues/52767#issuecomment-1028713943

We take the approach of 'reverse-engineering' the InputSection from the
output buffer offset. This provides for a cleaner Target API, and is
similar to LLD-ELF's implementation of getErrorPlace().

Reviewed By: #lld-macho, Roger

Differential Revision: https://reviews.llvm.org/D118903

show more ...

[lld-macho][nfc] Eliminate InputSection::Shared

Earlier in LLD's evolution, I tried to create the illusion that
subsections were indistinguishable from "top-level" sections. Thus, even
though the su

[lld-macho][nfc] Eliminate InputSection::Shared

Earlier in LLD's evolution, I tried to create the illusion that
subsections were indistinguishable from "top-level" sections. Thus, even
though the subsections shared many common field values, I hid those
common values away in a private Shared struct (see D105305). More
recently, however, @gkm added a public `Section` struct in D113241 that
served as an explicit way to store values that are common to an entire
set of subsections (aka InputSections). Now that we have another "common
value" struct, `Shared` has been rendered redundant. All its fields can
be moved into `Section` instead, and the pointer to `Shared` can be replaced
with a pointer to `Section`.

This `Section` pointer also has the advantage of letting us inspect other
subsections easily, simplifying the implementation of {D118798}.

P.S. I do think that having both `Section` and `InputSection` makes for
a slightly confusing naming scheme. I considered renaming `InputSection`
to `Subsection`, but that would break the symmetry with `OutputSection`.
It would also make us deviate from LLD-ELF's naming scheme.

This change is perf-neutral on my 3.2 GHz 16-Core Intel Xeon W machine:

base diff difference (95% CI)
sys_time 1.258 ± 0.031 1.248 ± 0.023 [ -1.6% .. +0.1%]
user_time 3.659 ± 0.047 3.658 ± 0.041 [ -0.5% .. +0.4%]
wall_time 4.640 ± 0.085 4.625 ± 0.063 [ -1.0% .. +0.3%]
samples 49 61

There's also no stat sig change in RSS (as measured by `time -l`):

base diff difference (95% CI)
time 998038627.097 ± 13567305.958 1003327715.556 ± 15210451.236 [ -0.2% .. +1.2%]
samples 31 36

Reviewed By: #lld-macho, oontvoo

Differential Revision: https://reviews.llvm.org/D118797

show more ...

[lld-macho][nfc] Comments and style fixes

Added some comments (particularly around finalize() and
finalizeContents()) as well as doing some rephrasing / grammar fixes for
existing comments.

Also di

[lld-macho][nfc] Comments and style fixes

Added some comments (particularly around finalize() and
finalizeContents()) as well as doing some rephrasing / grammar fixes for
existing comments.

Also did some minor style fixups, such as by putting methods together in
a class definition and having fields of similar types next to each
other.

Reviewed By: #lld-macho, oontvoo

Differential Revision: https://reviews.llvm.org/D118714

show more ...

[lld-macho] Move order file and call graph sorting into SectionPriorities

See https://reviews.llvm.org/D117354 for context and discussion.

[lld-macho] Rename CallGraphSort.{h,cpp} to SectionPriorities

This is in preparation for moving the code that parses and processes
order files into this file.

See https://reviews.llvm.org/D117354 f

[lld-macho] Rename CallGraphSort.{h,cpp} to SectionPriorities

This is in preparation for moving the code that parses and processes
order files into this file.

See https://reviews.llvm.org/D117354 for context and discussion.

show more ...

Re-land [LLD] Remove global state in lldCommon

Move all variables at file-scope or function-static-scope into a hosting structure (lld::CommonLinkerContext) that lives at lldMain()-scope. Drivers wi

Re-land [LLD] Remove global state in lldCommon

Move all variables at file-scope or function-static-scope into a hosting structure (lld::CommonLinkerContext) that lives at lldMain()-scope. Drivers will inherit from this structure and add their own global state, in the same way as for the existing COFFLinkerContext.

See discussion in https://lists.llvm.org/pipermail/llvm-dev/2021-June/151184.html

The previous land f860fe362282ed69b9d4503a20e5d20b9a041189 caused issues in https://lab.llvm.org/buildbot/#/builders/123/builds/8383, fixed by 22ee510dac9440a74b2e5b3fe3ff13ccdbf55af3.

Differential Revision: https://reviews.llvm.org/D108850

show more ...

Revert [LLD] Remove global state in lldCommon

It seems to be causing issues on https://lab.llvm.org/buildbot/#/builders/123/builds/8383