[JITLink][AArch64] Rename PointerToGOT and fix typo.

PointerToGOT lowering was accidentally changed from Delta32 to Delta64 in
db3722580335c. This patch moves it back to Delta32 and renames the gene

[JITLink][AArch64] Rename PointerToGOT and fix typo.

PointerToGOT lowering was accidentally changed from Delta32 to Delta64 in
db3722580335c. This patch moves it back to Delta32 and renames the generic
aarch64 edge to Delta32ToGOT to avoid the ambiguity.

No test case yet -- I haven't figured out how to write a succinct test case
(this typically appears in CIEs in eh-frames).

show more ...

[JITLink][AArch64] Unify table managers of ELF and MachO.

Unifies GOT/PLT table managers of ELF and MachO on aarch64 architecture. Additionally, it migrates table managers from PerGraphGOTAndPLTStub

[JITLink][AArch64] Unify table managers of ELF and MachO.

Unifies GOT/PLT table managers of ELF and MachO on aarch64 architecture. Additionally, it migrates table managers from PerGraphGOTAndPLTStubsBuilder to generic crtp TableManager.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D127558

show more ...

[JITLink][ELF][AArch64] Lift MachO/arm64 edges into aarch64.h, reuse for ELF.

This patch moves the aarch64 fixup logic from the MachO/arm64 backend to
aarch64.h header so that it can be re-used in t

[JITLink][ELF][AArch64] Lift MachO/arm64 edges into aarch64.h, reuse for ELF.

This patch moves the aarch64 fixup logic from the MachO/arm64 backend to
aarch64.h header so that it can be re-used in the ELF/aarch64 backend. This
significantly expands relocation support in the ELF/aarch64 backend.

Reviewed By: lhames, sgraenitz

Differential Revision: https://reviews.llvm.org/D126286

show more ...

[JITLink] Error instead of asserting on unrecognized edge kinds.

It's idiomatic to require that plugins (especially platform plugins) be
installed to handle special edge kinds. If the plugins are no

[JITLink] Error instead of asserting on unrecognized edge kinds.

It's idiomatic to require that plugins (especially platform plugins) be
installed to handle special edge kinds. If the plugins are not installed and an
object is loaded that uses one of the special edge kinds then we want to error
out rather than asserting.

show more ...

[JITLink] Refactor and expand DWARF pointer encoding support.

Adds support for pointer encodings commonly used in large/static models,
including non-pcrel, sdata/udata8, indirect, and omit.

Also re

[JITLink] Refactor and expand DWARF pointer encoding support.

Adds support for pointer encodings commonly used in large/static models,
including non-pcrel, sdata/udata8, indirect, and omit.

Also refactors pointer-encoding handling to consolidate error generation inside
common functions, rather than callees of those functions.

show more ...

Move DWARFRecordSectionSplitter code to its own file

With 229d576b31f4071ab68c85ac4fabb78cfa502b04 the class EHFrameSplitter was renamed to DWARFRecordSectionSplitter. This change merely moves it to

Move DWARFRecordSectionSplitter code to its own file

With 229d576b31f4071ab68c85ac4fabb78cfa502b04 the class EHFrameSplitter was renamed to DWARFRecordSectionSplitter. This change merely moves it to it's own .cpp/.h file

Differential Revision: https://reviews.llvm.org/D121721

show more ...

Rename EHFrameSplitter to DWARFRecordSectionSplitter

EHFrameSplitter does the exact same work to split up the eh_frame as it would need for any section that follows the DWARF record, therefore this

Rename EHFrameSplitter to DWARFRecordSectionSplitter

EHFrameSplitter does the exact same work to split up the eh_frame as it would need for any section that follows the DWARF record, therefore this patch just changes the name of it to DWARFRecordSectionSplitter to be more general.

Differential Revision: https://reviews.llvm.org/D121486

show more ...

Reapply 253ce92844f "[JITLink][AArch64] Add support for splitting.." with fixes.

This reapplies 253ce92844f72e3a6d0e423473f2765c2c5afd6a, which was reverted in
66b2ed477f902329c916bd4389b2034dae09c0

Reapply 253ce92844f "[JITLink][AArch64] Add support for splitting.." with fixes.

This reapplies 253ce92844f72e3a6d0e423473f2765c2c5afd6a, which was reverted in
66b2ed477f902329c916bd4389b2034dae09c02b due to bot failures.

I have added the `-phony-externals` option added, which should fix the
unresolved symbol errors.

show more ...

Revert "[JITLink][AArch64] Add support for splitting eh-frames on AArch64."

This reverts commit 253ce92844f72e3a6d0e423473f2765c2c5afd6a.
Breaks tests on Windows, see
https://github.com/llvm/llvm-pr

Revert "[JITLink][AArch64] Add support for splitting eh-frames on AArch64."

This reverts commit 253ce92844f72e3a6d0e423473f2765c2c5afd6a.
Breaks tests on Windows, see
https://github.com/llvm/llvm-project/issues/52921#issuecomment-1011118896

show more ...

[JITLink][AArch64] Add support for splitting eh-frames on AArch64.

This is needed for DWARF eh-frame exception handling on AArch64.

https://github.com/llvm/llvm-project/issues/52921.

Original patc

[JITLink][AArch64] Add support for splitting eh-frames on AArch64.

This is needed for DWARF eh-frame exception handling on AArch64.

https://github.com/llvm/llvm-project/issues/52921.

Original patch by David Nadlinger <[email protected]> (thanks David!),
testcase and comments added by me.

show more ...

[JITLink][AArch64] Use R-X permissions for the GOT.

This consistent with ld64's treatment of the GOT, but the main aim here is a
short-term workaround for a bad interaction between stub code sequenc

[JITLink][AArch64] Use R-X permissions for the GOT.

This consistent with ld64's treatment of the GOT, but the main aim here is a
short-term workaround for a bad interaction between stub code sequences and
memory layout: Stubs use LDRLiteral19 relocations to reference the GOT, but
BasicLayout currently puts RW- segments between R-- and R-X segments -- a large
RW- segment (or a large R-- for that matter) can cause the relocation to fail
with an out-of-range error.

Putting the GOT in R-X fixes this efficiently in practice. A more robust fix
will be to use a longer code sequence to materialize the GOT pointer and then
rewrite the stub to use a shorter sequence where possible.

show more ...

Re-apply "[JITLink] Update JITLink to use ExecutorAddr rather... " with fixes.

This re-applies 133f86e95492b2a00b944e070878424cfa73f87c, which was reverted in
c5965a411c635106a47738b8d2e24db822b7416

Re-apply "[JITLink] Update JITLink to use ExecutorAddr rather... " with fixes.

This re-applies 133f86e95492b2a00b944e070878424cfa73f87c, which was reverted in
c5965a411c635106a47738b8d2e24db822b7416f while I investigated bot failures.

The original failure contained an arithmetic conversion think-o (on line 419 of
EHFrameSupport.cpp) that could cause failures on 32-bit platforms. The issue
should be fixed in this patch.

show more ...

Revert "[JITLink] Update JITLink to use ExecutorAddr rather than..."

This reverts commit 133f86e95492b2a00b944e070878424cfa73f87c while I investigate
the bot failures at https://lab.llvm.org/buildbo

Revert "[JITLink] Update JITLink to use ExecutorAddr rather than..."

This reverts commit 133f86e95492b2a00b944e070878424cfa73f87c while I investigate
the bot failures at https://lab.llvm.org/buildbot#builders/186/builds/3370.

show more ...

[JITLink] Update JITLink to use ExecutorAddr rather than JITTargetAddress.

ExecutorAddr is the preferred representation for executor process addresses now.

[NFC][llvm] Inclusive language: remove uses of sanity in llvm/lib/ExecutionEngine/

Reworded and removed code comments to avoid using `sanity check` and `sanity
test`.

[JITLink][MachO] Fix "find-symbol-by-address" logic.

Only search within the requested section, and allow one-past-then-end addresses.

This is needed to support section-end-address references to sec

[JITLink][MachO] Fix "find-symbol-by-address" logic.

Only search within the requested section, and allow one-past-then-end addresses.

This is needed to support section-end-address references to sections with no
symbols in them.

show more ...

[ORC][ORC-RT] Enable the MachO platform for arm64

Enables the arm64 MachO platform, adds basic tests, and implements the
missing TLV relocations and runtime wrapper function. The TLV
relocations are

[ORC][ORC-RT] Enable the MachO platform for arm64

Enables the arm64 MachO platform, adds basic tests, and implements the
missing TLV relocations and runtime wrapper function. The TLV
relocations are just handled as GOT accesses.

rdar://84671534

Differential Revision: https://reviews.llvm.org/D112656

show more ...

[JITLink][MachO][arm64] Mask high bits out of immediate for LDRLiteral19.

Negative deltas for LDRLiteral19 have their high bits set. If these bits aren't
masked out then they will overwrite other in

[JITLink][MachO][arm64] Mask high bits out of immediate for LDRLiteral19.

Negative deltas for LDRLiteral19 have their high bits set. If these bits aren't
masked out then they will overwrite other instruction bits, leading to a bogus
encoding.

This long-standing relocation bug was exposed by e50aea58d59, "[JITLink][ORC]
Major JITLinkMemoryManager refactor.", which caused memory layouts to be
reordered, which in turn lead to a previously unseen negative delta. (Unseen
because LDRLiteral19s were only created in JITLink passes where they always
pointed at segments that were layed-out-after in the old layout).

No testcase yet: Our existing regression test infrastructure is good at checking
that operand bits are correct, but provides no easy way to test for bad opcode
bits. I'll have a think about the right way to approach this.

https://llvm.org/PR52153

show more ...

Re-apply e50aea58d59, "Major JITLinkMemoryManager refactor". with fixes.

Adds explicit narrowing casts to JITLinkMemoryManager.cpp.

Honors -slab-address option in llvm-jitlink.cpp, which was accide

Re-apply e50aea58d59, "Major JITLinkMemoryManager refactor". with fixes.

Adds explicit narrowing casts to JITLinkMemoryManager.cpp.

Honors -slab-address option in llvm-jitlink.cpp, which was accidentally
dropped in the refactor.

This effectively reverts commit 6641d29b70993bce6dbd7e0e0f1040753d38842f.

show more ...

Revert "[JITLink][ORC] Major JITLinkMemoryManager refactor."

This reverts commit e50aea58d59c8cfae807a7fee21c4227472c0678 while I
investigate bot failures.

[JITLink][ORC] Major JITLinkMemoryManager refactor.

This commit substantially refactors the JITLinkMemoryManager API to: (1) add
asynchronous versions of key operations, (2) give memory manager impl

[JITLink][ORC] Major JITLinkMemoryManager refactor.

This commit substantially refactors the JITLinkMemoryManager API to: (1) add
asynchronous versions of key operations, (2) give memory manager implementations
full control over link graph address layout, (3) enable more efficient tracking
of allocated memory, and (4) support "allocation actions" and finalize-lifetime
memory.

Together these changes provide a more usable API, and enable more powerful and
efficient memory manager implementations.

To support these changes the JITLinkMemoryManager::Allocation inner class has
been split into two new classes: InFlightAllocation, and FinalizedAllocation.
The allocate method returns an InFlightAllocation that tracks memory (both
working and executor memory) prior to finalization. The finalize method returns
a FinalizedAllocation object, and the InFlightAllocation is discarded. Breaking
Allocation into InFlightAllocation and FinalizedAllocation allows
InFlightAllocation subclassses to be written more naturally, and FinalizedAlloc
to be implemented and used efficiently (see (3) below).

In addition to the memory manager changes this commit also introduces a new
MemProt type to represent memory protections (MemProt replaces use of
sys::Memory::ProtectionFlags in JITLink), and a new MemDeallocPolicy type that
can be used to indicate when a section should be deallocated (see (4) below).

Plugin/pass writers who were using sys::Memory::ProtectionFlags will have to
switch to MemProt -- this should be straightworward. Clients with out-of-tree
memory managers will need to update their implementations. Clients using
in-tree memory managers should mostly be able to ignore it.

Major features:

(1) More asynchrony:

The allocate and deallocate methods are now asynchronous by default, with
synchronous convenience wrappers supplied. The asynchronous versions allow
clients (including JITLink) to request and deallocate memory without blocking.

(2) Improved control over graph address layout:

Instead of a SegmentRequestMap, JITLinkMemoryManager::allocate now takes a
reference to the LinkGraph to be allocated. The memory manager is responsible
for calculating the memory requirements for the graph, and laying out the graph
(setting working and executor memory addresses) within the allocated memory.
This gives memory managers full control over JIT'd memory layout. For clients
that don't need or want this degree of control the new "BasicLayout" utility can
be used to get a segment-based view of the graph, similar to the one provided by
SegmentRequestMap. Once segment addresses are assigned the BasicLayout::apply
method can be used to automatically lay out the graph.

(3) Efficient tracking of allocated memory.

The FinalizedAlloc type is a wrapper for an ExecutorAddr and requires only
64-bits to store in the controller. The meaning of the address held by the
FinalizedAlloc is left up to the memory manager implementation, but the
FinalizedAlloc type enforces a requirement that deallocate be called on any
non-default values prior to destruction. The deallocate method takes a
vector<FinalizedAlloc>, allowing for bulk deallocation of many allocations in a
single call.

Memory manager implementations will typically store the address of some
allocation metadata in the executor in the FinalizedAlloc, as holding this
metadata in the executor is often cheaper and may allow for clean deallocation
even in failure cases where the connection with the controller is lost.

(4) Support for "allocation actions" and finalize-lifetime memory.

Allocation actions are pairs (finalize_act, deallocate_act) of JITTargetAddress
triples (fn, arg_buffer_addr, arg_buffer_size), that can be attached to a
finalize request. At finalization time, after memory protections have been
applied, each of the "finalize_act" elements will be called in order (skipping
any elements whose fn value is zero) as

((char*(*)(const char *, size_t))fn)((const char *)arg_buffer_addr,
(size_t)arg_buffer_size);

At deallocation time the deallocate elements will be run in reverse order (again
skipping any elements where fn is zero).

The returned char * should be null to indicate success, or a non-null
heap-allocated string error message to indicate failure.

These actions allow finalization and deallocation to be extended to include
operations like registering and deregistering eh-frames, TLS sections,
initializer and deinitializers, and language metadata sections. Previously these
operations required separate callWrapper invocations. Compared to callWrapper
invocations, actions require no extra IPC/RPC, reducing costs and eliminating
a potential source of errors.

Finalize lifetime memory can be used to support finalize actions: Sections with
finalize lifetime should be destroyed by memory managers immediately after
finalization actions have been run. Finalize memory can be used to support
finalize actions (e.g. with extra-metadata, or synthesized finalize actions)
without incurring permanent memory overhead.

show more ...

[JITLink][MachO][arm64] Add support for splitting compact-unwind sections.

CompactUnwindSplitter splits compact-unwind sections on record boundaries and
adds keep-alive edges from target functions b

[JITLink][MachO][arm64] Add support for splitting compact-unwind sections.

CompactUnwindSplitter splits compact-unwind sections on record boundaries and
adds keep-alive edges from target functions back to their respective records.

In MachO_arm64.cpp, a CompactUnwindSplitter pass is added to the pre-prune pass
list when setting up the standard pipeline.

This patch does not provide runtime support for compact-unwind, but is a first
step towards enabling it.

show more ...

[JITLink][MachO][arm64] Build GOT entries for defined symbols too.

During the generic x86-64 support refactor in ecf6466f01c52 the implementation
of MachO_arm64_GOTAndStubsBuilder::isGOTEdgeToFix wa

[JITLink][MachO][arm64] Build GOT entries for defined symbols too.

During the generic x86-64 support refactor in ecf6466f01c52 the implementation
of MachO_arm64_GOTAndStubsBuilder::isGOTEdgeToFix was altered to only return
true for external symbols. This behavior is incorrect: GOT entries may be
required for defined symbols (e.g. in the large code model).

This patch fixes the bug and adds a test case for it (renaming an old test
case to avoid any ambiguity).

show more ...

[JITLink] Enable creation and management of mutable block content.

This patch introduces new operations on jitlink::Blocks: setMutableContent,
getMutableContent and getAlreadyMutableContent. The set

[JITLink] Enable creation and management of mutable block content.

This patch introduces new operations on jitlink::Blocks: setMutableContent,
getMutableContent and getAlreadyMutableContent. The setMutableContent method
will set the block content data and size members and flag the content as
mutable. The getMutableContent method will return a mutable copy of the existing
content value, auto-allocating and populating a new mutable copy if the existing
content is marked immutable. The getAlreadyMutableMethod asserts that the
existing content is already mutable and returns it.

setMutableContent should be used when updating the block with totally new
content backed by mutable memory. It can be used to change the size of the
block. The argument value should *not* be shared with any other block.

getMutableContent should be used when clients want to modify the existing
content and are unsure whether it is mutable yet.

getAlreadyMutableContent should be used when clients want to modify the existing
content and know from context that it must already be immutable.

These operations reduce copy-modify-update boilerplate and unnecessary copies
introduced when clients couldn't me sure whether the existing content was
mutable or not.

show more ...

[JITLink] Switch from StringRef to ArrayRef<char>, add some generic x86-64 utils

Adds utilities for creating anonymous pointers and jump stubs to x86_64.h. These
are used by the GOT and Stubs builde

[JITLink] Switch from StringRef to ArrayRef<char>, add some generic x86-64 utils

Adds utilities for creating anonymous pointers and jump stubs to x86_64.h. These
are used by the GOT and Stubs builder, but may also be used by pass writers who
want to create pointer stubs for indirection.

This patch also switches the underlying type for LinkGraph content from
StringRef to ArrayRef<char>. This avoids any confusion when working with buffers
that contain null bytes in the middle like, for example, a newly added null
pointer content array. ;)

show more ...