[llvm] Use = default (NFC)

Re-apply "[JITLink] Update JITLink to use ExecutorAddr rather... " with fixes.

This re-applies 133f86e95492b2a00b944e070878424cfa73f87c, which was reverted in
c5965a411c635106a47738b8d2e24db822b7416

Re-apply "[JITLink] Update JITLink to use ExecutorAddr rather... " with fixes.

This re-applies 133f86e95492b2a00b944e070878424cfa73f87c, which was reverted in
c5965a411c635106a47738b8d2e24db822b7416f while I investigated bot failures.

The original failure contained an arithmetic conversion think-o (on line 419 of
EHFrameSupport.cpp) that could cause failures on 32-bit platforms. The issue
should be fixed in this patch.

show more ...

Revert "[JITLink] Update JITLink to use ExecutorAddr rather than..."

This reverts commit 133f86e95492b2a00b944e070878424cfa73f87c while I investigate
the bot failures at https://lab.llvm.org/buildbo

Revert "[JITLink] Update JITLink to use ExecutorAddr rather than..."

This reverts commit 133f86e95492b2a00b944e070878424cfa73f87c while I investigate
the bot failures at https://lab.llvm.org/buildbot#builders/186/builds/3370.

show more ...

[JITLink] Update JITLink to use ExecutorAddr rather than JITTargetAddress.

ExecutorAddr is the preferred representation for executor process addresses now.

Re-apply e50aea58d59, "Major JITLinkMemoryManager refactor". with fixes.

Adds explicit narrowing casts to JITLinkMemoryManager.cpp.

Honors -slab-address option in llvm-jitlink.cpp, which was accide

Re-apply e50aea58d59, "Major JITLinkMemoryManager refactor". with fixes.

Adds explicit narrowing casts to JITLinkMemoryManager.cpp.

Honors -slab-address option in llvm-jitlink.cpp, which was accidentally
dropped in the refactor.

This effectively reverts commit 6641d29b70993bce6dbd7e0e0f1040753d38842f.

show more ...

Revert "[JITLink][ORC] Major JITLinkMemoryManager refactor."

This reverts commit e50aea58d59c8cfae807a7fee21c4227472c0678 while I
investigate bot failures.

[JITLink][ORC] Major JITLinkMemoryManager refactor.

This commit substantially refactors the JITLinkMemoryManager API to: (1) add
asynchronous versions of key operations, (2) give memory manager impl

[JITLink][ORC] Major JITLinkMemoryManager refactor.

This commit substantially refactors the JITLinkMemoryManager API to: (1) add
asynchronous versions of key operations, (2) give memory manager implementations
full control over link graph address layout, (3) enable more efficient tracking
of allocated memory, and (4) support "allocation actions" and finalize-lifetime
memory.

Together these changes provide a more usable API, and enable more powerful and
efficient memory manager implementations.

To support these changes the JITLinkMemoryManager::Allocation inner class has
been split into two new classes: InFlightAllocation, and FinalizedAllocation.
The allocate method returns an InFlightAllocation that tracks memory (both
working and executor memory) prior to finalization. The finalize method returns
a FinalizedAllocation object, and the InFlightAllocation is discarded. Breaking
Allocation into InFlightAllocation and FinalizedAllocation allows
InFlightAllocation subclassses to be written more naturally, and FinalizedAlloc
to be implemented and used efficiently (see (3) below).

In addition to the memory manager changes this commit also introduces a new
MemProt type to represent memory protections (MemProt replaces use of
sys::Memory::ProtectionFlags in JITLink), and a new MemDeallocPolicy type that
can be used to indicate when a section should be deallocated (see (4) below).

Plugin/pass writers who were using sys::Memory::ProtectionFlags will have to
switch to MemProt -- this should be straightworward. Clients with out-of-tree
memory managers will need to update their implementations. Clients using
in-tree memory managers should mostly be able to ignore it.

Major features:

(1) More asynchrony:

The allocate and deallocate methods are now asynchronous by default, with
synchronous convenience wrappers supplied. The asynchronous versions allow
clients (including JITLink) to request and deallocate memory without blocking.

(2) Improved control over graph address layout:

Instead of a SegmentRequestMap, JITLinkMemoryManager::allocate now takes a
reference to the LinkGraph to be allocated. The memory manager is responsible
for calculating the memory requirements for the graph, and laying out the graph
(setting working and executor memory addresses) within the allocated memory.
This gives memory managers full control over JIT'd memory layout. For clients
that don't need or want this degree of control the new "BasicLayout" utility can
be used to get a segment-based view of the graph, similar to the one provided by
SegmentRequestMap. Once segment addresses are assigned the BasicLayout::apply
method can be used to automatically lay out the graph.

(3) Efficient tracking of allocated memory.

The FinalizedAlloc type is a wrapper for an ExecutorAddr and requires only
64-bits to store in the controller. The meaning of the address held by the
FinalizedAlloc is left up to the memory manager implementation, but the
FinalizedAlloc type enforces a requirement that deallocate be called on any
non-default values prior to destruction. The deallocate method takes a
vector<FinalizedAlloc>, allowing for bulk deallocation of many allocations in a
single call.

Memory manager implementations will typically store the address of some
allocation metadata in the executor in the FinalizedAlloc, as holding this
metadata in the executor is often cheaper and may allow for clean deallocation
even in failure cases where the connection with the controller is lost.

(4) Support for "allocation actions" and finalize-lifetime memory.

Allocation actions are pairs (finalize_act, deallocate_act) of JITTargetAddress
triples (fn, arg_buffer_addr, arg_buffer_size), that can be attached to a
finalize request. At finalization time, after memory protections have been
applied, each of the "finalize_act" elements will be called in order (skipping
any elements whose fn value is zero) as

((char*(*)(const char *, size_t))fn)((const char *)arg_buffer_addr,
(size_t)arg_buffer_size);

At deallocation time the deallocate elements will be run in reverse order (again
skipping any elements where fn is zero).

The returned char * should be null to indicate success, or a non-null
heap-allocated string error message to indicate failure.

These actions allow finalization and deallocation to be extended to include
operations like registering and deregistering eh-frames, TLS sections,
initializer and deinitializers, and language metadata sections. Previously these
operations required separate callWrapper invocations. Compared to callWrapper
invocations, actions require no extra IPC/RPC, reducing costs and eliminating
a potential source of errors.

Finalize lifetime memory can be used to support finalize actions: Sections with
finalize lifetime should be destroyed by memory managers immediately after
finalization actions have been run. Finalize memory can be used to support
finalize actions (e.g. with extra-metadata, or synthesized finalize actions)
without incurring permanent memory overhead.

show more ...

[JITLink] Working memory shouldn't be subject to alignment constraints.

Refactors copyBlockContentToWorkingMemory to use offsets rather than direct
pointers to working memory. This simplifies the pr

[JITLink] Working memory shouldn't be subject to alignment constraints.

Refactors copyBlockContentToWorkingMemory to use offsets rather than direct
pointers to working memory. This simplifies the problem of maintaining
alignments between blocks in working memory, without requiring the working
memory itself to be aligned.

show more ...

[JITLink] Enable creation and management of mutable block content.

This patch introduces new operations on jitlink::Blocks: setMutableContent,
getMutableContent and getAlreadyMutableContent. The set

[JITLink] Enable creation and management of mutable block content.

This patch introduces new operations on jitlink::Blocks: setMutableContent,
getMutableContent and getAlreadyMutableContent. The setMutableContent method
will set the block content data and size members and flag the content as
mutable. The getMutableContent method will return a mutable copy of the existing
content value, auto-allocating and populating a new mutable copy if the existing
content is marked immutable. The getAlreadyMutableMethod asserts that the
existing content is already mutable and returns it.

setMutableContent should be used when updating the block with totally new
content backed by mutable memory. It can be used to change the size of the
block. The argument value should *not* be shared with any other block.

getMutableContent should be used when clients want to modify the existing
content and are unsure whether it is mutable yet.

getAlreadyMutableContent should be used when clients want to modify the existing
content and know from context that it must already be immutable.

These operations reduce copy-modify-update boilerplate and unnecessary copies
introduced when clients couldn't me sure whether the existing content was
mutable or not.

show more ...

[JITLink] Switch from StringRef to ArrayRef<char>, add some generic x86-64 utils

Adds utilities for creating anonymous pointers and jump stubs to x86_64.h. These
are used by the GOT and Stubs builde

[JITLink] Switch from StringRef to ArrayRef<char>, add some generic x86-64 utils

Adds utilities for creating anonymous pointers and jump stubs to x86_64.h. These
are used by the GOT and Stubs builder, but may also be used by pass writers who
want to create pointer stubs for indirection.

This patch also switches the underlying type for LinkGraph content from
StringRef to ArrayRef<char>. This avoids any confusion when working with buffers
that contain null bytes in the middle like, for example, a newly added null
pointer content array. ;)

show more ...

[JITLink] Don't issue lookups for empty symbol sets.

Issuing a lookup for an empty symbol set is legal, but can actually result in
unrelated work being done if there was a work queue left over from

[JITLink] Don't issue lookups for empty symbol sets.

Issuing a lookup for an empty symbol set is legal, but can actually result in
unrelated work being done if there was a work queue left over from the previous
lookup. We can avoid doing this unrelated work (reducing stack depth and
interleaving of debugging output) by not issuing these no-op lookups in the
first place.

show more ...

[JITLink][MachO][x86-64] Introduce generic x86-64 support.

This patch introduces generic x86-64 edge kinds, and refactors the MachO/x86-64
backend to use these edge kinds. This simplifies the implem

[JITLink][MachO][x86-64] Introduce generic x86-64 support.

This patch introduces generic x86-64 edge kinds, and refactors the MachO/x86-64
backend to use these edge kinds. This simplifies the implementation of the
MachO/x86-64 backend and makes it possible to write generic x86-64 passes and
utilities.

The new edge kinds are different from the original set used in the MachO/x86-64
backend. Several edge kinds that were not meaningfully distinguished in that
backend (e.g. the PCRelMinusN edges) have been merged into single edge kinds in
the new scheme (these edge kinds can be reintroduced later if we find a use for
them). At the same time, new edge kinds have been introduced to convey extra
information about the state of the graph. E.g. The Request*AndTransformTo**
edges represent GOT/TLVP relocations prior to synthesis of the GOT/TLVP
entries, and the 'Relaxable' suffix distinguishes edges that are candidates for
optimization from edges which should be left as-is (e.g. to enable runtime
redirection).

ELF/x86-64 will be refactored to use these generic edges at some point in the
future, and I anticipate a similar refactor to create a generic arm64 support
header too.

Differential Revision: https://reviews.llvm.org/D98305

show more ...

[JITLink] Add a new PostAllocationPasses list.

Passes in the new PostAllocationPasses list will run immediately after memory
allocation and address assignment for defined symbols, and before
JITLink

[JITLink] Add a new PostAllocationPasses list.

Passes in the new PostAllocationPasses list will run immediately after memory
allocation and address assignment for defined symbols, and before
JITLinkContext::notifyResolved is called. These passes can set up state
associated with the addresses of defined symbols before any query for these
addresses completes.

show more ...

[JITLink] Rename PostAllocationPasses to PreFixupPasses.

PreFixupPasses better reflects when these passes will run.

A future patch will (re)introduce a PostAllocationPasses list that will run
after

[JITLink] Rename PostAllocationPasses to PreFixupPasses.

PreFixupPasses better reflects when these passes will run.

A future patch will (re)introduce a PostAllocationPasses list that will run
after allocation, but before JITLinkContext::notifyResolved is called to notify
the rest of the JIT about the resolved symbol addresses.

show more ...

[JITLink][ORC] Enable creation / linking of raw jitlink::LinkGraphs.

Separates link graph creation from linking. This allows raw LinkGraphs to be
created and passed to a link. ObjectLinkingLayer is

[JITLink][ORC] Enable creation / linking of raw jitlink::LinkGraphs.

Separates link graph creation from linking. This allows raw LinkGraphs to be
created and passed to a link. ObjectLinkingLayer is updated to support emission
of raw LinkGraphs in addition to object buffers.

Raw LinkGraphs can be created by in-memory compilers to bypass object encoding /
decoding (though this prevents caching, as LinkGraphs have do not have an
on-disk representation), and by utility code to add programatically generated
data structures to the JIT target process.

show more ...

Re-apply 8904ee8ac7e with missing header included this time.

Revert "[JITLink] Add JITLinkDylib type, thread through JITLinkMemoryManager APIs."

This reverts commit 8904ee8ac7ebcc50a60de0914abc6862e28b6664.
Didn't `git add` llvm/ExecutionEngine/JITLink/JITLin

Revert "[JITLink] Add JITLinkDylib type, thread through JITLinkMemoryManager APIs."

This reverts commit 8904ee8ac7ebcc50a60de0914abc6862e28b6664.
Didn't `git add` llvm/ExecutionEngine/JITLink/JITLinkDylib.h and hence doesn't
build anywhere.

show more ...

[JITLink] Add JITLinkDylib type, thread through JITLinkMemoryManager APIs.

JITLinkDylib represents a target dylib for a JITLink link. By representing this
explicitly we can:
- Enable JITLinkMemory

[JITLink] Add JITLinkDylib type, thread through JITLinkMemoryManager APIs.

JITLinkDylib represents a target dylib for a JITLink link. By representing this
explicitly we can:
- Enable JITLinkMemoryManagers to manage allocations on a per-dylib basis
(e.g by maintaining a seperate allocation pool for each JITLinkDylib).
- Enable new features and diagnostics that require information about the
target dylib (not implemented in this patch).

show more ...

[llvm-jitlink] Add -phony-externals option to suppress unresolved externals.

The -phony-externals option adds a generator which explicitly defines any
otherwise unresolved externals as null. This tr

[llvm-jitlink] Add -phony-externals option to suppress unresolved externals.

The -phony-externals option adds a generator which explicitly defines any
otherwise unresolved externals as null. This transforms link-time
unresolved-symbol errors into potential runtime null pointer accesses
(if an unresolved external is actually accessed during execution).

This option can be useful in -harness mode to avoid having to mock a
large number of symbols that are not reachable at runtime (e.g. unused
methods referenced by a class vtable).

show more ...

[llvm-jitlink] Add -harness option to llvm-jitlink.

The -harness option enables new testing use-cases for llvm-jitlink. It takes a
list of objects to treat as a test harness for any regular objects

[llvm-jitlink] Add -harness option to llvm-jitlink.

The -harness option enables new testing use-cases for llvm-jitlink. It takes a
list of objects to treat as a test harness for any regular objects passed to
llvm-jitlink.

If any files are passed using the -harness option then the following
transformations are applied to all other files:

(1) Symbols definitions that are referenced by the harness files are promoted
to default scope. (This enables access to statics from test harness).

(2) Symbols definitions that clash with definitions in the harness files are
deleted. (This enables interposition by test harness).

(3) All other definitions in regular files are demoted to local scope.
(This causes untested code to be dead stripped, reducing memory cost and
eliminating spurious unresolved symbol errors from untested code).

These transformations allow the harness files to reference and interpose
symbols in the regular object files, which can be used to support execution
tests (including fuzz tests) of functions in relocatable objects produced by a
build.

show more ...

[JITLink] Allow JITLinkContext::notifyResolved to return an Error.

This allows clients to detect invalid transformations applied by JITLink passes
(e.g. inserting or removing symbols in unexpected w

[JITLink] Allow JITLinkContext::notifyResolved to return an Error.

This allows clients to detect invalid transformations applied by JITLink passes
(e.g. inserting or removing symbols in unexpected ways) and terminate linking
with an error.

This change is used to simplify the error propagation logic in
ObjectLinkingLayer.

show more ...

[JITLink] Display host -> target address mapping in debugging output.

This can be helpful for sanity checking JITLink memory manager behavior.

[JITLink] Add a MachO x86-64 GOT and Stub bypass optimization.

This optimization bypasses GOT loads and calls/branches through stubs when the
ultimate target of the access/branch is found to be with

[JITLink] Add a MachO x86-64 GOT and Stub bypass optimization.

This optimization bypasses GOT loads and calls/branches through stubs when the
ultimate target of the access/branch is found to be within range of the
reference.

Extra debugging output is also added to the generic JITLink algorithm and
basic GOT and Stubs builder utility to aid debugging.

show more ...

[ORC][JITLink] Add support for weak references, and improve handling of static
libraries.

This patch substantially updates ORCv2's lookup API in order to support weak
references, and to better suppo

[ORC][JITLink] Add support for weak references, and improve handling of static
libraries.

This patch substantially updates ORCv2's lookup API in order to support weak
references, and to better support static archives. Key changes:

-- Each symbol being looked for is now associated with a SymbolLookupFlags
value. If the associated value is SymbolLookupFlags::RequiredSymbol then
the symbol must be defined in one of the JITDylibs being searched (or be
able to be generated in one of these JITDylibs via an attached definition
generator) or the lookup will fail with an error. If the associated value is
SymbolLookupFlags::WeaklyReferencedSymbol then the symbol is permitted to be
undefined, in which case it will simply not appear in the resulting
SymbolMap if the rest of the lookup succeeds.

Since lookup now requires these flags for each symbol, the lookup method now
takes an instance of a new SymbolLookupSet type rather than a SymbolNameSet.
SymbolLookupSet is a vector-backed set of (name, flags) pairs. Clients are
responsible for ensuring that the set property (i.e. unique elements) holds,
though this is usually simple and SymbolLookupSet provides convenience
methods to support this.

-- Lookups now have an associated LookupKind value, which is either
LookupKind::Static or LookupKind::DLSym. Definition generators can inspect
the lookup kind when determining whether or not to generate new definitions.
The StaticLibraryDefinitionGenerator is updated to only pull in new objects
from the archive if the lookup kind is Static. This allows lookup to be
re-used to emulate dlsym for JIT'd symbols without pulling in new objects
from archives (which would not happen in a normal dlsym call).

-- JITLink is updated to allow externals to be assigned weak linkage, and
weak externals now use the SymbolLookupFlags::WeaklyReferencedSymbol value
for lookups. Unresolved weak references will be assigned the default value of
zero.

Since this patch was modifying the lookup API anyway, it alo replaces all of the
"MatchNonExported" boolean arguments with a "JITDylibLookupFlags" enum for
readability. If a JITDylib's associated value is
JITDylibLookupFlags::MatchExportedSymbolsOnly then the lookup will only
match against exported (non-hidden) symbols in that JITDylib. If a JITDylib's
associated value is JITDylibLookupFlags::MatchAllSymbols then the lookup will
match against any symbol defined in the JITDylib.

show more ...

[JITLink] Refactor EH-frame handling to support eh-frames with existing relocs.

Some targets (E.g. MachO/arm64) use relocations to fix some CFI record fields
in the eh-frame section. When relocation

[JITLink] Refactor EH-frame handling to support eh-frames with existing relocs.

Some targets (E.g. MachO/arm64) use relocations to fix some CFI record fields
in the eh-frame section. When relocations are used the initial (pre-relocation)
content of the eh-frame section can no longer be interpreted by following the
eh-frame specification. This causes errors in the existing eh-frame parser.

This patch moves eh-frame handling into two LinkGraph passes that are run after
relocations have been parsed (but before they are applied). The first] pass
breaks up blocks in the eh-frame section into per-CFI-record blocks, and the
second parses blocks of (potentially multiple) CFI records and adds the
appropriate edges to any CFI fields that do not have existing relocations.
These passes can be run independently of one another. By handling eh-frame
splitting/fixing with LinkGraph passes we can both re-use existing relocations
for CFI record fields and avoid applying eh-frame fixups before parsing the
section (which would complicate the linker and require extra temporary
allocations of working memory).

show more ...