[ORC][ORC_RT] Handle ELF .init_array with non-default priority

ELF-based platforms currently support defining multiple static
initializer table sections with differing priorities, for example
.init_

[ORC][ORC_RT] Handle ELF .init_array with non-default priority

ELF-based platforms currently support defining multiple static
initializer table sections with differing priorities, for example
.init_array.0 or .init_array.100; the default .init_array corresponds
to a priority of 65535. When building a shared library or executable,
the system linker normally sorts these sections and combines them into
a single .init_array section. This change adds the capability to
recognize ELF static initializers with priorities other than the
default, and to properly sort them by priority, to Orc and the Orc
runtime.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D127056

show more ...

[ORC][JITLink] Fix MachO absolute symbol handling, add test case.

This patch removes the unintended resolution of locally scoped absolute symbols
(which was causing unexpected definition errors).

I

[ORC][JITLink] Fix MachO absolute symbol handling, add test case.

This patch removes the unintended resolution of locally scoped absolute symbols
(which was causing unexpected definition errors).

It stops using the JITSymbolFlags::Absolute flag (it isn't set or used elsewhere,
and causes mismatch-flags asserts), and adds JITSymbolFlags::Exported to default
scoped absolute symbols.

Finally, we now set the scope of absolute symbols correctly in
MachOLinkGraphBuilder.

show more ...

[llvm] Use = default (NFC)

[ORC][JITLink] Merge JITLink AllocActionCall and ORC WrapperFunctionCall.

These types performed identical roles. Merging them simplifies interoperability
between JITLink and ORC APIs (allowing us to

[ORC][JITLink] Merge JITLink AllocActionCall and ORC WrapperFunctionCall.

These types performed identical roles. Merging them simplifies interoperability
between JITLink and ORC APIs (allowing us to address a few FIXMEs).

show more ...

Re-apply "[JITLink] Update JITLink to use ExecutorAddr rather... " with fixes.

This re-applies 133f86e95492b2a00b944e070878424cfa73f87c, which was reverted in
c5965a411c635106a47738b8d2e24db822b7416

Re-apply "[JITLink] Update JITLink to use ExecutorAddr rather... " with fixes.

This re-applies 133f86e95492b2a00b944e070878424cfa73f87c, which was reverted in
c5965a411c635106a47738b8d2e24db822b7416f while I investigated bot failures.

The original failure contained an arithmetic conversion think-o (on line 419 of
EHFrameSupport.cpp) that could cause failures on 32-bit platforms. The issue
should be fixed in this patch.

show more ...

Revert "[JITLink] Update JITLink to use ExecutorAddr rather than..."

This reverts commit 133f86e95492b2a00b944e070878424cfa73f87c while I investigate
the bot failures at https://lab.llvm.org/buildbo

Revert "[JITLink] Update JITLink to use ExecutorAddr rather than..."

This reverts commit 133f86e95492b2a00b944e070878424cfa73f87c while I investigate
the bot failures at https://lab.llvm.org/buildbot#builders/186/builds/3370.

show more ...

[JITLink] Update JITLink to use ExecutorAddr rather than JITTargetAddress.

ExecutorAddr is the preferred representation for executor process addresses now.

[ORC] Add custom object interface support to StaticLibaryDefinitionGenerator.

This adds a GetObjectFileInterface callback member to
StaticLibraryDefinitionGenerator, and adds an optional argument fo

[ORC] Add custom object interface support to StaticLibaryDefinitionGenerator.

This adds a GetObjectFileInterface callback member to
StaticLibraryDefinitionGenerator, and adds an optional argument for initializing
that member to StaticLibraryDefinitionGenerator's named constructors. If not
supplied, it will default to getObjectFileInterface from ObjectFileInterface.h.

To enable testing a `-hidden-l<x>` option is added to the llvm-jitlink tool.
This allows archives to be loaded with all contained symbol visibilities demoted
to hidden.

The ObjectLinkingLayer::setOverrideObjectFlagsWithResponsibilityFlags method is
(belatedly) hooked up, and enabled in llvm-jitlink when `-hidden-l<x>` is used
so that the demotion is also applied at symbol resolution time (avoiding any
"mismatched symbol flags" crashes).

show more ...

[ORC] Add a MaterializationUnit::Interface struct.

MaterializationUnit::Interface holds the values that make up the interface
(for ORC's purposes) of a materialization unit: the symbol flags map and

[ORC] Add a MaterializationUnit::Interface struct.

MaterializationUnit::Interface holds the values that make up the interface
(for ORC's purposes) of a materialization unit: the symbol flags map and
initializer symbol.

Having a type for this will make functions that build materializer interfaces
more readable and maintainable.

show more ...

[ORC] Fix materialization of weak local symbols

We were adding all defined weak symbols to the materialization
responsibility, but local symbols will not be in the symbol table, so it
failed to mate

[ORC] Fix materialization of weak local symbols

We were adding all defined weak symbols to the materialization
responsibility, but local symbols will not be in the symbol table, so it
failed to materialize due to the "missing" symbol.

Local weak symbols come up in practice when using `ld -r` with a hidden
weak symbol.

rdar://85574696

show more ...

[ORC][ORC-RT] Register type metadata from __swift5_types MachO section

Similar to how the other swift sections are registered by the ORC
runtime's macho platform, add the __swift5_types section, whi

[ORC][ORC-RT] Register type metadata from __swift5_types MachO section

Similar to how the other swift sections are registered by the ORC
runtime's macho platform, add the __swift5_types section, which contains
type metadata. Add a simple test that demonstrates that the swift
runtime recognized the registered types.

rdar://85358530

Differential Revision: https://reviews.llvm.org/D113811

show more ...

[ORC] Don't try to perform empty deallocations.

Re-apply e50aea58d59, "Major JITLinkMemoryManager refactor". with fixes.

Adds explicit narrowing casts to JITLinkMemoryManager.cpp.

Honors -slab-address option in llvm-jitlink.cpp, which was accide

Re-apply e50aea58d59, "Major JITLinkMemoryManager refactor". with fixes.

Adds explicit narrowing casts to JITLinkMemoryManager.cpp.

Honors -slab-address option in llvm-jitlink.cpp, which was accidentally
dropped in the refactor.

This effectively reverts commit 6641d29b70993bce6dbd7e0e0f1040753d38842f.

show more ...

Revert "[JITLink][ORC] Major JITLinkMemoryManager refactor."

This reverts commit e50aea58d59c8cfae807a7fee21c4227472c0678 while I
investigate bot failures.

[JITLink][ORC] Major JITLinkMemoryManager refactor.

This commit substantially refactors the JITLinkMemoryManager API to: (1) add
asynchronous versions of key operations, (2) give memory manager impl

[JITLink][ORC] Major JITLinkMemoryManager refactor.

This commit substantially refactors the JITLinkMemoryManager API to: (1) add
asynchronous versions of key operations, (2) give memory manager implementations
full control over link graph address layout, (3) enable more efficient tracking
of allocated memory, and (4) support "allocation actions" and finalize-lifetime
memory.

Together these changes provide a more usable API, and enable more powerful and
efficient memory manager implementations.

To support these changes the JITLinkMemoryManager::Allocation inner class has
been split into two new classes: InFlightAllocation, and FinalizedAllocation.
The allocate method returns an InFlightAllocation that tracks memory (both
working and executor memory) prior to finalization. The finalize method returns
a FinalizedAllocation object, and the InFlightAllocation is discarded. Breaking
Allocation into InFlightAllocation and FinalizedAllocation allows
InFlightAllocation subclassses to be written more naturally, and FinalizedAlloc
to be implemented and used efficiently (see (3) below).

In addition to the memory manager changes this commit also introduces a new
MemProt type to represent memory protections (MemProt replaces use of
sys::Memory::ProtectionFlags in JITLink), and a new MemDeallocPolicy type that
can be used to indicate when a section should be deallocated (see (4) below).

Plugin/pass writers who were using sys::Memory::ProtectionFlags will have to
switch to MemProt -- this should be straightworward. Clients with out-of-tree
memory managers will need to update their implementations. Clients using
in-tree memory managers should mostly be able to ignore it.

Major features:

(1) More asynchrony:

The allocate and deallocate methods are now asynchronous by default, with
synchronous convenience wrappers supplied. The asynchronous versions allow
clients (including JITLink) to request and deallocate memory without blocking.

(2) Improved control over graph address layout:

Instead of a SegmentRequestMap, JITLinkMemoryManager::allocate now takes a
reference to the LinkGraph to be allocated. The memory manager is responsible
for calculating the memory requirements for the graph, and laying out the graph
(setting working and executor memory addresses) within the allocated memory.
This gives memory managers full control over JIT'd memory layout. For clients
that don't need or want this degree of control the new "BasicLayout" utility can
be used to get a segment-based view of the graph, similar to the one provided by
SegmentRequestMap. Once segment addresses are assigned the BasicLayout::apply
method can be used to automatically lay out the graph.

(3) Efficient tracking of allocated memory.

The FinalizedAlloc type is a wrapper for an ExecutorAddr and requires only
64-bits to store in the controller. The meaning of the address held by the
FinalizedAlloc is left up to the memory manager implementation, but the
FinalizedAlloc type enforces a requirement that deallocate be called on any
non-default values prior to destruction. The deallocate method takes a
vector<FinalizedAlloc>, allowing for bulk deallocation of many allocations in a
single call.

Memory manager implementations will typically store the address of some
allocation metadata in the executor in the FinalizedAlloc, as holding this
metadata in the executor is often cheaper and may allow for clean deallocation
even in failure cases where the connection with the controller is lost.

(4) Support for "allocation actions" and finalize-lifetime memory.

Allocation actions are pairs (finalize_act, deallocate_act) of JITTargetAddress
triples (fn, arg_buffer_addr, arg_buffer_size), that can be attached to a
finalize request. At finalization time, after memory protections have been
applied, each of the "finalize_act" elements will be called in order (skipping
any elements whose fn value is zero) as

((char*(*)(const char *, size_t))fn)((const char *)arg_buffer_addr,
(size_t)arg_buffer_size);

At deallocation time the deallocate elements will be run in reverse order (again
skipping any elements where fn is zero).

The returned char * should be null to indicate success, or a non-null
heap-allocated string error message to indicate failure.

These actions allow finalization and deallocation to be extended to include
operations like registering and deregistering eh-frames, TLS sections,
initializer and deinitializers, and language metadata sections. Previously these
operations required separate callWrapper invocations. Compared to callWrapper
invocations, actions require no extra IPC/RPC, reducing costs and eliminating
a potential source of errors.

Finalize lifetime memory can be used to support finalize actions: Sections with
finalize lifetime should be destroyed by memory managers immediately after
finalization actions have been run. Finalize memory can be used to support
finalize actions (e.g. with extra-metadata, or synthesized finalize actions)
without incurring permanent memory overhead.

show more ...

[ORC] Hold shared_ptr<SymbolStringPool> in errors containing SymbolStringPtrs.

This allows these error values to remain valid, even if they tear down the JIT
itself.

[llvm] Use pop_back_val (NFC)

[ORC][ORC-RT] Reapply "Introduce ELF/*nix Platform and runtime..." with fixes.

This reapplies e256445bfff, which was reverted in 45ac5f54418 due to bot errors
(e.g. https://lab.llvm.org/buildbot/#/b

[ORC][ORC-RT] Reapply "Introduce ELF/*nix Platform and runtime..." with fixes.

This reapplies e256445bfff, which was reverted in 45ac5f54418 due to bot errors
(e.g. https://lab.llvm.org/buildbot/#/builders/112/builds/8599). The issue that
caused the bot failure was fixed in 2e6a4fce356.

show more ...

Revert "[ORC-RT][ORC] Introduce ELF/*nix Platform and runtime support."

This reverts commit e256445bfff12013c3c4ad97da4aa69d25b175b5.

This commit broke some of the bots (see e.g.
https://lab.llvm.o

Revert "[ORC-RT][ORC] Introduce ELF/*nix Platform and runtime support."

This reverts commit e256445bfff12013c3c4ad97da4aa69d25b175b5.

This commit broke some of the bots (see e.g.
https://lab.llvm.org/buildbot/#/builders/112/builds/8599). Reverting while I
investigate.

show more ...

[ORC-RT][ORC] Introduce ELF/*nix Platform and runtime support.

This change adds support to ORCv2 and the Orc runtime library for static
initializers, C++ static destructors, and exception handler re

[ORC-RT][ORC] Introduce ELF/*nix Platform and runtime support.

This change adds support to ORCv2 and the Orc runtime library for static
initializers, C++ static destructors, and exception handler registration for
ELF-based platforms, at present Linux and FreeBSD on x86_64. It is based on the
MachO platform and runtime support introduced in bb5f97e3ad1.

Patch by Peter Housel. Thanks very much Peter!

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D108081

show more ...

[ORC] Require ExecutorProcessControl when constructing an ExecutionSession.

Wrapper function call and dispatch handler helpers are moved to
ExecutionSession, and existing EPC-based tools are re-writ

[ORC] Require ExecutorProcessControl when constructing an ExecutionSession.

Wrapper function call and dispatch handler helpers are moved to
ExecutionSession, and existing EPC-based tools are re-written to take an
ExecutionSession argument instead.

Requiring an ExecutorProcessControl instance simplifies existing EPC based
utilities (which only need to take an ES now), and should encourage more
utilities to use the EPC interface. It also simplifies process termination,
since the session can automatically call ExecutorProcessControl::disconnect
(previously this had to be done manually, and carefully ordered with the
rest of JIT tear-down to work correctly).

show more ...

[Orc] Remove unnecessary <string> include dependency from Orc headers. NFC.

At most these use the StringRef/Twine wrappers and don't have any implicit uses of std::string.

Move the include down to

[Orc] Remove unnecessary <string> include dependency from Orc headers. NFC.

At most these use the StringRef/Twine wrappers and don't have any implicit uses of std::string.

Move the include down to any cpp implementation where std::string is actually used.

show more ...

[ORC] Improve computeLocalDeps / computeNamedSymbolDependencies performance.

The computeNamedSymbolDependencies and computeLocalDeps methods on
ObjectLinkingLayerJITLinkContext are responsible for c

[ORC] Improve computeLocalDeps / computeNamedSymbolDependencies performance.

The computeNamedSymbolDependencies and computeLocalDeps methods on
ObjectLinkingLayerJITLinkContext are responsible for computing, for each symbol
in the current MaterializationResponsibility, the set of non-locally-scoped
symbols that are depended on. To calculate this we have to consider the effect
of chains of dependence through locally scoped symbols in the LinkGraph. E.g.

.text
.globl foo
foo:
callq bar ## foo depneds on external 'bar'
movq Ltmp1(%rip), %rcx ## foo depends on locally scoped 'Ltmp1'
addl (%rcx), %eax
retq

.data
Ltmp1:
.quad x ## Ltmp1 depends on external 'x'

In this example symbol 'foo' depends directly on 'bar', and indirectly on 'x'
via 'Ltmp1', which is locally scoped.

Performance of the existing implementations appears to have been mediocre:
Based on flame graphs posted by @drmeister (in #jit on the LLVM discord server)
the computeLocalDeps function was taking up a substantial amount of time when
starting up Clasp (https://github.com/clasp-developers/clasp).

This commit attempts to address the performance problems in three ways:

1. Using jitlink::Blocks instead of jitlink::Symbols as the nodes of the
dependencies-introduced-by-locally-scoped-symbols graph.

Using either Blocks or Symbols as nodes provides the same information, but since
there may be more than one locally scoped symbol per block the block-based
version of the dependence graph should always be a subgraph of the Symbol-based
version, and so faster to operate on.

2. Improved worklist management.

The older version of computeLocalDeps used a fixed worklist containing all
nodes, and iterated over this list propagating dependencies until no further
changes were required. The worklist was not sorted into a useful order before
the loop started.

The new version uses a variable work-stack, visiting nodes in DFS order and
only adding nodes when there is meaningful work to do on them.

Compared to the old version the new version avoids revisiting nodes which
haven't changed, and I suspect it converges more quickly (due to the DFS
ordering).

3. Laziness and caching.

Mappings of...

jitlink::Symbol* -> Interned Name (as SymbolStringPtr)
jitlink::Block* -> Immediate dependencies (as SymbolNameSet)
jitlink::Block* -> Transitive dependencies (as SymbolNameSet)

are all built lazily and cached while running computeNamedSymbolDependencies.

According to @drmeister these changes reduced Clasp startup time in his test
setup (averaged over a handful of starts) from 4.8 to 2.8 seconds (with
ORC/JITLink linking ~11,000 object files in that time), which seems like
enough to justify switching to the new algorithm in the absence of any other
perf numbers.

show more ...

[ORC] Don't try to obtain a ref to a non-existent buffer.

[ORC] Add support for adding LinkGraphs directly to ObjectLinkingLayer.

This is separate from (but builds on) the support added in ec6b71df70a for
emitting LinkGraphs in the context of an active mat

[ORC] Add support for adding LinkGraphs directly to ObjectLinkingLayer.

This is separate from (but builds on) the support added in ec6b71df70a for
emitting LinkGraphs in the context of an active materialization. This commit
makes LinkGraphs a first-class data structure with features equivalent to
object files within ObjectLinkingLayer.

show more ...