Handle OOM in the rest of Wasmtime's non-component, -async, -compilation APIs (#12858)

* Handle OOM in more places in the public API

A bunch of random places:

* Add: `Trap::try_new` to handle OOM

Handle OOM in the rest of Wasmtime's non-component, -async, -compilation APIs (#12858)

* Handle OOM in more places in the public API

A bunch of random places:

* Add: `Trap::try_new` to handle OOM while creating traps
* Use: `TryVec` inside `Func::call_impl_do_call` and `wasm_val_raw_storage` to
hold the args and rets
* Add: `Instance::try_exports` for iterating over an instance's exports while
handling OOM
* `Linker:try_get`, like `Linker::get` but handling OOM
* `Linker:try_get_by_import`, like `Linker::get_by_import` but handling OOM
* Use `try_new` to box things in `SharedMemory::new`
* Use `TryVec` instead of `Vec` in our dynamic tables

* Add OOM tests for most of Wasmtime's public API

Excludes component-, async-, and compilation-related APIs.

* address review feedback

* fix test compilation

* fix c-api

show more ...

Use `TryPrimaryMap` in an instance's `OwnedImports` (#12847)

Relax panics in async/futures to traps/errors (#12688)

* Relax panics in async/futures to traps/errors

This commit is an admittance that I don't believe we're going to get
to a point where we are c

Relax panics in async/futures to traps/errors (#12688)

* Relax panics in async/futures to traps/errors

This commit is an admittance that I don't believe we're going to get
to a point where we are confident enough in the fuzzing of
component-model-async such that we could confidently say we're
exercising the vast majority of possible panics. Development of
component-model-async has shown a steady trickle of panics over the
course of the development of the feature, and this trend has been
persistent over time as well.

An attempt was made in #12119 to add a fuzzer dedicated to async events
but that didn't actually find anything in development and it has missed
a number of panics present before and discovered after its introduction.
Overall I do not know how to improve the fuzzer to the point that it
would find pretty much all of the existing async-related panics over
time.

To help address this concern of the `concurrent.rs` implementation this
commit goes through and replaces things like `unwrap()`, `assert!`,
`panic!`, and `unreachable!` with an error-producing form. The benefit
of this is that a bug in the implementation is less likely to result in
a panic and instead just results in a non-spec-compliant trap. The
downside of doing this though is that it can become unclear what errors
are "first class traps", or expected to be guest reachable, and which
are expected to be bugs in Wasmtime. To help address this I've performed
a few refactorings here as well.

* Some traps previously present as error strings are now promoted to
using `Trap::Foo` instead. This has some refactoring of the Rust/C
side as well to make it easier to define new variants. Tests were
additionally added for any trap messages that weren't previously
tested as being reachable.

* A new `bail_bug!` macro was added (internally) for Wasmtime. This is
coupled with a concrete `WasmtimeBug` error type (exported as
`wasmtime::WasmtimeBug`). The intention is that `bail!` continues to
be "here's a string and I'm a bit too lazy to make a concrete error"
while `bail_bug!` indicates "this is a bug in wasmtime please report
this if you see it".

The rough vision is that if an error condition is reached, and the system
is not broken in such a way that panicking is required, then `bail_bug!`
can be used to indicate a bug in Wasmtime as opposed to panicking. This
reduces the real-world impact of hitting these scenarios by downgrading a
CVE-worthy `panic!` into a bug-worthy non-spec-compliant trap. Not all
panics are able to be transitioned to this as some are load bearing from
a safety perspective or similar (or indicate something equally broken),
but the vast majority of cases are suitable for "return a trap, lock
down the store, and let destructors take care of everything else".

This change additionally has resulted in API changes for `FutureReader`
and `StreamReader`. For example creation of these types now returns a
`Result` for when the `ResourceTable` is full, for example, instead of
panicking.

* Fix CI build

* Translate `WasmtimeBug` to panics in debug mode

* Review comments

* Refactor some stream methods for fewer panics

show more ...

Debugging: apply single-stepping patches to modules instantiated after setting changes. (#12663)

We currently do a store-wide state-change on all registered modules
when the "single stepping" flag c

Debugging: apply single-stepping patches to modules instantiated after setting changes. (#12663)

We currently do a store-wide state-change on all registered modules
when the "single stepping" flag changes, patching in or out all
breakpoints. However, this design didn't account for modules
registered with the store *after* single-stepping is enabled (and new
modules may be registered any time an instantiation occurs). In
particular this is problematic when a debugger, e.g., sets the
single-step flag right at the beginning of execution of a "host main
function" that calls Wasm, before the main instantiation occurs.

This PR threads through the breakpoint state in the registration path,
with the narrow waist at `ModuleRegistry::register` which is the only
place where modules are added to the `LoadedCode`.

show more ...

Refactor `String` usage in `wasmtime_environ::Module` (#12565)

* Refactor `String` usage in `wasmtime_environ::Module`

Do not hold regular `String`s; instead use our own OOM-handling
`wasmtime_core

Refactor `String` usage in `wasmtime_environ::Module` (#12565)

* Refactor `String` usage in `wasmtime_environ::Module`

Do not hold regular `String`s; instead use our own OOM-handling
`wasmtime_core::alloc::String` or, even better, an interned `Atom` from the
`StringPool`.

Also avoid `IndexMap`, as it doesn't handle OOMs.

* Use OOM-handling `IndexMap` in `wasmtime_environ::Module`

* review feedback

show more ...

Refactor borrow state tracking for async tasks (#12550)

* Refactor borrow state tracking for async tasks

This commit is a somewhat deep refactoring of how the state of
`borrow<T>` is managed for b

Refactor borrow state tracking for async tasks (#12550)

* Refactor borrow state tracking for async tasks

This commit is a somewhat deep refactoring of how the state of
`borrow<T>` is managed for both the host and the guest with respect to
async tasks. This additionally refactors how some async task management
is done for host-called functions.

The fundamental problem being tackled here is #12510. In that issue it
was discovered that the way `CallContext`, the borrow tracking mechanism
in Wasmtime, is managed is incompatible with async tasks. Specifically
the previous assumption of the scope being mutated for a borrow is
somewhere on the call stack is no longer true. It's possible for an
async task to be suspended, for example, and then a sibling task drops a
borrow which should update the scope of the suspended task. There were a
number of other small issues I noticed here and there which this PR
additionally has tests for, all of which failed before this change and
pass afterwards.

The manner in which borrow state is manipulated is a pretty old part of
the component model implementation dating back to the original
implementation of resources. I decided to forgo any possible quick fix
and have attempted to more deeply refactor and integrate async tasks
into all of this infrastructure. A list of the changes made here are:

* The `CallContexts` structure, a stack of `CallContext`, was removed.
Tasks now directly store a `CallContext` which is the source of truth
for borrow tracking for that call, and it does not move from this
location. The store `CallContexts` is now deleted in favor of updating
the `Option<ConcurrentState>` in the store to be an `enum` of either
concurrent state or a stack. In this manner the old stack-based
structure is still used sometimes, but it's impossible to reach when
concurrency is enabled.

* Entry to the host from guests now reliably pushes a `HostTask` into
the store. Previously where a frame were always pushed into a
`CallContext` a `HostTask` is pushed into the store. This is still
expected to be a bit too expensive for cheap host calls, but it
doesn't meaningfully change the performance profile of before.

* The `resource_enter_call` and `resource_exit_call` libcalls have been
removed. These are now folded into the `enter_sync_call` and
`exit_sync_call` libcalls. Emission of these hooks has been updated
accordingly. The concept of entering a call more generally has been
removed. This is more formally known in the async world as a task
starting, so the task creation is now responsible for the demarcation
of entering a call. Additionally this means that the concept of
exiting a call has somewhat gone away. Instead this method was renamed
to `validate_scope_exit` which double-checks that a borrow-scope can
be exited but doesn't actually remove the task. Task removal is
deferred to preexisting mechanisms.

* Management of a `GuestTask`'s previous `Option<CallContext>` field,
for example taking/restoring and pushing/popping onto `CallContexts`
is now all gone. All related code is outright deleted as the
`GuestTask`'s now non-optional `CallContext` field is the source of truth.

* The `ConcurrentState` structure now stores a `CurrentThread` enum
instead of `Option<QualifiedThreadId>`. This represents how the
currently executing thread could be a host thread, not just a guest
thread, which is required for borrow-tracking.

* `HostTask` creation in `poll_and_block` and `first_poll`, the two main
entrypoints of async host tasks when called by the guest, is now
externalized from these functions. Instead these functions assume that
the currently running thread is already a `HostTask` of some kind.

* In `poll_and_block` the host's result is no longer stored in the guest
task but in the host task instead.

Overall this enables the `*.wast` test for #12510 to fix the original
issue. This then adds new tests to ensure that cleanup of various
constructs happens appropriately, such as cancelling a host task should
clean up its associated resources. Additionally synchronously calling an
async host task no longer leaks resources in a `Store` and should
properly clean up everything.

There is still more work to do in this area (e.g. #12544) but that's
going to be deferred to a future PR at this point.

Closes #12510

prtest:full

* Review comments/CI fixes

show more ...

Consolidate component-related store data (#12549)

Instead of having `#[cfg]` within `store.rs` move it all to
`component/store.rs` to cut down on `#[cfg]`. It's a bit awkward in some
places trying t

Consolidate component-related store data (#12549)

Instead of having `#[cfg]` within `store.rs` move it all to
`component/store.rs` to cut down on `#[cfg]`. It's a bit awkward in some
places trying to borrow a bunch of fields at once, but it's not the end
of the world.

show more ...

Move `ResourceTables` constructors to `StoreOpaque` (#12546)

* Move `ResourceTables` constructors to `StoreOpaque`

This is intended to consolidate where they're constructed, provide
access to the h

Move `ResourceTables` constructors to `StoreOpaque` (#12546)

* Move `ResourceTables` constructors to `StoreOpaque`

This is intended to consolidate where they're constructed, provide
access to the host data at all times, and pave the way for a future
refactoring here to touch fewer lines.

* Fix a cfg

show more ...

Check may-leave flags in trampolines, not Rust (#12427)

This commit moves all may-leave flag handling into compiled trampolines
rather than doing this in Rust. This means it can't be forgotten on th

Check may-leave flags in trampolines, not Rust (#12427)

This commit moves all may-leave flag handling into compiled trampolines
rather than doing this in Rust. This means it can't be forgotten on the
Rust side of things and will be slightly more efficient to boot. This
then additionally exempts some intrinsics from checking may-leave since
Wasmtime erroneously checked when it shouldn't have.

Closes #12397
Closes #12403

show more ...

Introduce `wasmtime::Store::try_new`, which handles OOM (#12415)

* Introduce `wasmtime::Store::try_new`, which handles OOM

`Store::new` is an infallible constructor, so there is not a direct way to

Introduce `wasmtime::Store::try_new`, which handles OOM (#12415)

* Introduce `wasmtime::Store::try_new`, which handles OOM

`Store::new` is an infallible constructor, so there is not a direct way to make
it return an error on OOM. Additionally, it is one of the most-used functions in
the Wasmtime embedder API, so changing its signature to return a `Result` is a
non-starter -- it would cause way too much pain. So instead we define
`Store::try_new` which returns a `Result` and make `Store::new` call and unwrap
that new constructor.

Part of https://github.com/bytecodealliance/wasmtime/issues/12069

* update disas tests and fix winch

* Disable concurrency support in `Store::try_new` OOM test

* Add attributes that were lost in rebase

show more ...

Refactor how concurrency support is enabled in a `Store` (#12416)

* Document panics from using CM async machinery when CM async is not enabled

* Refactor how concurrency support is enabled in a `St

Refactor how concurrency support is enabled in a `Store` (#12416)

* Document panics from using CM async machinery when CM async is not enabled

* Refactor how concurrency support is enabled in a `Store`

This commit is an extension/refactor of #12377 and #12379. Notably this
decouples the runtime behavior of Wasmtime from enabled/disabled
WebAssembly proposals. This enables the `wasmtime serve` subcommand, for
example, to continue to disallow component-model-async by default but
continue to use `*_concurrent` under the hood.

Specifically a new `Config::concurrency_support` knob is added. This is
plumbed directly through to `Tunables` and takes over the preexisting
`component_model_concurrency` field. This field configures whether
tasks/etc are enabled at runtime for component-y things. The default
value of this configuration option is the same as `cfg!(feature =
"component-model-async")`, and this field is required if
component-model-async wasm proposals are enabled. It's intended that
eventually this'll affect on-by-default wasm features in Wasmtime
depending if the support is compiled in.

This results in a subtle shift in behavior where component-model-async
concurrency is used by default now because the feature is turned on by
default, even though the wasm features are off-by-default. This required
adjusting a few indices expected in runtime tests due to tasks/threads
being allocated in index spaces.

Finally, this additionally denies access at runtime to
`Linker::*_concurrent` when concurrent support is disabled as otherwise
the various runtime data structures won't be initialized and panics will
happen.

Closes #12393

* Add a `-Wconcurrency-support` CLI flag

Used to update disas tests to show that, when disabled, old codegen
quality is preserved

* Ungate `Config` flag

* Review comments

---------

Co-authored-by: Nick Fitzgerald <[email protected]>

show more ...

consistently create thread and task when entering component instance (#12379)

* consistently create thread and task when entering component instance

Previously, we weren't creating a new thread or

consistently create thread and task when entering component instance (#12379)

* consistently create thread and task when entering component instance

Previously, we weren't creating a new thread or task in all cases when entering
a component instance, even when component model async features were enabled. In
particular, entering an instance via a sync-to-sync, guest-to-guest adapter, via
`Linker::instantiate[_async]`, or via `[Typed]Func::call` all skipped creating a
thread or task, creating panics and/or instance mismatches in certain cases.

This commit addresses all those cases and also adds assertions to all CM async
intrinsics to verify that the caller instance matches the most-recently-pushed
task. Note that we still skip pushing and popping threads and tasks if no CM
async features are enabled in the `Config`.

In order to populate the `GuestTask::instance` field for tasks created as part
of `Linker::instantiate[_async]` calls, I had to add a
`RuntimeComponentInstanceIndex` field to `GlobalInitializer::InstantiateModule`
and friends so it would be available when needed.

While testing this, I uncovered and fixed a couple of related issues:

- We weren't checking the `may_leave` flag when guest-to-guest calling a resource destructor
- We weren't checking whether a subtask was ready to delete (e.g. that no threads were still running) before attempting to delete it while deleting its supertask

* fix warnings when component-model-async feature disabled

* address review feedback

* push a task when calling a resource dtor host-to-guest

* add tests which call `thread.index` from realloc and post-return functions

...and assert that the indexes match as expected.

Getting the post-return test to pass required moving the call to
`StoreOpaque::exit_sync_call` to after the post-return function is called.

show more ...

Remove need for explicit `Config::async_support` knob (#12371)

* Refactor component model host function definitions

Push the `async`-ness down one layer.

* Remove need for explicit `Config::async

Remove need for explicit `Config::async_support` knob (#12371)

* Refactor component model host function definitions

Push the `async`-ness down one layer.

* Remove need for explicit `Config::async_support` knob

This commit is an attempt to step towards reconciling "old async" and
"new async" in Wasmtime. The old async style is the original async
support in Wasmtime with `call_async`, `func_wrap_async`, etc, where the
main property is that the store is "locked" during an async operation.
Put another way, a store can only execute at most one async operation at
a time. This is in contrast to "new async" support in Wasmtime with the
component-model-async (WASIp3) support, where stores can have more than
one async operation in flight at once.

This commit does not fully reconcile these differences, but it does
remove one hurdle along the way: `Config::async_support`. Since the
beginning of Wasmtime this configuration knob has existed to explicitly
demarcate a config/engine/store as "this thing requires `async` stuff
internally." This has started to make less and less sense over time
where the line between sync and async has become more murky with WASIp3
where the two worlds comingle. The goal of this commit is to deprecate
`Config::async_support` and make the function not actually do anything.

In isolation this can't simply be done, however, because there are many
load-bearing aspects of Wasmtime that rely on this `async_support` knob.
For example once epochs + yielding are enabled it's required that all
Wasm is executed on a fiber lest it hit an epoch and not know how to
yield. That means that this commit is not a simple removal of
`async_support` but instead a refactoring/rearchitecting of how async is
used internally within Wasmtime. The high-level ideas within Wasmtime
now are:

* A `Store` has a "requires async" boolean stored within it.
* All configuration options which end up requiring async, such as
yielding with epochs, turn this boolean on.
* Creation of host functions which use async
(e.g. `func_wrap_{async,concurrent}`) will also turn this option on.
* Synchronous API entrypoints into Wasmtime ensure that this boolean is
disabled.
* Asynchronous APIs are usable at any time.

This means that the concept of an async store vs a sync store is now
gone. All stores are equally capable of executing sync/async, and the
change now is that dynamically some stores will require that async is
used with certain configuration. Additionally all panicking conditions
around `async_support` have been converted to errors instead. All
relevant APIs already returned an error and things are murky enough now
that it's not necessarily trivial to get this right at the embedder
level. In the interest of avoiding panics all detected async mismatches
are now first-class `wasmtime::Error` values.

The end result of this commit is that `Config::async_support` is a
deprecated `#[doc(hidden)]` function that does nothing. While many
internal changes happened as well as having new tests for all this sort
of behavior this is not expected to have a great impact on external
consumers. In general a deletion of `async_support(true)` is in theory
all that's required. This is intended to make it easier to think about
async/sync/etc in the future with WASIp3 and eventually reconcile
`func_wrap_async` and `func_wrap_concurrent` for example. That's left
for future refactorings however.

prtest:full

* Review comments

* Fix CI failures

show more ...

Do not re-export `anyhow!` in the `wasmtime::prelude` (#12298)

We are trying to move to `format_err!` instead.

add missing may-block checks for sync-to-sync guest-to-guest calls (#12282)

* add missing may-block checks for sync-to-sync guest-to-guest calls

Previously, we weren't updating or checking the may-

add missing may-block checks for sync-to-sync guest-to-guest calls (#12282)

* add missing may-block checks for sync-to-sync guest-to-guest calls

Previously, we weren't updating or checking the may-block status of a task
across sync-to-sync, guest-to-guest calls, meaning we were allowing blocking in
cases we shouldn't have.

This fixes that by adding a new `task_may_block` field to `VMComponentContext`,
plus code to update it every time we switch threads or do a sync-to-sync,
guest-to-guest call. We use that field as the source of truth about whether a
blocking operation is permitted.

I've updated various tests to match, and Luke has an item on his to-do list to
add sad-path coverage for various cases to the upstream `component-model` test
suite.

* address review feedback and fix component_instance_size_limit test

* remove `TaskMayBlock` type per review feedback

* bless disas tests

show more ...

allow recursive Wasm invocation from concurrent host functions (#12152)

* allow recursive Wasm invocation from concurrent host functions

The core changes here are:

- remove an unnecessary assertio

allow recursive Wasm invocation from concurrent host functions (#12152)

* allow recursive Wasm invocation from concurrent host functions

The core changes here are:

- remove an unnecessary assertion from `concurrent::prepare_call`
- track instance states (e.g. backpressure, etc.) on a per `(Instance, RuntimeComponentInstanceIndex)` basis
- both parts of that key are needed now that concurrent state is tracked on a per-store basis rather than a per-instance basis since `RuntimeComponentInstanceIndex`es are not globally unique

While discussing the above with Alex, we realized the use of a `HashMap` to
track per-instance states was both pessimal and unnecessary. Consequently, I've
folded that state into `ComponentInstance::instance_handle_tables`, renaming it
to `instance_states`. That involved a fair amount of code churn, but doesn't
change behavior except as described in the second bullet point above.

Thanks to Alex for the test case!

Fixes #12098

Co-authored-by: Alex Crichton <[email protected]>
Signed-off-by: Joel Dice <[email protected]>

* use new `RuntimeInstance` type instead of tuples

Signed-off-by: Joel Dice <[email protected]>

---------

Signed-off-by: Joel Dice <[email protected]>
Co-authored-by: Alex Crichton <[email protected]>

show more ...

Debug: create private code memories per store when debugging is enabled. (#12051)

* Debug: create private code memories per store when debugging is enabled.

This will allow patching code to implem

Debug: create private code memories per store when debugging is enabled. (#12051)

* Debug: create private code memories per store when debugging is enabled.

This will allow patching code to implement e.g. breakpoints. (That is,
for now the copies are redundant, but soon they will not be.)

This change follows the discussion [here] and offline to define a few
types that better encapsulate the distinction we want to enforce.
Basically, there is almost never a bare `CodeMemory`; they are always
wrapped in an `EngineCode` or `StoreCode`, the latter being a per-store
instance of the former. Accessors are moved to the relevant place so
that, for example, one cannot get a pointer to a Wasm function's body
without being in the context of a `Store` where the containing module
has been registered. The registry then returns a `ModuleWithCode` that
boxes up a `Module` reference and `StoreCode` together for cases where
we need both the metadata from the module and the raw code to derive
something.

The only case where we return raw code pointers to the `EngineCode`
directly have to do with Wasm-to-array trampolines: in some cases, e.g.
`InstancePre` pre-creating data structures with references to host
functions, it breaks our expected performance characteristics to make
the function pointers store-specific. This is fine as long as the
Wasm-to-array trampolines never bake in direct calls to Wasm functions;
the strong invariant is that Wasm functions never execute from
`EngineCode` directly. Some parts of the component runtime would also
have to be substantially refactored if we wanted to do away with this
exception.

The per-`Store` module registry is substantially refactored in this PR.
I got rid of the modules-without-code distinction (the case where a
module only has trampolines and no defined functions still works fine),
and organized the BTreeMaps to key on start address rather than end
address, which I find a little more intuitive (one then queries with the
dual to the range -- 0-up-to-PC and last entry found).

[here]: https://github.com/bytecodealliance/wasmtime/pull/12051#pullrequestreview-3493711812

* Review feedback: do not assume a reasonable code alignment; error when it cannot be known

* Review feedback: fail properly in profiler when we are cloning code

* Fix guest-profiler C API.

* Review feedback: make private-code representation impossible in non-debugging-support builds.

* Add TODO comment referencing issue for cloning only .text.

* clang-format

* Review feedback: add back Component::image_range.

* Review feedback: error on registering profiling metadata when debug is enabled.

* rustfmt

* Remove early bail on profiling-data registration when debugging is enabled: this always happens so we cannot error out.

show more ...

Prevent using shared memories with `Memory` (#12022)

* Prevent using shared memories with `Memory`

This commit fixes a few issues where it was possible to represent a wasm
shared linear memory with

Prevent using shared memories with `Memory` (#12022)

* Prevent using shared memories with `Memory`

This commit fixes a few issues where it was possible to represent a wasm
shared linear memory with the `wasmtime::Memory` type. This is not sound
because `wasmtime::Memory` provides safe Rust access to the bytes where
that is not possible with wasm shared memories. Shared memories in Rust
must be represented by `SharedMemory`, not `wasmtime::Memory`.

Specifically this commit prevents two vectors of this happening:

1. `Memory::new` now requires that the memory type specified is
non-shared. Instead `SharedMemory::new` must be used instead.

2. Core dumps now skip over shared memories when iterating over all
memories in the store. Supporting shared memories is left to a future
feature request for now.

* CI fixes

show more ...

Enable borrowing components and stores mutable at the same time (#11987)

* Use `OptionsIndex` more internally in components

This updates various runtimes bits for components to use `OptionsIndex`

Enable borrowing components and stores mutable at the same time (#11987)

* Use `OptionsIndex` more internally in components

This updates various runtimes bits for components to use `OptionsIndex`
more aggressively and ultimately deletes the old `Options` type. The
`Options` type is a heavyweight package of all possible options which is
effectively a duplication of what `OptionsIndex` points to, so that's
removed in favor of directly accessing options.

* Enable borrowing components and stores mutable at the same time

This commit adds a new, safe, function to the `Instance` type for
internal use in Wasmtime which enables simultaneously borrowing the
`Component`-within-the-`Instance` as well as the original store at the
same time. This is not possible to do in safe Rust when the store is
mutable and must be implemented with `unsafe` code.

The motivation for this commit is performance. In #11974 it was
discovered that the addition of a single `Arc::clone` was enough to
reduce throughput in the embedding by 20%. While `Arc::clone` is
generally cheap I can see how a "contended" `Arc::clone` could get quite
expensive. This particular benchmark was running multiple instances of
the same component across multiple threads which were all doing many
host calls. Each host call, after the refactoring of #10959, contained
an `Arc::clone` to the component itself. This in turn led to many
threads constantly incrementing/decrementing the `Arc::clone` count of
the same `Arc` instance.

At a high level this clone is not necessary. The component lives within
the store and cannot be mutated/deleted during execution. Safe Rust,
however, forbids access to the component and mutably using the store at
the same time. Thus, this helper function enters the picture. The goal
here is to remove the `Arc::clone` calls without requiring major surgery
or such to refactor the implementations of various functions throughout
Wasmtime. The `unsafe` block used to implement this function documents
more rationale as to why this should be safe.

* Update crates/wasmtime/src/runtime/component/instance.rs

Co-authored-by: Nick Fitzgerald <[email protected]>

---------

Co-authored-by: Nick Fitzgerald <[email protected]>

show more ...

Implement unsafe intrinsics for compile-time builtins (#11825)

* Implement unsafe intrinsics for compile-time builtins

This commit adds the extremely unsafe
`wasmtime::CodeBuilder::expose_unsafe_in

Implement unsafe intrinsics for compile-time builtins (#11825)

* Implement unsafe intrinsics for compile-time builtins

This commit adds the extremely unsafe
`wasmtime::CodeBuilder::expose_unsafe_intrinsics` method. When enabled, the Wasm
being compiled is given access to special imports that correspond to direct,
unchecked and unsandboxed, native load and store operations. These intrinsics
are intended to be used for implementing fast, inline-able versions of WASI
interfaces that are special-cased to a particular host embedding, for example.

Compile-time builtins, as originally described in [the
RFC](https://github.com/bytecodealliance/rfcs/pull/43), are basically made up of
three parts:

1. A function inliner
2. Unsafe intrinsics
3. Component composition to encapsulate the usage of unsafe intrinsics in a safe
interface

Part (1) has been implemented in Wasmtime and Cranelift for a little while
now (see `wasmtime::Config::compiler_inlining`). This commit is part (2). After
this commit lands, part (3) can be done with `wac` and `wasm-compose`, although
follow up work is required to make the developer experience nicer and more
integrated into Wasmtime so that the APIs can look like those proposed in the
RFC.

* fill out some more docs

* fix non component model builds

* start filling out the doc example

* Factor abi params/returns out; truncate/extend pointers

* Compile unsafe intrinsics on winch as well

* prtest:full

* have the macro define the signature

* ignore tests in MIRI because MIRI can't compile Wasm

* juggle pointer provenance in `Store::data[_mut]`

* add a test for store data provenance and also fix it

* use `VmPtr` for the store data pointer

* finish writing unsafe intrinsics example

* fix up docs and rules around only accessing data from `T` in a `Store<T>`

* Only reserve space for the intrinsics' `VMFuncRef`s if they are in use

* use dangling pointers instead of options

* Rename `StoreInner::data` to `data_no_provenance` and fix some accesses to use the method accessors

* Add comments about the provenance juggling inside `StoreInner::data[_mut]`

* only compile intrinsics that are used

Turns out we don't need to add phases, we already have the info available to do
this.

* fix duplicate symbol names

show more ...

disallow exiting a component instance during a post-return call (#11688)

* disallow exiting a component instance during a post-return call

This is a relatively recent change to the spec.

In order

disallow exiting a component instance during a post-return call (#11688)

* disallow exiting a component instance during a post-return call

This is a relatively recent change to the spec.

In order to check the `may_leave` flag in all the relevant intrinsics, I had to
plumb the caller `RuntimeComponentInstanceIndex` through a bunch of trampolines
that didn't previously need it, hence the large diff.

Note that I've added a slightly tweaked version of `trap-in-post-return.wast`
and left the upstream version disabled in `test-util/src/wast.rs` due to #11683.

Fixes #11676.

Signed-off-by: Joel Dice <[email protected]>

* fix test regressions

Signed-off-by: Joel Dice <[email protected]>

---------

Signed-off-by: Joel Dice <[email protected]>

show more ...

Make table/memory creation async functions (#11470)

* Make core instance allocation an `async` function

This commit is a step in preparation for #11430, notably core instance
allocation, or `Store

Make table/memory creation async functions (#11470)

* Make core instance allocation an `async` function

This commit is a step in preparation for #11430, notably core instance
allocation, or `StoreOpaque::allocate_instance` is now an `async fn`.
This function does not actually use the `async`-ness just yet so it's a
noop from that point of view, but this propagates outwards to enough
locations that I wanted to split this off to make future changes more
digestable.

Notably some creation functions here such as making an `Instance`,
`Table`, or `Memory` are refactored internally to use this new `async`
function. Annotations of `assert_ready` or `one_poll` are used as
appropriate as well.

For reference this commit was benchmarked with our `instantiation.rs`
benchmark in the pooling allocator and shows no changes relative to the
original baseline from before-`async`-PRs.

* Make table/memory creation `async` functions

This commit is a large-ish refactor which is made possible by the many
previous refactorings to internals w.r.t. async-in-Wasmtime. The end
goal of this change is that table and memory allocation are both `async`
functions. Achieving this, however, required some refactoring to enable
it to work:

* To work with `Send` neither function can close over `dyn VMStore`.
This required changing their `Option<&mut dyn VMStore>` arugment to
`Option<&mut StoreResourceLimiter<'_>>`
* Somehow a `StoreResourceLimiter` needed to be acquired from an
`InstanceAllocationRequest`. Previously the store was stored here as
an unsafe raw pointer, but I've refactored this now so
`InstanceAllocationRequest` directly stores `&StoreOpaque` and
`Option<&mut StoreResourceLimiter>` meaning it's trivial to acquire
them. This additionally means no more `unsafe` access of the store
during instance allocation (yay!).
* Now-redundant fields of `InstanceAllocationRequest` were removed since
they can be safely inferred from `&StoreOpaque`. For example passing
around `&Tunables` is now all gone.
* Methods upwards from table/memory allocation to the
`InstanceAllocator` trait needed to be made `async`. This includes new
`#[async_trait]` methods for example.
* `StoreOpaque::ensure_gc_store` is now an `async` function. This
internally carries a new `unsafe` block carried over from before with
the raw point passed around in `InstanceAllocationRequest`. A future
PR will delete this `unsafe` block, it's just temporary.

I attempted a few times to split this PR up into separate commits but
everything is relatively intertwined here so this is the smallest
"atomic" unit I could manage to land these changes and refactorings.

* Shuffle `async-trait` dep

* Fix configured build

show more ...

Gut `vm::Export` to mostly be `crate::Extern` (#11229)

* Remove `Table::from_wasmtime_table`

This commit removes the unsafe function `Table::from_wasmtime_table`.
This goes a bit further and remove

Gut `vm::Export` to mostly be `crate::Extern` (#11229)

* Remove `Table::from_wasmtime_table`

This commit removes the unsafe function `Table::from_wasmtime_table`.
This goes a bit further and removes `ExportTable` entirely as well which
means that table lookup on a `vm::Instance` directly returns a
`wasmtime::Table` without any need to translate back-and-forth.

* Remove `Tag::from_wasmtime_tag`

Like the previous commit, but for tags.

* Remove `Global::from_wasmtime_global`

Like the previous commit, but for globals.

* Remove `Memory::from_wasmtime_memory`

Like the previous commit, but for memories.

* Remove `Func::from_wasmtime_function`

Like previous commits, but for functions.

* Fix lints

* Fill out missing safety comment

* Review comments

show more ...

Fix another case of Miri unsoundness (#11056)

This commit fixes another issue we've discovered in the wasip3
prototyping repository about a code pattern in wasm which Miri flags as
un-sound. Specifi

Fix another case of Miri unsoundness (#11056)

This commit fixes another issue we've discovered in the wasip3
prototyping repository about a code pattern in wasm which Miri flags as
un-sound. Specifically what happened was:

* Invocation of WebAssembly went through `VMFuncRef::array_call` which
takes a `&self` parameter.

* Inside of WebAssembly though a `ref.func` instruction, or anything
else that references the original exported function, will
re-initialize the `VMFuncRef` which writes the `&self` up the stack,
which is not sound.

Fixing this required changing the signature of `array_call` from `&self`
to `me: NonNull<VMFuncRef>`, and the signature was already `unsafe` so
this is a new unsafe contract for that signature.

In fixing this, however, it was discovered that a mistake was made
in #10943 where some internal functions for re-initializing a
`VMFuncRef` relied on the previous signature of `&mut self` but that PR
switche to `&self`. This PR corrects these signatures to `Pin<&mut Self>`
and then plumbs around the necessary changes, notably causing some
refactoring in component-related bits.

show more ...

Use `Pin<&mut ComponentInstance>` (#11042)

This commit is the continuation of #10943 for component instances. The
allocation/vmctx infrastructure was additionally refactored to be shared for
both co

Use `Pin<&mut ComponentInstance>` (#11042)

This commit is the continuation of #10943 for component instances. The
allocation/vmctx infrastructure was additionally refactored to be shared for
both core and component instances since they behave the exact same way anyway.
This further enables sharing various methods like `vmctx_plus_offset` which are
pretty unsafe internally.

Like #10943 this necessitated removal of `Index` implementations because
`IndexMut` is not compatible with the returned type being `Pin<&mut T>`
so they were replaced by inherent `get` and `get_mut` methods on the
component instance id type.

Closes #10933

show more ...