Debugging: refactor stack frame cursor into frame handle abstraction. (#12566)

* Debugging: refactor stack frame cursor into frame handle abstraction.

This addresses some of the issues described #1

Debugging: refactor stack frame cursor into frame handle abstraction. (#12566)

* Debugging: refactor stack frame cursor into frame handle abstraction.

This addresses some of the issues described #12486: we need the ability
to keep a handle to a stack frame as long as execution is frozen, and
keep multiple of these handles around, alongside the `Store`, without
any handle directly holding a borrow of the store.

The frame handles work by means of an "execution version" scheme: the
idea is that whenever any execution resumes in a given store, all
handles to existing frames could be invalidated, but if no such
execution occurs, all handles should still be valid. A tuple of
(globally unique for process lifetime) store ID, and execution version
within that store, should be sufficient to uniquely identify any
frozen-stack period during execution. This accomplishes cheap handle
invalidation without the need to track existing handles.

This PR also implements a cache of parsed frame-table data. Previously
this was lazily parsed by the cursor as it walked up a stack, but with
multiple handles hanging around, and with handles meant to be cheap to
hold and clone, and with handles being invalidated eagerly, it makes
much more sense to persist this parsed metadata at the `Store` level.
(It cannot persist at the `Engine` level because PCs are local per
store.)

* Re-bless disas tests (offsets in VMStoreContext changed).

* Handle invalidation tests.

* Review comments, and make API return `Result`s rather than panic'ing on stale handles.

* Review feedback.

* Doc-comment link fix.

* Review feedback.

* cfg-gate Activation method to `debug` feature only.

* Fix unused-import warning in no-debug cfg.

* Fix doc link (again, after rename from latest feedback).

show more ...

add missing may-block checks for sync-to-sync guest-to-guest calls (#12282)

* add missing may-block checks for sync-to-sync guest-to-guest calls

Previously, we weren't updating or checking the may-

add missing may-block checks for sync-to-sync guest-to-guest calls (#12282)

* add missing may-block checks for sync-to-sync guest-to-guest calls

Previously, we weren't updating or checking the may-block status of a task
across sync-to-sync, guest-to-guest calls, meaning we were allowing blocking in
cases we shouldn't have.

This fixes that by adding a new `task_may_block` field to `VMComponentContext`,
plus code to update it every time we switch threads or do a sync-to-sync,
guest-to-guest call. We use that field as the source of truth about whether a
blocking operation is permitted.

I've updated various tests to match, and Luke has an item on his to-do list to
add sad-path coverage for various cases to the upstream `component-model` test
suite.

* address review feedback and fix component_instance_size_limit test

* remove `TaskMayBlock` type per review feedback

* bless disas tests

show more ...

Update upstream wasm spec test suite submodule (#11926)

* Update upstream wasm spec test suite submodule

Juggle some configuration options here and there to accommodate all of
the new tests.

* Fix

Update upstream wasm spec test suite submodule (#11926)

* Update upstream wasm spec test suite submodule

Juggle some configuration options here and there to accommodate all of
the new tests.

* Fix wasmtime-fuzzing compile

* Fix miri script for table.wast

* Fix a miri provenance issue

show more ...

Implement unsafe intrinsics for compile-time builtins (#11825)

* Implement unsafe intrinsics for compile-time builtins

This commit adds the extremely unsafe
`wasmtime::CodeBuilder::expose_unsafe_in

Implement unsafe intrinsics for compile-time builtins (#11825)

* Implement unsafe intrinsics for compile-time builtins

This commit adds the extremely unsafe
`wasmtime::CodeBuilder::expose_unsafe_intrinsics` method. When enabled, the Wasm
being compiled is given access to special imports that correspond to direct,
unchecked and unsandboxed, native load and store operations. These intrinsics
are intended to be used for implementing fast, inline-able versions of WASI
interfaces that are special-cased to a particular host embedding, for example.

Compile-time builtins, as originally described in [the
RFC](https://github.com/bytecodealliance/rfcs/pull/43), are basically made up of
three parts:

1. A function inliner
2. Unsafe intrinsics
3. Component composition to encapsulate the usage of unsafe intrinsics in a safe
interface

Part (1) has been implemented in Wasmtime and Cranelift for a little while
now (see `wasmtime::Config::compiler_inlining`). This commit is part (2). After
this commit lands, part (3) can be done with `wac` and `wasm-compose`, although
follow up work is required to make the developer experience nicer and more
integrated into Wasmtime so that the APIs can look like those proposed in the
RFC.

* fill out some more docs

* fix non component model builds

* start filling out the doc example

* Factor abi params/returns out; truncate/extend pointers

* Compile unsafe intrinsics on winch as well

* prtest:full

* have the macro define the signature

* ignore tests in MIRI because MIRI can't compile Wasm

* juggle pointer provenance in `Store::data[_mut]`

* add a test for store data provenance and also fix it

* use `VmPtr` for the store data pointer

* finish writing unsafe intrinsics example

* fix up docs and rules around only accessing data from `T` in a `Store<T>`

* Only reserve space for the intrinsics' `VMFuncRef`s if they are in use

* use dangling pointers instead of options

* Rename `StoreInner::data` to `data_no_provenance` and fix some accesses to use the method accessors

* Add comments about the provenance juggling inside `StoreInner::data[_mut]`

* only compile intrinsics that are used

Turns out we don't need to add phases, we already have the info available to do
this.

* fix duplicate symbol names

show more ...

Remove `AllCallFunc` (#11694)

* Remove `AllCallFunc`

And add `FuncKeyKind` and `FuncKeyNamespace` types.

Split out from https://github.com/bytecodealliance/wasmtime/pull/11630

* fix warning and d

Remove `AllCallFunc` (#11694)

* Remove `AllCallFunc`

And add `FuncKeyKind` and `FuncKeyNamespace` types.

Split out from https://github.com/bytecodealliance/wasmtime/pull/11630

* fix warning and disriminant gap

* add module arg to `Module::new()` calls in tests

show more ...

Replace setjmp/longjmp usage in Wasmtime (#11592)

Since Wasmtime's inception it's used the `setjmp` and `longjmp`
functions in C to implement handling of traps. While this solution was
easy to imple

Replace setjmp/longjmp usage in Wasmtime (#11592)

Since Wasmtime's inception it's used the `setjmp` and `longjmp`
functions in C to implement handling of traps. While this solution was
easy to implement, relatively portable, and performant enough, there are
a number of downsides that have evolved over time to make this an
unattractive approach in the long run:

* Using `setjmp` fundamentally requires using C because Rust does not
understand a function that returns twice. It's fundamentally unsound
to invoke `setjmp` in Rust meaning that Wasmtime has forever needed a
C compiler configured and set up to build. This notably means that
`cargo check` cannot check other targets easily.

* Using `longjmp` means that Rust function frames are unwound on the
stack without running destructors. This is a dangerous operation of
which we get no protection from the compiler about. Both frames
entering wasm and frames exiting wasm are all skipped. Absolutely
minimizing this has been beneficial for portability to platforms such
as Pulley.

* Currently the no_std implementation of Wasmtime requires embedders to
provide `wasmtime_{setjmp,longjmp}` which is a thorn in the side of
what is otherwise a mostly entirely independent implementation of
Wasmtime.

* There is a performance floor to using `setjmp` and `longjmp`. Calling
`setjmp` requires using C but Wasmtime is otherwise written in Rust
meaning that there's a Rust->C->Rust->Wasm boundary which
fundamentally can't be inlined without cross-language LTO which is
difficult to configure.

* With the implementation of the WebAssembly exceptions proposal
Wasmtime now has two means of unwinding the stack. Ideally Wasmtime
would only have one, and the more general one is the method of
exceptions.

* Jumping out of a signal handler on Unix is tricky business. While
we've made it work it's generally most robust of the signal handler
simply returns which it now does.

With all of that in mind the purpose of this commit is to replace the
setjmp/longjmp mechanism of handling traps with the recently implemented
support for exceptions in Cranelift. That is intended to resolve all of
the above points in one swoop.

One point in particular though that's nice about setjmp/longjmp is that
unwinding the stack on a trap is an O(1) operation. For situations such
as stack overflow that's a particularly nice property to have as we can
guarantee embedders that traps are a constant time (albeit somewhat
expensive with signals) operation. Exceptions naively require unwinding
the entire stack, and although frame pointers mean we're just traversing
a linked list I wanted to preserve the O(1) property here nonetheless.
To achieve this a solution is implemented where the array-to-wasm
(host-to-wasm) trampolines setup state in `VMStoreContext` so looking up
the current trap handler frame is an O(1) operation. Namely the sp/fp/pc
values for a `Handler` are stored inline.

Implementing this feature required supporting
relocations-to-offsets-in-functions which was not previously supported
by Wasmtime. This required Cranelift refactorings such as #11570, #11585,
and #11576. This then additionally required some more refactoring in
this commit which was difficult to split out as it otherwise wouldn't be
tested.

Apart from the relocation-related business much of this change is about
updating the platform signal handlers to use exceptions instead of
longjmp to return. For example on Unix this means updating the
`ucontext_t` with register values that the handler specifies. Windows
involves updating similar contexts, and macOS mach ports ended up not
needing too many changes.

In terms of overall performance the relevant benchmark from this
repository, compared to before this commit, is:

sync/no-hook/core - host-to-wasm - typed - nop
time: [10.552 ns 10.561 ns 10.571 ns]
change: [−7.5238% −7.4011% −7.2786%] (p = 0.00 < 0.05)
Performance has improved.

Closes #3927
cc #10923

prtest:full

show more ...

cranelift: stack-switching support (#11003)

* cranelift: stack-switching support

This initial commit represents the "pr2" base commit with
minimal merge conflicts resolved. Due to OOB conflicts, t

cranelift: stack-switching support (#11003)

* cranelift: stack-switching support

This initial commit represents the "pr2" base commit with
minimal merge conflicts resolved. Due to OOB conflicts, this
commit is not functional as-is, but using it as a base in
order to allow for easier reviewing of the delta from
this commit to what will be used for the PR against upstream.

Co-authored-by: Daniel Hillerström <[email protected]>
Co-authored-by: Paul Osborne <[email protected]>

* cranelift: stack-switching updates pass 1

This first set of changes updates the base pr in order to
compiled and pass basic checks (compile, clippy, fmt) with
the biggest part of the change being to eliminate injection
of tracing/assertions in JIT'ed code.

* cranelift: stack-switching: restore original visibility for a few func_environ members

* cranelift: stack-switching conditional compilation

At this point, the only bit we really branch on is what we
do in order to avoid problems tying into wasmtime_environ.
This is basd on the approach and macro used by the gc code for
converting presence/absence of the cranelift feature flag to
cranelift compile time. This is a bit of a half-measure for now
as we still compile most stack-switching code in cranelift, but
this does enough to avoid causing problems with missing definitions
in wasmtime_environ.

* cranelift: avoid "as" casts in stack-switching

Replace either with infallible From or fallible, panicing
TryFrom alternatives where required.

* cranelift: cleanup stack-switching control_effect signatures

After removing emission of runtime trace logging and assertions,
there were several unused parameters. Remove those from the
ControlEffect signatures completely.

* cranelift: rename stack-switching VMArray to VMHostArray

This matches a change to the mirrored runtime type in
the upstream changes.

* stack-switching: fix typo

Co-authored-by: Daniel Hillerström <[email protected]>

* stack-switching: used Index impl for get_stack_slot_data

* stack-switching: use smallvec over vec in several cases

* stack-switching: avoid resumetable naming confusion

* stack-switching: cleanup unused params from unchecked_get_continuation

The extra parameters here used to be used for emitting runtime
assertions, but with those gone we just had unused params
and lifetimes, clean those out.

* stack_switching: simplify store_data_entries assertion

* stack-switching: simplify translate_table_{grow,fill} control flow

* stack-switching: remove translate_resume_throw stub

There's already a stub elsewhere and this is not called, when
exceptions are added and it is time to revisit, this method
can be restored.

* stack-switching: compute control_context_size based on target triple

* stack-switching: VMHostArrayRef updates

Rename VMHostArray -> VMHostArrayRef
Change impl to compute address with offset upfront rather than
on each load.

* stack-switching: move cranelift code to live under func_environ

This matches the directory structure for gc and aids in visibility
for a few members required by stack-switching code in cranelift.

* stack-switching: formatting fix

* stack-switching: reduce visibility on a few additional items

* stack-switching: simplify contobj fatptr con/de-struction

* stack-switching: add disas tests to cover new instructions

* stack-switching: fix layout of VMContObj

In the course of the various runtime updates, the layout of the
runtime VMContObj got switched around. This resulted in failures
when doing certain table operations on continuations.

This change fixes that layout problem and adds some tests with
offsets to avoid the problem. Due to the way that we interact
with the VMContObj in cranelift, we don't use these offsets outside
of the tests.

* Fix formatting of merge conflict resolution

* cranelift: remove ir::function::get_stack_slot_data

This method isn't required as sized_stack_slots is already pub.

* stack-switching: reduce visibility of a couple func_environ methods

* stack-switching: define VMContObj as two words

This change migrates VMContObj and its usages in cranelift and runtime
to work with the VMContObj fat pointer as two words in order to better
target different architectures (still gated to x86_64 for now).

To support this, a size type was plumbed into the builtins function
signature types (as is done for component types) that maps to
usize.

* fixup! stack-switching: define VMContObj as two words

* stack-switching: add stub Val::ContRef

This type is not fully complete until continuation/gc integration
is revisited (#10248) but without these changes, test cases are
now failing on panics as we need some representation of
continuation references in the runtime Val enumeration.

Runtime errors with TODO notes are added for the stubbed
code paths to revisit later.

* fixup! stack-switching: add stub Val::ContRef

* fixup! stack-switching: add stub Val::ContRef

* fixup! stack-switching: define VMContObj as two words

prtest:full

* stack-switching: don't conflate host and target pointer sizes

Disas tests were failing on i686 targeting x86_64 as the size of the
host pointer was leaking into what we were using to do codegen
in a few paths. This patch is a bit of a hack as it seems like
using a generic <T> for T: *mut u8 (as an example) is a bit
questionable. To keep things small, I do a hacky typecheck to map
pointers to the target pointer size here.

* stack-switching: VMHostArray entry sizes based off env PtrSize

Revisiting the previous commit with an approach that should be
less brittle.

---------

Co-authored-by: Frank Emrich <[email protected]>
Co-authored-by: Daniel Hillerström <[email protected]>

show more ...

WebAssembly exception-handling support. (#11326)

* WebAssembly exception-handling support.

This PR introduces support for the [Wasm exception-handling proposal],
which introduces a conventional try

WebAssembly exception-handling support. (#11326)

* WebAssembly exception-handling support.

This PR introduces support for the [Wasm exception-handling proposal],
which introduces a conventional try/catch mechanism to WebAssembly. The
PR supports modules that use `try_table` to register handlers for a
lexical scope; and provides `throw` and `throw_ref` that allocate (in
the first case) and throw exception objects.

This PR builds on top of the work in #10510 for Cranelift-level
exception support, #10919 for an unwinder, and #11230 for exception
objects built on top of GC, in addition a bunch of smaller fix and
enabling PRs around those.

[Wasm exception-handling proposal]: https://github.com/WebAssembly/exception-handling/

prtest:full

* Permit UnwindToWasm to have unused fields in Pulley builds (for now).

* Resolve miri-caught reborrowing issue.

* Ignore exceptions tests in miri for now (Pulley not supported).

* Use wasmtime_test on exceptions tests.

* Get tests passing on pulley platforms

* Add a check to `supports_host` for the generated test and assert
failure also when that is false.
* Remove `pulley_unsupported` test as it falls out of `#[wasmtime_test]`
* Remove `exceptions_store` helper as it falls out of `#[wasmtime_test]`
* Remove miri annotations as they fall out of `#[wasmtime_test]`

* Remove dead import

* Skip some unsupported tests entirely in `#[wasmtime_test]`

If the selected compiler doesn't support the host at all then there's no
need to run it. Actually running it could misinterpret `CraneliftNative`
as "run with pulley" otherwise, so avoid such false negatives.

* Cranelift: dynamic contexts: account for outgoing-args area.

---------

Co-authored-by: Alex Crichton <[email protected]>

show more ...

Minimize lazy allocation of the GC store (#11411)

* Minimize lazy allocation of the GC store

This commit is an effort to minimize the number of entrypoints which
might lazily allocate a GC store. T

Minimize lazy allocation of the GC store (#11411)

* Minimize lazy allocation of the GC store

This commit is an effort to minimize the number of entrypoints which
might lazily allocate a GC store. The is currently done through
`StoreOpaque::gc_store_mut` but this method is very commonly used
meaning that there are many many places to audit for lazily allocating a
GC store. The reason that this needs an audit is that lazy allocation
is an async operation right now that must be on a fiber and is something
I'm looking to fix as part of #11262.

This commit performs a few refactorings to achieve this:

* `gc_store_mut` is renamed to `ensure_gc_store`. This is intended to be
an `async` function in the future and clearly demarcates where lazy
allocation of a GC store is occurring.

* `require_gc_store{,_mut}` is now added which is a pure accessor of the
GC store with no lazy allocation. Most locations previously using
`gc_store_mut` are updated to use this instead.

Documentation is added to store methods to clearly indicate which ones
are allocating and which ones should only be called in a context where
allocation should already have happened.

* Fix configured build

* Relax GC store restrictions in more places

* Review comments on documentation

* Move `ensure_gc_store` calls during instantiation

Instead update `needs_gc_heap` with the tables that are added to a
module and rely on instantiation to create the GC heap.

* Shuffle around some code

* Fix CI and review comments

* Add in a few more i31 cases for externref

show more ...

Remove `cranelift_entity::{Signed, Unsigned}` (#11400)

Use `*::cast_{un,}signed` in the Rust standard library stabilized in
1.87.

Deny `unsafe_op_in_unsafe_fn` in `wasmtime::runtime::vm` (#11312)

* Deny `unsafe_op_in_unsafe_fn` in `wasmtime::runtime::vm`

Slowly expanding this lint to more of the crate.

prtest:full

* Fix lin

Deny `unsafe_op_in_unsafe_fn` in `wasmtime::runtime::vm` (#11312)

* Deny `unsafe_op_in_unsafe_fn` in `wasmtime::runtime::vm`

Slowly expanding this lint to more of the crate.

prtest:full

* Fix lints in custom module

* Fix some lints with miri

* Fix non-VM build

* Fix arm windows

show more ...

Implement exception objects. (#11230)

* WIP: Working exception objects

* Clean build with gc disabled (`cargo check -p wasmtime --no-default-features --features runtime`).

* Review feedback.

* St

Implement exception objects. (#11230)

* WIP: Working exception objects

* Clean build with gc disabled (`cargo check -p wasmtime --no-default-features --features runtime`).

* Review feedback.

* Stub out C-API support.

* Fix Clippy complaints.

* Fix dead-code warning in c-api build.

* Actually fix 27->26 reserved bit rename and test.

* Fix exnref doc-test.

* fix fuzzing build

* fix feature-flagging on Instance::id

* Bless disas test diff due to reserved-bits change.

* Review feedback.

show more ...

Add `#[inline]` to some small functions (#11235)

This adds `#[inline]` to some functions which are otherwise not
available for cross-crate inlining. These functions started being more
heavily used a

Add `#[inline]` to some small functions (#11235)

This adds `#[inline]` to some functions which are otherwise not
available for cross-crate inlining. These functions started being more
heavily used after historical refactorings such as #10877 so this helps
benchmarks which access linear memory in host functions, for example.

show more ...

Delete `vm::Instance::table_grow` (#11216)

This commit is a further inch towards #11179 by removing internal
reliance on reading `VMContext` pointers and casting them to `Pin<&mut
Instance>` in vari

Delete `vm::Instance::table_grow` (#11216)

This commit is a further inch towards #11179 by removing internal
reliance on reading `VMContext` pointers and casting them to `Pin<&mut
Instance>` in various contexts. Notably now only a defined table needs
to be grown, which simplifies the internals of `vm::Instance` ever so
slightly. This is a similar change as #11211 which transitions libcalls
to using `DefinedTableIndex` instead of `TableIndex`.

show more ...

Make `vm::Instance::get_defined_memory` safe (#11211)

This is a small step forward to making `vm::Instance` safe internally.
Notably the `get_defined_memory` helper now returns a safe mutable
refere

Make `vm::Instance::get_defined_memory` safe (#11211)

This is a small step forward to making `vm::Instance` safe internally.
Notably the `get_defined_memory` helper now returns a safe mutable
reference instead of a raw pointer. This is then additionally coupled
with the canonicalization of always working with memories as "instance
plus defined memory index" instead of "instance plus memory index". This
enables removing some unsafe `VMContext` to `Instance` conversion as
well. This change, however, required updating libcalls to special-case
when an imported memory is operated on to load the vmcontext/index in
the libcall itself.

This change notably does not update the `memory_init` libcall just yet
due to the fact that the `DataIndex` is relative to the owning instance
even if the memory is owned by a different instance (e.g. it's
imported). Otherwise this chips away at some of the `unsafe` related to
memory/table management in `vm::Instance`.

show more ...

Enable `allow_attributes_without_reason` (#11195)

* Enable `allow_attributes_without_reason`

This commit enables the `clippy::allow_attributes_without_reason` for
the `wasmtime` crate which previou

Enable `allow_attributes_without_reason` (#11195)

* Enable `allow_attributes_without_reason`

This commit enables the `clippy::allow_attributes_without_reason` for
the `wasmtime` crate which previously forcibly allowed it. The reason
this was allowed was that when the workspace was first migrated the
Wasmtime crate had too many instances that I was willing to fix. I've
now come back around and tried to fix everything.

In short: ideally delete `#[allow]`, otherwise use `#[expect]`,
otherwise use `#[allow]`.

prtest:full

* Adjust some directives

* Fix some warnings

* Fix stack switching size tests on unix

* Don't have a conditional `Drop` impl

* Force `testing_freelist` method to be used

Too lazy to write `#[cfg]`, but not too lazy to write a test.

show more ...

generalize async fiber abstraction (#11114)

* generalize async fiber abstraction

As part of the work implementing the new Component Model async ABI in the
`wasip3-prototyping` repo, I've generalize

generalize async fiber abstraction (#11114)

* generalize async fiber abstraction

As part of the work implementing the new Component Model async ABI in the
`wasip3-prototyping` repo, I've generalized the `FiberFuture` abstraction in
`wasmtime::runtime::store::async_` to support fibers which can either retain
exclusive access to the store across suspend points or release it. The latter
allows the store to be used by the `component-model-async` event loop and/or
other fibers to run before the original fiber resumes, which is the key to
allowing multiple fibers to run concurrently, passing control of the store back
and forth.

In the case of Pulley, the above generalization means we also need to give each
fiber its own `Interpreter` so that multiple concurrent fibers don't clobber
each other's state.

Concretely, this moves a lot of the code out of `async_.rs` and into a new
`fiber.rs` submodule which will be shared with the `component-model-async`
implementation.

This also pulls in a new `StoreToken<T>` utility which has been useful in
`wasip3-prototyping` to safely convert from a `&mut dyn VMStore` to a
`StoreContextMut<'a, T>` when we previously witnessed a conversion in the other
direction.

Note that I've added a `'static` bound to the `VMStore` trait, which simplifies
use of `&mut dyn VMStore`, avoiding thorny lifetime issues.

Signed-off-by: Joel Dice <[email protected]>

* address review feedback

Signed-off-by: Joel Dice <[email protected]>

* fix miri-flagged stacked borrow violation

As part of my earlier effort to unify the fiber abstractions in the `wasmtime`
crate, I changed a `*mut StoreFiber` field to a `&mut StoreFiber`, not realizing
that it resulted in a mutable alias at runtime and thus undefined behavior.
Miri caught it, fortunately.

Signed-off-by: Joel Dice <[email protected]>

* remove unneeded `Send` bounds

Signed-off-by: Joel Dice <[email protected]>

* address more review feedback

Main changes:

- Make `resume_fiber[_raw]` take a `&mut StoreOpaque` parameter to make its unsafe internals easier to reason about, safety-wise.

- Panic if `StoreFiber::drop` is called on an in-progress fiber without having called `StoreFiber::dispose` to gracefully end it first.

- (Re)introduce `FiberFuture`, which closes over a `&mut StoreOpaque` and uses it to call `StoreFiber::dispose` on drop.

This will require a few more changes to make it usable by `concurrent.rs`, but
I'll save those changes for a later PR.

Signed-off-by: Joel Dice <[email protected]>

* address more review feedback

Signed-off-by: Joel Dice <[email protected]>

* update `impl Send For StoreFiber` comment

Signed-off-by: Joel Dice <[email protected]>

* Remove currently-extraneous `Result<()>` from fibers

May be needed for concurrent bits, but for now not necessary.

* Use safe pointers instead of raw pointers

It's predicted Miri won't like this, but for now in-repo it's ok with it.

* Fold more responsibility into `resume_fiber_raw`

Remove the need for the function entirely and replace it with
`resume_fiber`.

* Remove channels from async fibers

Can use stack-based closures/results to transmit the result instead of
needing a channel.

* Fold `on_fiber_raw` directly into `on_fiber`

The `on_fiber` function is small enough it should be possible to do so.

* Don't use `Option` in `FiberFuture`

Leave the fiber non-optional at-rest so it's always available for the
destructor.

* Fold `suspend` functions together

Small shims, not otherwise public at this time, so remove a layer of
indirection.

* Move stack limit management to `FiberResumeState`

Helps remove some raw pointers that are held for a long time within
`AsyncCx`

* add some doc comments to `fiber.rs`

Signed-off-by: Joel Dice <[email protected]>

* update `fiber.rs` and friends to match CM async requirements

This adds a `resolve_or_release` function, which `Instance::resume_fiber` will
use when current `concurrent.rs` stub is replaced by a real implementation.

Signed-off-by: Joel Dice <[email protected]>

* fix non-component-model-async build warnings

Signed-off-by: Joel Dice <[email protected]>

* make `resume_fiber` private in `fiber.rs`

Signed-off-by: Joel Dice <[email protected]>

* Shrink `PollContext` state

Move management of the async guard range elsewhere to the normal
save/restore area.

* Refactor `AsyncCx`, reduce `unsafe`

* Remove the `AsyncCx` type from Wasmtime as it's inherently `unsafe` to
use, instead bundle operations directly on a `Store*` reference.

* Don't retain pointers-to-pointers within the roughly-equivalent
`BlockingContext` created in this PR. Instead when a blocking context
is created "take" the metadata from the store to assert exclusive
ownership of the pointers.

* Refactor how `&mut Context<'_>` is passed around, namely thread it
through fiber parameters to model resumption as registering a new
context to poll with.

* Remove `PollContext` in favor of directly storing a pointer as it's
now mostly an empty structure.

* Minor refactorings to make things more future-refactorable and/or
clear in a few places.

* Refactor management of the "current suspend" and "current future
context" pointers. These are now null'd out on resumption and asserted
null on suspension.

* Remove the need for a generic `Reset` structure in the fiber bits as
it's a pretty dangerous structure to have in general.

The end result of this refactoring is that all usage of `block_on` is
now safe and additionally many of the internals of the implementation
are safer than they were before

* Adjust some lint attributes

* Make manipulation of `AsyncState` safe

No need for raw pointers with recent refactorings.

* Fix dead code warning

* More dead code warnings

* Cut down on raw pointers in fiber.rs

* Move executor save/restore to normal fiber state save/restore

* Bikeshed a method name

* update comment in make_fiber

Signed-off-by: Joel Dice <[email protected]>

* fix machports build

Signed-off-by: Joel Dice <[email protected]>

---------

Signed-off-by: Joel Dice <[email protected]>
Co-authored-by: Alex Crichton <[email protected]>

show more ...

Fix another case of Miri unsoundness (#11056)

This commit fixes another issue we've discovered in the wasip3
prototyping repository about a code pattern in wasm which Miri flags as
un-sound. Specifi

Fix another case of Miri unsoundness (#11056)

This commit fixes another issue we've discovered in the wasip3
prototyping repository about a code pattern in wasm which Miri flags as
un-sound. Specifically what happened was:

* Invocation of WebAssembly went through `VMFuncRef::array_call` which
takes a `&self` parameter.

* Inside of WebAssembly though a `ref.func` instruction, or anything
else that references the original exported function, will
re-initialize the `VMFuncRef` which writes the `&self` up the stack,
which is not sound.

Fixing this required changing the signature of `array_call` from `&self`
to `me: NonNull<VMFuncRef>`, and the signature was already `unsafe` so
this is a new unsafe contract for that signature.

In fixing this, however, it was discovered that a mistake was made
in #10943 where some internal functions for re-initializing a
`VMFuncRef` relied on the previous signature of `&mut self` but that PR
switche to `&self`. This PR corrects these signatures to `Pin<&mut Self>`
and then plumbs around the necessary changes, notably causing some
refactoring in component-related bits.

show more ...

Refine `VMArrayCallNative` (#11047)

This commit refines the definition of the `VMArrayCallNative` type which
is the type signature of array-call functions which are used for
entering and exiting was

Refine `VMArrayCallNative` (#11047)

This commit refines the definition of the `VMArrayCallNative` type which
is the type signature of array-call functions which are used for
entering and exiting wasm. The first two parameters of this function are
the callee/caller VMContext values but they are both ascribed as
`VMOpaqueContext`. This is because for the `callee` it's not known
exactly what type the pointer has except within the context of the
defining function, so this value was not changed.

For the `caller` parameter though it's always the case that the value
passed in is indeed a `VMContext`. This commit reflects this fact in the
type signature and removes a number of now-unnecessary casts.

show more ...

Use `Pin<&mut ComponentInstance>` (#11042)

This commit is the continuation of #10943 for component instances. The
allocation/vmctx infrastructure was additionally refactored to be shared for
both co

Use `Pin<&mut ComponentInstance>` (#11042)

This commit is the continuation of #10943 for component instances. The
allocation/vmctx infrastructure was additionally refactored to be shared for
both core and component instances since they behave the exact same way anyway.
This further enables sharing various methods like `vmctx_plus_offset` which are
pretty unsafe internally.

Like #10943 this necessitated removal of `Index` implementations because
`IndexMut` is not compatible with the returned type being `Pin<&mut T>`
so they were replaced by inherent `get` and `get_mut` methods on the
component instance id type.

Closes #10933

show more ...

Stack switching: Infrastructure and runtime support (#10388)

* [pr1] base

* prtest:full

* make sure to use ControlFlow result in trace_suspended_continuation

* stack-switching: cleanup: remove st

Stack switching: Infrastructure and runtime support (#10388)

* [pr1] base

* prtest:full

* make sure to use ControlFlow result in trace_suspended_continuation

* stack-switching: cleanup: remove stray c-api changes

These are remnants of unrelated wasmfx wasmtime experiments, possibly
suitable for later submission against upstream.

* stack-switching: reuse async_stack_size

* stack-switching: delete delete_me debugging

* stack-switching: address feedback in environ::types

* stack-switching: remove unused code from vmoffsets

* stack-switching: drop dependency on std

* stack-switching: add compilation checks to ci matrix

* stack-switching: remove debug_println cruft

* stack-switching: export environ consts consistently

* stack-switching: export vm pub items consistently

* table_pool: reduced capacity for large elements

VMContRef elements which takes up two words and we don't want to
double the size of all tables in order to support storing these.
This change changes the table to target storing the requested
max number of elements if they are "nominally" sized with
(potentially) reduced capacity for non-nominally sized types when
encountered.

Continuations are the only type of element which may result in
fewer table slots being available than requested.

* stack-switching: extend conditional compilation

A fair bit of the definitions for stack switching are still
enabled, but this patch takes things a bit further to avoid
compilation problems; notably, cont_new is now not compiled
in unless the feature is enabled.

* stack-switching: formatting fixes

* stack-switching: address new clippy checks

In addition, to get clippy to fully pass, plumbed in
additional config to make winch paths happy; there's no
impl for winch yet but plumbing through the feature is
required to make paths incorporating macros at various
layers satisfied (and it is expected we'll use the
features in the future).

* stack-switching: more conditional compilation fixes

* stack-switching: additional conditional compile on table builtins for continuations

* stack-switching: additional conditional compile fixes

* stack-switching: additional conditional compile in store

* stack-switching: remove overly strict assertion

* stack-switching: remove errantly dropped no_mangle in config c-api

* stack-switching: VMContObj::from_raw_parts

* stack-switching: remove duplicate async_stack_size feature check

* stack-switching: VMArray -> VMHostArray

* stack-switching: remove unnecessary clippy exception

* stack-switching: fix docs referenced VMRuntimeLimits

* stack-switching: fix doc typo

* stack-switching: follow recommendations for type casts

* stack-switching: use usize::next_multiple_of

* stack-switching: update outdated comment

* stack-switching: use feature gate instead of allow(dead_code)

* stack-switching: rework backtrace using chunks/zip

* stack-switching: move tests to footer module

This is a bit more consistent with the prevailing style
in tree and (subjectively) makes finding the tests
as a reader more straightforward.

Tests left unchanged sans some import cleanup.

* stack-swictchding: verify stack_chain offsets at runtime

* fixup! stack-switching: use feature gate instead of allow(dead_code)

* stack-switching: document continuation roots tracing using match arms

---------

Co-authored-by: Paul Osborne <[email protected]>

show more ...

Remove dead accessors for a field of `VM*Import` (#10905)

These aren't actually read from compiled wasm code, only from the host.

Refactor globals to no longer use `Stored` (#10902)

This commit refactors the `wasmtime::Global` to avoid the usage of
`Stored<T>` internally. This makes conversion from internal global state
to ext

Refactor globals to no longer use `Stored` (#10902)

This commit refactors the `wasmtime::Global` to avoid the usage of
`Stored<T>` internally. This makes conversion from internal global state
to external global state a noop along the lines of previous commits. The
end goal is to remove `Stored` entirely and enable more pervasively using
external types internally within Wasmtime as well.

Globals were different than the prior iterations of memories, tags, and
tables. Globals have three different ways of defining them: wasm
instances, the host embedder, and component flags. Representing these
in all the various locations required a bit of finesse in how everything
is represented and stored at rest and such. In the end there's a small
amount of "type punning" in a few instance/vmctx fields related to
globals now since everything is squeezed into one slot. This is required
because the `VMGlobalImport` structure must have a size known to
wasm-compiled code and `wasmtime::Global` must have a known layout for C
code.

In the end while this is more code to manage globals my hope is that the
end result will be a net negative in terms of complexity by ensuring
that the embedder API is additionally suitable for use internally within
Wasmtime as well.

show more ...

Add a `DefinedTableIndex` to table imports/exports (#10879)

This commit extends `ExportTable` and `VMTableImport` (renamed from
`VMTable`) to include a `DefinedTableIndex` in the same manner that
me

Add a `DefinedTableIndex` to table imports/exports (#10879)

This commit extends `ExportTable` and `VMTableImport` (renamed from
`VMTable`) to include a `DefinedTableIndex` in the same manner that
memories carry their index they're defined at as well. The main goal of
this change is to power the next change which will update how tables are
stored in a store.

show more ...

Add `VMContext` and `DefinedTagIndex` fields to tag-related imports/exports (#10882)

* Add `VMContext` and `DefinedTagIndex` fields to tag-related imports/exports

Similar to memories/tables, will b

Add `VMContext` and `DefinedTagIndex` fields to tag-related imports/exports (#10882)

* Add `VMContext` and `DefinedTagIndex` fields to tag-related imports/exports

Similar to memories/tables, will be used in the next commit.

* Fix VMOffsets size of tag

show more ...