Remove need for explicit `Config::async_support` knob (#12371)

* Refactor component model host function definitions

Push the `async`-ness down one layer.

* Remove need for explicit `Config::async

Remove need for explicit `Config::async_support` knob (#12371)

* Refactor component model host function definitions

Push the `async`-ness down one layer.

* Remove need for explicit `Config::async_support` knob

This commit is an attempt to step towards reconciling "old async" and
"new async" in Wasmtime. The old async style is the original async
support in Wasmtime with `call_async`, `func_wrap_async`, etc, where the
main property is that the store is "locked" during an async operation.
Put another way, a store can only execute at most one async operation at
a time. This is in contrast to "new async" support in Wasmtime with the
component-model-async (WASIp3) support, where stores can have more than
one async operation in flight at once.

This commit does not fully reconcile these differences, but it does
remove one hurdle along the way: `Config::async_support`. Since the
beginning of Wasmtime this configuration knob has existed to explicitly
demarcate a config/engine/store as "this thing requires `async` stuff
internally." This has started to make less and less sense over time
where the line between sync and async has become more murky with WASIp3
where the two worlds comingle. The goal of this commit is to deprecate
`Config::async_support` and make the function not actually do anything.

In isolation this can't simply be done, however, because there are many
load-bearing aspects of Wasmtime that rely on this `async_support` knob.
For example once epochs + yielding are enabled it's required that all
Wasm is executed on a fiber lest it hit an epoch and not know how to
yield. That means that this commit is not a simple removal of
`async_support` but instead a refactoring/rearchitecting of how async is
used internally within Wasmtime. The high-level ideas within Wasmtime
now are:

* A `Store` has a "requires async" boolean stored within it.
* All configuration options which end up requiring async, such as
yielding with epochs, turn this boolean on.
* Creation of host functions which use async
(e.g. `func_wrap_{async,concurrent}`) will also turn this option on.
* Synchronous API entrypoints into Wasmtime ensure that this boolean is
disabled.
* Asynchronous APIs are usable at any time.

This means that the concept of an async store vs a sync store is now
gone. All stores are equally capable of executing sync/async, and the
change now is that dynamically some stores will require that async is
used with certain configuration. Additionally all panicking conditions
around `async_support` have been converted to errors instead. All
relevant APIs already returned an error and things are murky enough now
that it's not necessarily trivial to get this right at the embedder
level. In the interest of avoiding panics all detected async mismatches
are now first-class `wasmtime::Error` values.

The end result of this commit is that `Config::async_support` is a
deprecated `#[doc(hidden)]` function that does nothing. While many
internal changes happened as well as having new tests for all this sort
of behavior this is not expected to have a great impact on external
consumers. In general a deletion of `async_support(true)` is in theory
all that's required. This is intended to make it easier to think about
async/sync/etc in the future with WASIp3 and eventually reconcile
`func_wrap_async` and `func_wrap_concurrent` for example. That's left
for future refactorings however.

prtest:full

* Review comments

* Fix CI failures

show more ...

GC: replace ManuallyRooted with OwnedRooted. (#11514)

* GC: replace ManuallyRooted with OwnedRooted.

This implements the ideas from #11445: it replaces `ManuallyRooted`,
which requires an explicit

GC: replace ManuallyRooted with OwnedRooted. (#11514)

* GC: replace ManuallyRooted with OwnedRooted.

This implements the ideas from #11445: it replaces `ManuallyRooted`,
which requires an explicit unroot action with a mut borrow of the store
(making it impossible to implement in a standard `Drop` impl), with
`OwnedRooted`, which holds an `Arc` only to a small auxiliary memory
allocation (an `Arc<()>`) and uses this externalized "liveness flag" to
allow for a `Store`-less drop. These liveness flags are scanned during a
"trim" pass, which happens both when new owned roots are created, and
just before a GC.

This should greatly increase safety for host-side users of GC: it
provides a way to have a handle whose ownership works like any other
Rust value, alive until dropped. It is still not quite as efficient as
LIFO-scoped handles (by analogy, for the same reason that
individually-freed RAII types are not as efficient as arena allocation),
so those remain for efficiency-minded users that have a clear picture of
reference lifetimes.

At some later time we may wish to use `OwnedRooted` exclusively in our
public APIs rather than `Rooted`, and we may wish to rename `Rooted` to
`ScopedRooted`, but I haven't done either of those things yet.

I opted to *replace* `ManuallyRooted` rather than add a third kind of
root, after discussion with fitzgen. One implication of this is that the
C API's `anyref` and `externref` types are now 24 or 20 bytes rather
than 16 (because of the `Arc` pointer), and correspondingly the Val
union grew to that size. I *believe* this is an acceptable tradeoff, but
I'm happy to put `ManuallyRooted` back if not.

Fixes #11445.

* Review feedback.

* Fix for riscv32imac: loosen asserts on struct size slightly to allow for different padding.

* C API: use a `*const ()` to pass the liveness-flag Arc through C.

* Add some additional documentation warning about unrooting in the C API.

* Fix size-of test on 32-bit platforms.

* New amortized algorithm for root trimming.

* core::cmp, not std::cmp

* Always set high-water mark, even if eager.

* Make val size-assert tolerant of padding bytes in crates/c-api/src/val.rs too.

show more ...

Remove unsoundness of widening store borrows (#11481)

* Remove unsoundness of widening store borrows

This commit removes preexisting unsoundness in Wasmtime where a
`&mut StoreOpaque` borrow was "

Remove unsoundness of widening store borrows (#11481)

* Remove unsoundness of widening store borrows

This commit removes preexisting unsoundness in Wasmtime where a
`&mut StoreOpaque` borrow was "widened" into encompassing the limiter on
the `T` in `StoreInner<T>`, for example, by using the self-pointer
located in an instance or the store. This fix is done by threading
`&mut StoreOpaque` as a parameter separately from a
`StoreResourceLimiter`. This means that various callers now take a new
`Option<&mut StoreResourceLimiter<'_>>` parameter in various locations.

Closes #11409

* Fix gc-less build

show more ...

Deduplicate some more GC functions (#11480)

Use the `async` version from sync API entrypoints to deduplicate the
implementation.

Refactor exception objects to share more implementation with structs. (#11467)

* Refactor exception objects to share more implementation with structs.

This PR updates the implementation of Wasm exc

Refactor exception objects to share more implementation with structs. (#11467)

* Refactor exception objects to share more implementation with structs.

This PR updates the implementation of Wasm exception objects to share
more layout-computation code with structs, except for a single fixed
tag slot. The former is simply a nice refactor, and the latter is in
preparation for full exception support, which will require accessing
an exception's tag without knowing its layout dynamically at runtime.

* Review feedback.

show more ...

Make const-expr evaluation `async` (#11468)

* Make const-expr evaluation `async`

This commit is extracted from #11430 to accurately reflect how
const-expr evaluation is an async operation due to GC

Make const-expr evaluation `async` (#11468)

* Make const-expr evaluation `async`

This commit is extracted from #11430 to accurately reflect how
const-expr evaluation is an async operation due to GC pauses that may
happen. The changes in this commit are:

* Const-expr evaluation is, at its core, now an `async` function.
* To leverage this new `async`-ness all internal operations are switched
from `*_maybe_async` to `*_async` meaning all the `*_maybe_async`
methods can be removed.
* Some libcalls using `*_maybe_async` are switch to using `*_async` plus
the `block_on!` utility to help jettison more `*_maybe_async` methods.
* Instance initialization is now an `async` function. This is
temporarily handled with `block_on` during instance initialization to
avoid propagating the `async`-ness further upwards. This `block_on`
will get deleted in future refactorings.
* Const-expr evaluation has been refactored slightly to enable having a
fast path in global initialization which skips an `await` point
entirely, achieving performance-parity in benchmarks prior to this commit.

This ended up fixing a niche issue with GC where if a wasm execution was
suspended during `table.init`, for example, during a const-expr
evaluation triggering a GC then if the wasm execution was cancelled it
would panic the host. This panic was because the GC operation returned
`Result` but it was `unwrap`'d as part of the const-expr evaluation
which can fail not only to invalid-ness but also due to "computation is
cancelled" traps.

* Fix configured build

* Undo rebase mistake

show more ...

Minimize lazy allocation of the GC store (#11411)

* Minimize lazy allocation of the GC store

This commit is an effort to minimize the number of entrypoints which
might lazily allocate a GC store. T

Minimize lazy allocation of the GC store (#11411)

* Minimize lazy allocation of the GC store

This commit is an effort to minimize the number of entrypoints which
might lazily allocate a GC store. The is currently done through
`StoreOpaque::gc_store_mut` but this method is very commonly used
meaning that there are many many places to audit for lazily allocating a
GC store. The reason that this needs an audit is that lazy allocation
is an async operation right now that must be on a fiber and is something
I'm looking to fix as part of #11262.

This commit performs a few refactorings to achieve this:

* `gc_store_mut` is renamed to `ensure_gc_store`. This is intended to be
an `async` function in the future and clearly demarcates where lazy
allocation of a GC store is occurring.

* `require_gc_store{,_mut}` is now added which is a pure accessor of the
GC store with no lazy allocation. Most locations previously using
`gc_store_mut` are updated to use this instead.

Documentation is added to store methods to clearly indicate which ones
are allocating and which ones should only be called in a context where
allocation should already have happened.

* Fix configured build

* Relax GC store restrictions in more places

* Review comments on documentation

* Move `ensure_gc_store` calls during instantiation

Instead update `needs_gc_heap` with the tables that are added to a
module and rely on instantiation to create the GC heap.

* Shuffle around some code

* Fix CI and review comments

* Add in a few more i31 cases for externref

show more ...

Implement exception objects. (#11230)

* WIP: Working exception objects

* Clean build with gc disabled (`cargo check -p wasmtime --no-default-features --features runtime`).

* Review feedback.

* St

Implement exception objects. (#11230)

* WIP: Working exception objects

* Clean build with gc disabled (`cargo check -p wasmtime --no-default-features --features runtime`).

* Review feedback.

* Stub out C-API support.

* Fix Clippy complaints.

* Fix dead-code warning in c-api build.

* Actually fix 27->26 reserved bit rename and test.

* Fix exnref doc-test.

* fix fuzzing build

* fix feature-flagging on Instance::id

* Bless disas test diff due to reserved-bits change.

* Review feedback.

show more ...

Enable `allow_attributes_without_reason` (#11195)

* Enable `allow_attributes_without_reason`

This commit enables the `clippy::allow_attributes_without_reason` for
the `wasmtime` crate which previou

Enable `allow_attributes_without_reason` (#11195)

* Enable `allow_attributes_without_reason`

This commit enables the `clippy::allow_attributes_without_reason` for
the `wasmtime` crate which previously forcibly allowed it. The reason
this was allowed was that when the workspace was first migrated the
Wasmtime crate had too many instances that I was willing to fix. I've
now come back around and tried to fix everything.

In short: ideally delete `#[allow]`, otherwise use `#[expect]`,
otherwise use `#[allow]`.

prtest:full

* Adjust some directives

* Fix some warnings

* Fix stack switching size tests on unix

* Don't have a conditional `Drop` impl

* Force `testing_freelist` method to be used

Too lazy to write `#[cfg]`, but not too lazy to write a test.

show more ...

Stack switching: Infrastructure and runtime support (#10388)

* [pr1] base

* prtest:full

* make sure to use ControlFlow result in trace_suspended_continuation

* stack-switching: cleanup: remove st

Stack switching: Infrastructure and runtime support (#10388)

* [pr1] base

* prtest:full

* make sure to use ControlFlow result in trace_suspended_continuation

* stack-switching: cleanup: remove stray c-api changes

These are remnants of unrelated wasmfx wasmtime experiments, possibly
suitable for later submission against upstream.

* stack-switching: reuse async_stack_size

* stack-switching: delete delete_me debugging

* stack-switching: address feedback in environ::types

* stack-switching: remove unused code from vmoffsets

* stack-switching: drop dependency on std

* stack-switching: add compilation checks to ci matrix

* stack-switching: remove debug_println cruft

* stack-switching: export environ consts consistently

* stack-switching: export vm pub items consistently

* table_pool: reduced capacity for large elements

VMContRef elements which takes up two words and we don't want to
double the size of all tables in order to support storing these.
This change changes the table to target storing the requested
max number of elements if they are "nominally" sized with
(potentially) reduced capacity for non-nominally sized types when
encountered.

Continuations are the only type of element which may result in
fewer table slots being available than requested.

* stack-switching: extend conditional compilation

A fair bit of the definitions for stack switching are still
enabled, but this patch takes things a bit further to avoid
compilation problems; notably, cont_new is now not compiled
in unless the feature is enabled.

* stack-switching: formatting fixes

* stack-switching: address new clippy checks

In addition, to get clippy to fully pass, plumbed in
additional config to make winch paths happy; there's no
impl for winch yet but plumbing through the feature is
required to make paths incorporating macros at various
layers satisfied (and it is expected we'll use the
features in the future).

* stack-switching: more conditional compilation fixes

* stack-switching: additional conditional compile on table builtins for continuations

* stack-switching: additional conditional compile fixes

* stack-switching: additional conditional compile in store

* stack-switching: remove overly strict assertion

* stack-switching: remove errantly dropped no_mangle in config c-api

* stack-switching: VMContObj::from_raw_parts

* stack-switching: remove duplicate async_stack_size feature check

* stack-switching: VMArray -> VMHostArray

* stack-switching: remove unnecessary clippy exception

* stack-switching: fix docs referenced VMRuntimeLimits

* stack-switching: fix doc typo

* stack-switching: follow recommendations for type casts

* stack-switching: use usize::next_multiple_of

* stack-switching: update outdated comment

* stack-switching: use feature gate instead of allow(dead_code)

* stack-switching: rework backtrace using chunks/zip

* stack-switching: move tests to footer module

This is a bit more consistent with the prevailing style
in tree and (subjectively) makes finding the tests
as a reader more straightforward.

Tests left unchanged sans some import cleanup.

* stack-swictchding: verify stack_chain offsets at runtime

* fixup! stack-switching: use feature gate instead of allow(dead_code)

* stack-switching: document continuation roots tracing using match arms

---------

Co-authored-by: Paul Osborne <[email protected]>

show more ...

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtim

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtime to the 2024 Edition to keep
up-to-date with Rust idioms and access many of the edition features
exclusive to the 2024 edition.

prtest:full

* Reformat with the 2024 edition

show more ...

Add `T: 'static` to `Store<T>` (#10760)

* Add `T: 'static` to `Store<T>

Since the beginning the `T` type parameter on `Store<T>` has had no
bounds on it. This was intended for maximal flexibility i

Add `T: 'static` to `Store<T>` (#10760)

* Add `T: 'static` to `Store<T>

Since the beginning the `T` type parameter on `Store<T>` has had no
bounds on it. This was intended for maximal flexibility in terms of what
embedders place within a `Store<T>` and I've personally advocated that
we need to keep it this way. In the development of the WASIp3 work,
however, I've at least personally reached the conclusion that this is no
longer tenable and proceeding will require adding a `'static` bound to
data within a store.

Wasmtime today [already] carries unsafe `transmute`s to work around this
lack of `'static` bound, and while the number of `unsafe` parts is
relatively small right now we're still fundamentally lying to the
compiler about lifetime bounds internally. With the WASIp3 async work
this degree of "lying" has become even worse. Joel has written up some
examples [on Zulip] about how the Rust compiler is requiring `'static`
bounds in surprising ways. These patterns are cropping up quite
frequently in the WASIp3 work and it's becoming particularly onerous
maintaining all of the `unsafe` and ensuring that everything is in sync.

In the WASIp3 repository I've additionally [prototyped a change] which
would additionally practically require `T: 'static` in more locations.
This change is one I plan on landing in Wasmtime in the near future and
while its main motivations are for enabling WASIp3 work it is also a
much nicer system than what we have today, in my opinion.

Overall the cost of not having `T: 'static` on `Store<T>` is effectively
becoming quite costly, in particular with respect to WASIp3 work. This
is coupled with all known embedders already using `T: 'static` data
within a `Store<T>` so the expectation of the impact of this change is
not large. The main downside of this change as a result is that when and
where to place `'static` bounds is sort of a game of whack-a-mole with
the compiler. For example I changed `Store<T>` to require `'static`
here, but the rest of the change is basically "hit compile until rustc
says it's ok". There's not necessarily a huge amount of rhyme-or-reason
to where `'static` bounds crop up, which can be surprising or difficult
to work with for users.

In the end I feel that this change is necessary and one we can't shy
away from. If problems crop up we'll need to figure out how to thread
that needle at that time, but I'm coming around to thinking that
`T: 'static` is just a fundamental constraint we'll have to take on at
this time. Maybe a future version of Rust that fixes some of Joel's
examples (if they can be fixed, we're not sure of that) we could
consider relaxing this but that's left for future work.

[already]: https://github.com/bytecodealliance/wasmtime/blob/35053d6d8d1a5d4692cf636cba0c920b4a79a44b/crates/wasmtime/src/runtime/store.rs#L602-L611
[on Zulip]: https://rust-lang.zulipchat.com/#narrow/channel/122651-general/topic/.22type.20may.20not.20live.20long.20enough.22.20for.20generic.20closure/near/473862072
[prototyped a change]: https://github.com/bytecodealliance/wasip3-prototyping/pull/158

* Remove a no-longer-necessary `unsafe` block

* Update test expectations

* Fix gc-disabled builds

show more ...

Reuse Wasm linear memories code for GC heaps (#10503)

* Reuse code for Wasm linear memories for GC heaps

Instead of bespoke code paths and structures for Wasm GC, this commit makes it
so that we no

Reuse Wasm linear memories code for GC heaps (#10503)

* Reuse code for Wasm linear memories for GC heaps

Instead of bespoke code paths and structures for Wasm GC, this commit makes it
so that we now reuse VM structures like `VMMemoryDefinition` and bounds-checking
logic. Notably, we also reuse all the associated bounds-checking optimizations
and, when possible, virtual-memory techniques to completely elide them.

Furthermore, this commit adds support for growing GC heaps, reusing the
machinery for growing memories, and makes it so that GC heaps always start out
empty. This allows us to properly delay allocating the GC heap's storage until a
GC object is actually allocated.

Fixes #9350

* fix c api compilation

* use assert_contains

* remove no-longer-necessary extra memory config from limiter tests

* Helper for retry-after-maybe-async-gc in libcalls

* Clean up some comments

* fix wasmtime-fuzzing and no-gc compilation

* fix examples

* fix no-gc+compiler build

* fix build without pooling allocator

* fix +cranelift +gc-drc -gc-null builds

* fix table hash key stability test

* fix oracle usage of `ExternRef::new`

* fix +gc -gc-null -gc-drc build

* fix wasmtime-fuzzing

* make `StorePtr` wrap a `NonNull`

* Fix some doc tests

* Remove some unnecessary retry helpers now that `FooRef::new` will auto-gc

* fix things after rebase

* Reorganize collection/growth methods for GC heap

* rename BoundsCheck variants

* fix cfg'ing of gc only code

* Fix doc tests

* fix one more gc cfg

* disable GC heap OOM test on non-64-bit targets

show more ...

Automatically trigger GC in `{Array,Extern,Struct}Ref` allocation functions (#10560)

* Automatically trigger GC in `{Array,Extern,Struct}Ref` allocation functions

Rather than forcing all callers to

Automatically trigger GC in `{Array,Extern,Struct}Ref` allocation functions (#10560)

* Automatically trigger GC in `{Array,Extern,Struct}Ref` allocation functions

Rather than forcing all callers to check for `GcHeapOutOfMemory`, trigger a GC,
and then try again. This does force us to define `*_async` variations for when
async is enabled, however; it's ultimately worth it.

* cargo fmt

* review feedback and fix tests

* fix +runtime -gc build

* more feedback and build cfg fixes

* remove copy-paste assertion that doesn't apply to this method

* move assertion into retry methods

show more ...

Rely on `core::error::Error` (#9702)

* Rely on `core::error::Error`

With Wasmtime's new MSRV at 1.81 this means that `core::error::Error` is
available which means that in `no_std` mode the `Error`

Rely on `core::error::Error` (#9702)

* Rely on `core::error::Error`

With Wasmtime's new MSRV at 1.81 this means that `core::error::Error` is
available which means that in `no_std` mode the `Error` trait can be
used. This has been integrated into `anyhow::Error` already upstream and
means that we can remove our own local hacks such as the `Err2Anyhow` trait.

This commit removes the `Err2Anyhow` trait and all usage, going back to
idiomatic Rust error propagation and conversion even in the `no_std`
world. This should make code more portable by default and remove some
weird idioms we had for supporting this.

prtest:full

* Add some trusted vets

* Audit object crate update

* Disable backtraces on CI

show more ...

Implement the `array.copy` Wasm GC instruction (#9389)

* Implement the `array.copy` Wasm GC instruction

This also involved fixing some `VMGcRef` initialization and GC barrier code for
globals. That

Implement the `array.copy` Wasm GC instruction (#9389)

* Implement the `array.copy` Wasm GC instruction

This also involved fixing some `VMGcRef` initialization and GC barrier code for
globals. That in turn involved making global initialization fallible, which
meant that it needed to be pulled out of `vmctx` initialization and put into
instance initialization.

Co-Authored-By: Alex Crichton <[email protected]>

* fix compile error due to code moving where a trait method was no longer in scope

* use the result of `MaybeUninit::write`

---------

Co-authored-by: Alex Crichton <[email protected]>

show more ...

Improve a couple debug assertions for `{Array,Struct}Ref::from_cloned_gc_ref` (#9359)

Don't just assert that the given GC refs aren't `i31ref`s, but also that they
are of the correct type.

Note tha

Improve a couple debug assertions for `{Array,Struct}Ref::from_cloned_gc_ref` (#9359)

Don't just assert that the given GC refs aren't `i31ref`s, but also that they
are of the correct type.

Note that the other `from_cloned_gc_ref` methods (on, for example, `AnyRef`)
already do their equivalents of these checks. Just these two were a little
looser than they should have been.

show more ...

Introduce the `wasmtime::EqRef` type (#9285)

* Introduce the `wasmtime::EqRef` type

This commit introduces the `wasmtime::EqRef` type, which corresponds to Wasm's
`(ref eq)` type, and statically re

Introduce the `wasmtime::EqRef` type (#9285)

* Introduce the `wasmtime::EqRef` type

This commit introduces the `wasmtime::EqRef` type, which corresponds to Wasm's
`(ref eq)` type, and statically represents Wasm references that can be tested
for equality.

* fix no-gc builds

show more ...

Split GC type layout computation into a shared trait in `wasmtime_environ` (#9246)

* Split GC type layout computation into a shared trait in `wasmtime_environ`

This stuff was previously living in t

Split GC type layout computation into a shared trait in `wasmtime_environ` (#9246)

* Split GC type layout computation into a shared trait in `wasmtime_environ`

This stuff was previously living in the `GcRuntime` trait, but it turns out that the
compiler will also need to know the layout of GC types, who'd have thunk it.

* Fix disabled GC builds

show more ...

Introduce `wasmtime::ArrayRef` and allocating Wasm GC arrays (#9145)

* Introduce `wasmtime::ArrayRef` and allocating Wasm GC arrays

This commit introduces the `wasmtime::ArrayRef` type and support

Introduce `wasmtime::ArrayRef` and allocating Wasm GC arrays (#9145)

* Introduce `wasmtime::ArrayRef` and allocating Wasm GC arrays

This commit introduces the `wasmtime::ArrayRef` type and support for allocating
Wasm GC arrays from the host. This commit does *not* add support for the
`array.new` family of Wasm instructions; guests still cannot allocate Wasm GC
objects yet, but initial support should be pretty straightforward after this
commit lands.

The `ArrayRef` type has everything you expect from other value types in the
`wasmtime` crate:

* A method to get its type or check whether it matches a given type

* An implementation of `WasmTy` so that it can be used with `Func::wrap`-style
APIs

* The ability to upcast it into an `AnyRef` and to do checked downcasts in the
opposite direction

There are, additionally, methods for getting, setting, and enumerating a
`ArrayRef`'s elements.

Similar to how allocating a Wasm GC struct requires a `StructRefPre`, allocating
a Wasm GC array requires an `ArrayRefPre`, and this is motivated by the same
reasons.

* fix some doc tests and add docs for Func::wrap-style APIs

* Add a comment about why we can't user `iter::repeat(elem).take(len)`

* Fix some warnings in no-gc builds

show more ...