Remove need for explicit `Config::async_support` knob (#12371)

* Refactor component model host function definitions

Push the `async`-ness down one layer.

* Remove need for explicit `Config::async

Remove need for explicit `Config::async_support` knob (#12371)

* Refactor component model host function definitions

Push the `async`-ness down one layer.

* Remove need for explicit `Config::async_support` knob

This commit is an attempt to step towards reconciling "old async" and
"new async" in Wasmtime. The old async style is the original async
support in Wasmtime with `call_async`, `func_wrap_async`, etc, where the
main property is that the store is "locked" during an async operation.
Put another way, a store can only execute at most one async operation at
a time. This is in contrast to "new async" support in Wasmtime with the
component-model-async (WASIp3) support, where stores can have more than
one async operation in flight at once.

This commit does not fully reconcile these differences, but it does
remove one hurdle along the way: `Config::async_support`. Since the
beginning of Wasmtime this configuration knob has existed to explicitly
demarcate a config/engine/store as "this thing requires `async` stuff
internally." This has started to make less and less sense over time
where the line between sync and async has become more murky with WASIp3
where the two worlds comingle. The goal of this commit is to deprecate
`Config::async_support` and make the function not actually do anything.

In isolation this can't simply be done, however, because there are many
load-bearing aspects of Wasmtime that rely on this `async_support` knob.
For example once epochs + yielding are enabled it's required that all
Wasm is executed on a fiber lest it hit an epoch and not know how to
yield. That means that this commit is not a simple removal of
`async_support` but instead a refactoring/rearchitecting of how async is
used internally within Wasmtime. The high-level ideas within Wasmtime
now are:

* A `Store` has a "requires async" boolean stored within it.
* All configuration options which end up requiring async, such as
yielding with epochs, turn this boolean on.
* Creation of host functions which use async
(e.g. `func_wrap_{async,concurrent}`) will also turn this option on.
* Synchronous API entrypoints into Wasmtime ensure that this boolean is
disabled.
* Asynchronous APIs are usable at any time.

This means that the concept of an async store vs a sync store is now
gone. All stores are equally capable of executing sync/async, and the
change now is that dynamically some stores will require that async is
used with certain configuration. Additionally all panicking conditions
around `async_support` have been converted to errors instead. All
relevant APIs already returned an error and things are murky enough now
that it's not necessarily trivial to get this right at the embedder
level. In the interest of avoiding panics all detected async mismatches
are now first-class `wasmtime::Error` values.

The end result of this commit is that `Config::async_support` is a
deprecated `#[doc(hidden)]` function that does nothing. While many
internal changes happened as well as having new tests for all this sort
of behavior this is not expected to have a great impact on external
consumers. In general a deletion of `async_support(true)` is in theory
all that's required. This is intended to make it easier to think about
async/sync/etc in the future with WASIp3 and eventually reconcile
`func_wrap_async` and `func_wrap_concurrent` for example. That's left
for future refactorings however.

prtest:full

* Review comments

* Fix CI failures

show more ...

`wit-bindgen`: Add an option to use `anyhow::Result` instead of `wasmtime::Result` (#12331)

* wit-bindgen: Add an option to use `anyhow::Result` instead of `wasmtime::Result`

* Add option to refere

`wit-bindgen`: Add an option to use `anyhow::Result` instead of `wasmtime::Result` (#12331)

* wit-bindgen: Add an option to use `anyhow::Result` instead of `wasmtime::Result`

* Add option to reference docs

* Rename `anyhow` dev dependency to `anyhow-for-testing`

show more ...

Migrate the Wasmtime CLI to `wasmtime::error` (#12295)

* Migrate wasmtime-cli to `wasmtime::error`

* migrate benches to `wasmtime::error` as well

* Remove new usage of anyhow that snuck in

trap on blocking call in sync task before return (#12043)

* trap on blocking call in sync task before return

This implements a spec change (PR pending) such that tasks created for calls to
synchron

trap on blocking call in sync task before return (#12043)

* trap on blocking call in sync task before return

This implements a spec change (PR pending) such that tasks created for calls to
synchronous exports may not call potentially-blocking imports or return `wait`
or `poll` callback codes prior to returning a value. Specifically, the
following are prohibited in that scenario:

- returning callback-code.{wait,poll}
- sync calling an async import
- sync calling subtask.cancel
- sync calling {stream,future}.{read,write}
- sync calling {stream,future}.cancel-{read,write}
- calling waitable-set.{wait,poll}
- calling thread.suspend

This breaks a number of tests, which will be addressed in follow-up commits:

- The `{tcp,udp}-socket.bind` implementation in `wasmtime-wasi` is implemented
using `Linker::func_wrap_concurrent` and thus assumed to be async, whereas the
WIT interface says they're sync, leading to a type mismatch error at runtime.
Alex and I have discussed this and have a general plan to address it.

- A number of tests in the tests/component-model submodule that points to the
spec repo are failing. Those will presumably be fixed as part of the upcoming
spec PR (although some could be due to bugs in this implementation, in which
case I'll fix them).

- A number of tests in tests/misc_testsuite are failing. I'll address those in
a follow-up commit.

Signed-off-by: Joel Dice <[email protected]>

* call `check_may_leave` before `check_blocking`

`check_blocking` needs access to the current task, but that's not set for
post-return functions since those should not be calling _any_ imports at all, so
first check for that.

Signed-off-by: Joel Dice <[email protected]>

* fix `misc_testsuite` test regressions

This amounts to adding `async` to any exported component functions that might
need to block.

Signed-off-by: Joel Dice <[email protected]>

* simplify code in `ConcurrentState::check_blocking`

Signed-off-by: Joel Dice <[email protected]>

* make `thread.yield` a no-op in non-blocking contexts

Per the proposed spec changes, `thread.yield` should return control to the guest
immediately without allowing any other thread to run. Similarly, when an
async-lifted export or callback returns `CALLBACK_CODE_YIELD`, we should call
the callback again immediately without allowing another thread to run.

Signed-off-by: Joel Dice <[email protected]>

* fix build when `component-model-async` feature disabled

Signed-off-by: Joel Dice <[email protected]>

* fix more test regressions

Signed-off-by: Joel Dice <[email protected]>

* fix more test regressions

Note that this temporarily updates the `tests/component-model` submodule to the
branch for https://github.com/WebAssembly/component-model/pull/577 until that PR
is merged.

Signed-off-by: Joel Dice <[email protected]>

* tweak `Trap::CannotBlockSyncTask` message

This clarifies that such a task cannot block prior to returning.

Signed-off-by: Joel Dice <[email protected]>

* fix cancel_host_future test

Signed-off-by: Joel Dice <[email protected]>

* trap sync-lowered, guest->guest async calls in sync tasks

I somehow forgot to address this earlier. Thanks to Luke for catching this.

Note that this commit doesn't include test coverage, but Luke's forthecoming
tests in the `component-model` repo will cover it, and we'll pull that in with
the next submodule update.

Signed-off-by: Joel Dice <[email protected]>

* switch back to `main` branch of `component-model` repo

...and skip or `should_fail` the tests that won't pass until
https://github.com/WebAssembly/component-model/pull/578 is merged.

Signed-off-by: Joel Dice <[email protected]>

* add `trap-if-block-and-sync.wast`

We'll remove this again in favor of the upstream version once
https://github.com/WebAssembly/component-model/pull/578 has been merged.

Signed-off-by: Joel Dice <[email protected]>

* address review feedback

- Assert that `StoreOpaque::suspend` is not called in a non-blocking context except in specific circumstances

- Typecheck async-ness for dynamic host functions

- Use type parameter instead of value parameter in `call_host[_dynamic]`

Signed-off-by: Joel Dice <[email protected]>

* add explanation comments to `check_blocking` calls

Signed-off-by: Joel Dice <[email protected]>

* fix fuzz test oracle for async functions

Signed-off-by: Joel Dice <[email protected]>

---------

Signed-off-by: Joel Dice <[email protected]>

show more ...

Update wasm-tools dependencies (#12031)

* Change separator style

* Update wasm-tools dependencies

* Also update wit-bindgen
* Drop `[async]` name prefixes
* Plumb `async` as part of a function ty

Update wasm-tools dependencies (#12031)

* Change separator style

* Update wasm-tools dependencies

* Also update wit-bindgen
* Drop `[async]` name prefixes
* Plumb `async` as part of a function type

Runtime handling of async functions and new traps are to be implemented
in subsequent commits. This is just getting everything running again.

* Run clang-format, also add test

* Fix some more wast tests

---------

Co-authored-by: Sy Brand <[email protected]>

show more ...

move `ConcurrentState` from `ComponentInstance` to `Store` (#11796)

* move `ConcurrentState` from `ComponentInstance` to `Store`

This has a few benefits:

- No need to specify an instance when crea

move `ConcurrentState` from `ComponentInstance` to `Store` (#11796)

* move `ConcurrentState` from `ComponentInstance` to `Store`

This has a few benefits:

- No need to specify an instance when creating or piping from a stream or future.
- No need to track the instance in an `Accessor`.
- You may now execute tasks for multiple instances in a single event loop.

The main drawback is that, if one of several instances within a single store
traps, it effectively means all instances have trapped, and the store can't be
used to create new instances. The way to avoid that is to use separate stores
for instances which must be isolated from others.

As a result of this change, a lot of code had to move from e.g. `impl Instance`
to e.g. `impl StoreOpaque`, so the diff is pretty huge, but the changes
themselves are almost entirely non-functional.

Fixes #11226

Signed-off-by: Joel Dice <[email protected]>

* fix non-component-model-async build

Signed-off-by: Joel Dice <[email protected]>

* fix outdated doc comment

Signed-off-by: Joel Dice <[email protected]>

* address review feedback

- restore `ComponentStoreData` encapsulation
- avoid conditional code duplication in `LiftContext::new`

Signed-off-by: Joel Dice <[email protected]>

---------

Signed-off-by: Joel Dice <[email protected]>

show more ...

Redesign function configuration in `bindgen!` (#11328)

* Redesign function configuration in `bindgen!`

This commit is a redesign of how function-level configuration works in
Wasmtime's `bindgen!` m

Redesign function configuration in `bindgen!` (#11328)

* Redesign function configuration in `bindgen!`

This commit is a redesign of how function-level configuration works in
Wasmtime's `bindgen!` macro. The main goal of this redesign is to
better support WASIp3 and component model async functions. Prior to this
redesign there was a mish mash of mechanisms to configure behavior of
imports/exports:

* The `async` configuration could turn everything async, nothing async,
only some imports async, or everything except some imports async.

* The `concurrent_{imports,exports}` keys were required to explicitly
opt-in to component model async signatures and applied to all
imports/exports.

* The `trappable_imports` configuration would indicate a list of imports
allowed to trap and it had special configuration for everything,
nothing, and only a certain list.

* The `tracing` and `verbose_tracing` keys could be applied to either
nothing or all functions.

Overall the previous state of configuration in `bindgen!` was clearly a
hodgepodge of systems that organically grew over time. In my personal
opinion it was in dire need of a refresh to take into account how
component-model-async ended up being implemented as well as
consolidating the one-off systems amongst all of these configuration
keys. A major motivation of this redesign, for example, was to inherit
behavior from WIT files by default. An `async` function in WIT should
not require `concurrent_*` keys to be configured, but rather it should
generate correct bindings by default.

In this commit, all of the above keys were removed. All keys have been
replaced with `imports` and `exports` configuration keys. Each behaves
the same way and looks like so:

bindgen!({
// ...
imports: {
// enable tracing for just this function
"my:local/interface/func": tracing,

// enable verbose tracing for just this function
"my:local/interface/other-func": tracing | verbose_tracing,

// this is blocking in WIT, but generate async bindings for
// it
"my:local/interface/[method]io.block": async,

// like above, but use "concurrent" bindings which have
// access to the store.
"my:local/interface/[method]io.block-again": async | store,

// everything else is, by default, trappable
default: trappable,
},
});

Effectively all the function-level configuration items are now bitflags.
These bitflags are by default inherited from the WIT files itself (e.g.
`async` functions are `async | store` by default). Further configuration
is then layered on top at the desires of the embedder. Supported keys are:

* `async` - this means that a Rust-level `async` function should be
generated. This is either `CallStyle::Async` or
`CallStyle::Concurrent` as it was prior, depending on ...

* `store` - this means that the generated function will have access to
the store on the host. This is only implemented right now for `async |
store` functions which map to `CallStyle::Concurrent`. In the future
I'd like to support just-`store` functions which means that you could
define a synchronous function with access to the store in addition to
an asynchronous function.

* `trappable` - this means that the function returns a
`wasmtime::Result<TheWitBindingType>`. If `trappable_errors` is
applicable then it means just a `Result<TheWitOkType,
TrappableErrorType>` is returned (like before)

* `tracing` - this enables `tracing!` integration for this function.

* `verbose_tracing` - this logs all argument values for this function
(including lists).

* `ignore_wit` - this ignores the WIT-level defaults of the function
(e.g. ignoring WIT `async`).

The way this then works is all modeled is that for any WIT function
being generated there are a set of flags associated with that function.
To calculate the flags the algorithm looks like:

1. Find the first matching rule in the `imports` or `exports` map
depending on if the function is imported or exported. If there is no
matching rule then use the `default` rule if present. This is the
initial set of flags for the function (or empty if nothing was
found).

2. If `ignore_wit` is present, return the flags from step 1. Otherwise
add in `async | store` if the function is `async` in WIT.

The resulting set of flags are then used to control how everything is
generated. For example the same split traits of today are still
generated and it's controlled based on the flags. Note though that the
previous `HostConcurrent` trait was renamed to `HostWithStore` to make
space for synchronous functions in this trait in the future too.

The end result of all these changes is that configuring imports/exports
now uses the exact same selection system as the `with` replacement map,
meaning there's only one system of selecting functions instead of 3.
WIT-level `async` is now respected by default meaning that bindings work
by default without further need to configure anything (unless more
functionality is desired).

One final minor change made here as well is that auto-generated
`instantiate` methods are now always synchronous and an
`instantiate_async` method is unconditionally generated for async mode.
This means that bindings always generate both functions and it's up to
the embedder to choose the appropriate one.

Closes #11246
Closes #11247

* Update expanded test expectations

prtest:full

* Fix the min platform embedding example

* Fix doc tests

* Always generate `*WithStore` traits

This helps when using the `with` mapping since that can always assume
that `HostWithStore` is available in the generated bindings, avoiding
the need to duplicate configuration options.

* Update test expectations

* Review comments

show more ...

Rename `Instance::run_with` to `Instance::run_concurrent` (#11259)

* Rename `Instance::run_with` to `Instance::run_concurrent`

This commit renames `run_with` to `run_concurrent` and additionally
de

Rename `Instance::run_with` to `Instance::run_concurrent` (#11259)

* Rename `Instance::run_with` to `Instance::run_concurrent`

This commit renames `run_with` to `run_concurrent` and additionally
deletes the `Instance::run` method. With `Accessor`-related changes
there's no need to have a "just run with a future" method as almost
everything is going to require `run_concurrent` anyway to acquire an
accessor.

Closes #11224

* Fix compile issues

* Update crates/wasmtime/src/runtime/component/func/typed.rs

Co-authored-by: Joel Dice <[email protected]>

---------

Co-authored-by: Joel Dice <[email protected]>

show more ...

Start to use `&Accessor<T, D>` more in concurrent code (#11238)

* Start to use `&Accessor<T, D>` more in concurrent code

After discussion with Joel we've concluded that while `&mut Accessor<T, D>`

Start to use `&Accessor<T, D>` more in concurrent code (#11238)

* Start to use `&Accessor<T, D>` more in concurrent code

After discussion with Joel we've concluded that while `&mut Accessor<T, D>`
was originally added to model host functions it is also appropriate to
use it to model embedder-rooted invocations of items such as wasm as
well. Effectively the conclusion we reached was that
`*::call_concurrent` should be taking `&Accessor`, not
`StoreContextMut`. This has a number of benefits to it over the previous
iteration:

* This makes exports behave more like imports where `Accessor` means
"you're in the concurrent world".

* This makes exports have an `async fn` signature which is easier to
read and understand.

* This automatically enforces the guarantee that the returned future is
only polled within the main event loop because the future is always
considered to close over the `&Accessor` provided meaning it
statically cannot live outside of the event loop.

* This paves the way forward to future refactorings to avoid storing so
much state within a `Store<T>` and instead try to store state directly
in futures themselves. This should make cancellation more natural and
eventually also remove `'static` bounds on params/results. Furthermore
this should make it easier to avoid spawning tasks internally by
storing state in futures instead of spawned tasks.

In doing this one of the main questions we were faced with was what to
do about `&mut Accessor<T, D>`, namely the `mut` part. With a mutable
accessor it would be only possible to call one function concurrently.
One option considered was to add combinators like `Accessor::join` and
`Accessor::race` but in the end we decided to avoid going that direction
and instead switch to `&Accessor<T, D>` everywhere, freely enabling
aliasing of the accessor. This has the downside that `Accessor::with` is
now a relatively dangerous function in that it can panic, but idiomatic
usage of it is not expected to panic as the distinction between the
`async` and sync boundary of `Accessor` vs `StoreContextMut` is expected
to naturally make the recursive panic condition of `with` rare to come
up in practice.

Concrete changes in this commit are:

* `Accessor::with` now requires `&self`.
* `Accessor::spawn` now requires `&self`.
* Host functions are now given `&Accessor`, not `&mut Accessor`.
* `{Typed,}Func::call_concurrent` is now an `async fn` which takes an
`&Accessor` instead of `StoreContextMut`.
* Guest bindings generation for concurrent invocations now looks exactly
like async bindings generation except for replacing `StoreContextMut`
with `Accessor`.

Note that this commit does not yet update the internal implementations
of these functions to benefit from the new abilities that taking
`&Accessor` implies. Instead that's deferred to a future update as
necessary. Instead this is only updating the public API of the
`wasmtime` crate to enable these refactorings in the future.

Also note that this does not yet update all functions to take
`&Accessor`. Notably futures and streams still need to be updated.

cc #11224

* Review comments

---------

Co-authored-by: Joel Dice <[email protected]>

show more ...

add Component Model async ABI tests (#11136)

* add Component Model async ABI tests

This pulls in the tests from the `wasip3-prototyping` repo, minus the ones
requiring WASIp3 support in `wasmtime-w

add Component Model async ABI tests (#11136)

* add Component Model async ABI tests

This pulls in the tests from the `wasip3-prototyping` repo, minus the ones
requiring WASIp3 support in `wasmtime-wasi[-http]`, which will be PR'd
separately.

* add audits and exemptions for new `component-async-tests` deps

In order to convince `cargo vet` that we only needed these deps to be
`safe-to-run` (not necessarily `safe-to-deploy`, since it's test code), I've
moved the `wasm-compose` dep to the `dev-dependencies` section of the
`Cargo.toml` file, which required rearranging some code.

I've exempted `wasm-compose` since it's a BA project, and also exempted all but
one of the remaining new deps since they each get well over 10,000 downloads per
day from crates.io. I've audited and certified the remaining dep, `im-rc`,
which came in a bit shy of the 10,000-per-day mark.

Signed-off-by: Joel Dice <[email protected]>

* simplify `component_async_tests::util::sleep`

Signed-off-by: Joel Dice <[email protected]>

---------

Signed-off-by: Joel Dice <[email protected]>

show more ...

Deps: Update `wasm-tools` crates to 0.233.0 (#10915)

* Update `wasm-tools` deps to 0.233.0

* Update `cargo vet` metadata

Just a prune and wildcard renewals

* cargo fmt

* Fix type export identifi

Deps: Update `wasm-tools` crates to 0.233.0 (#10915)

* Update `wasm-tools` deps to 0.233.0

* Update `cargo vet` metadata

Just a prune and wildcard renewals

* cargo fmt

* Fix type export identifiers

* Fix more type identifiers in exports in WAT

show more ...

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtim

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtime to the 2024 Edition to keep
up-to-date with Rust idioms and access many of the edition features
exclusive to the 2024 edition.

prtest:full

* Reformat with the 2024 edition

show more ...

Replace `GetHost` with a function pointer, add `HasData` (#10770)

* Replace `GetHost` with a function pointer, add `HasData`

This commit is a refactoring to the fundamentals of the `bindgen!` macro

Replace `GetHost` with a function pointer, add `HasData` (#10770)

* Replace `GetHost` with a function pointer, add `HasData`

This commit is a refactoring to the fundamentals of the `bindgen!` macro
and the functions that it generates. Prior to this change the
fundamental entrypoint generated by `bindgen!` was a function
`add_to_linker_get_host` which takes a value of type `G: GetHost`. This
`GetHost` implementation is effectively an alias for a closure whose
return value is able to close over the parameter given lfietime-wise.

The `GetHost` abstraction was added to Wasmtime originally to enable
using any type that implements `Host` traits, not just `&mut U` as was
originally supported. The definition of `GetHost` was _just_ right to
enable a type such as `MyThing<&mut T>` to implement `Host` and a
closure could be provided that could return it. At the time that
`GetHost` was added it was known to be problematic from an
understandability point of view, namely:

* It has a non-obvious definition.
* It's pretty advanced Rust voodoo to understand what it's actually
doing
* Using `GetHost` required lots of `for<'a> ...` in places which is
unfamiliar syntax for many.
* `GetHost` values couldn't be type-erased (e.g. put in a trait object)
as we couldn't figure out the lifetime syntax to do so.

Despite these issues it was the only known solution at hand so we landed
it and kept the previous `add_to_linker` style (`&mut T -> &mut U`) as a
convenience. While this has worked reasonable well (most folks just try
to not look at `GetHost`) it has reached a breaking point in the WASIp3
work.

In the WASIp3 work it's effectively now going to be required that the
`G: GetHost` value is packaged up and actually stored inside of
accessors provided to host functions. This means that `GetHost` values
now need to not only be taken in `add_to_linker` but additionally
provided to the rest of the system through an "accessor". This was made
possible in #10746 by moving the `GetHost` type into Wasmtime itself (as
opposed to generated code where it lived prior).

While this worked with WASIp3 and it was possible to plumb `G: GetHost`
safely around, this ended up surfacing more issues. Namely all
"concurrent" host functions started getting significantly more
complicated `where` clauses and type signatures. At the end of the day I
felt that we had reached the end of the road to `GetHost` and wanted to
search for alternatives, hence this change.

The fundamental purpose of `GetHost` was to be able to express, in a
generic fashion:

* Give me a closure that takes `&mut T` and returns `D`.
* The `D` type can close over the lifetime in `&mut T`.
* The `D` type must implement `bindgen!`-generated traits.

A realization I had was that we could model this with a generic
associated type in Rust. Rust support for generic associated types is
relatively new and not something I've used much before, but it ended up
being a perfect model for this. The definition of the new `HasData`
trait is deceptively simple:

trait HasData {
type Data<'a>;
}

What this enables us to do though is to generate `add_to_linker`
functions that look like this:

fn add_to_linker<T, D>(
linker: &mut Linker<T>,
getter: fn(&mut T) -> D::Data<'_>,
) -> Result<()>
where
D: HasData,
for<'a> D::Data<'a>: Host;

This definition here models `G: GetHost` as a literal function pointer,
and the ability to close over the `&mut T` lifetime with type (not just
`&mut U`) is expressed through the type constructor `type Data<'a>`).
Ideally we could take a generic generic associated type (I'm not even
sure what to call that), but that's not something Rust has today.

Overall this felt like a much simpler way of modeling `GetHost` and its
requirements. This plumbed well throughout the WASIp3 work and the
signatures for concurrent functions felt much more appropriate in terms
of complexity after this change. Taking this change to the limit means
that `GetHost` in its entirety could be purged since all usages of it
could be replaced with `fn(&mut T) -> D::Data<'a>`, a hopefully much
more understandable type.

This change is not all rainbows however, there are some gotchas that
remain:

* One is that all `add_to_linker` generated functions have a `D:
HasData` type parameter. This type parameter cannot be inferred and
must always be explicitly specified, and it's not easy to know what to
supply here without reading documentation. Actually supplying the type
parameter is quite easy once you know what to do (and what to fill
in), but it may involve defining a small struct with a custom
`HasData` implementation which can be non-obvious.

* Another is that the `G: GetHost` value was previously a full Rust
closure, but now it's transitioning to a function pointer. This is
done in preparation for WASIp3 work where the function needs to be
passed around, and doing that behind a generic parameter is more
effort than it's worth. This means that embedders relying on the true
closure-like nature here will have to update to using a function
pointer instead.

* The function pointer is stored in locations that require `'static`,
and while `fn(T)` might be expected to be `'static` regardless of `T`
is is, in fact, not. This means that practically `add_to_linker`
requires `T: 'static`. Relative to just before this change this is a
possible regression in functionality, but there orthogonal reasons
beyond just this that we want to start requiring `T: 'static` anyway.
That means that this isn't actually a regression relative to #10760, a
related change.

The first point is partially ameliorated with WASIp3 work insofar that
the `D` type parameter will start serving as a location to specify where
concurrent implementations are found. These concurrent methods don't
take `&mut self` but instead are implemented for `T: HasData` types. In
that sense it's more justified to have this weird type parameter, but in
the meantime without this support it'll feel a bit odd to have this
little type parameter hanging off the side.

This change has been integrated into the WASIp3 prototyping repository
with success. This has additionally been integrated into the Spin
embedding which has one of the more complicated reliances on
`*_get_host` functions known. Given that it's expected that while this
is not necessarily a trivial change to rebase over it should at least be
possible.

Finally the `HasData` trait here has been included with what I'm hoping
is a sufficient amount of documentation to at least give folks a spring
board to understand it. If folks have confusion about this `D` type
parameter my hope is they'll make their way to `HasData` which showcases
various patterns for "librarifying" host implementations of WIT
interfaces. These patterns are all used throughout Wasmtime and WASI
currently in crates and tests and such.

* Update expanded test expectations

show more ...

Remove extraneous `Send` bound from bindgen-generated exports (#10685)

Accidental regression from #10044.

Closes #10677

Remove some `#![expect(clippy::allow_attributes_without_reason)]` (#10661)

Clean up some crates by migrating from `#[allow]` to `#[expect]`
(ideally) or `#[allow]`-with-reason

Remove async-trait in bindgen (#9867)

* use trait_variant and async-func-in-trait instead of async-trait

* remove all async_trait::async_trait related to bindgen! proc_macro

* remove pub use async

Remove async-trait in bindgen (#9867)

* use trait_variant and async-func-in-trait instead of async-trait

* remove all async_trait::async_trait related to bindgen! proc_macro

* remove pub use async_trait::async_trait in __internal

* bless `component-macro` tests to reflect `async_trait` removal

Signed-off-by: Joel Dice <[email protected]>

* add `trait-variant` exemption to supply-chain/config.toml

This exempts the `trait-variant` crate from vetting since it is published and
maintained by the rust-lang organization and is officially recommended by the
Rust project.

Signed-off-by: Joel Dice <[email protected]>

---------

Signed-off-by: Joel Dice <[email protected]>
Co-authored-by: Joel Dice <[email protected]>

show more ...

Configure WIT feature gates at runtime & implement wasi-cli `exit-with-code` (#9381)

* Remove `features` configuration from component::bindgen! and always emit all unstable members. These features w

Configure WIT feature gates at runtime & implement wasi-cli `exit-with-code` (#9381)

* Remove `features` configuration from component::bindgen! and always emit all unstable members. These features will be gated at runtime.

* Implement wasi-cli's unstable `exit-with-code`

* Add codegen test for unstable features

* Add LinkOptions type and add a parameter to all add_to_linker functions in for worlds/interfaces that use any unstable feature.

* More descriptive test feature names.

* Generate feature gate `if` checks

* Expose `cli-exit-with-code` as CLI option

* Generate bespoke option types per interface and world.

* Add unit test

* Remove exit code restriction on Windows

* Add cli_exit_with_code test

* Use BTreeSet to generate the options in consistent order

* Change wasmtime-wasi's `add_to_linker_(a)sync` signature back to how it was and add new variants that take the option parameters.

* Lift Windows exit code restriction in tests

* Lift Windows exit code restriction

show more ...

Enforce `uninlined_format_args` for the workspace (#9065)

* Enforce `uninlined_format_args` for the workspace

* fix: failing `Monolith Checks` job

* fix: formatting

Redesign how component exports work (#8786)

* Un-nest exports in a component

This commit flattens the representation of exports in a component to
make them more easily indexable without forcing tra

Redesign how component exports work (#8786)

* Un-nest exports in a component

This commit flattens the representation of exports in a component to
make them more easily indexable without forcing traversal through the
hierarchy of instance imports/exports to get there.

* Guarantee type information on component exports

Don't have it optional in some cases and present in others, instead
ensure there's type information for all component exports immediately
available.

* Refactor how component instance exports are loaded

This commit is a change to Wasmtime's public API for
`wasmtime::component::Instance` that reorganizes how component exports
are loaded. Previously there was a system where `Instance::exports()`
was called that that was sort of "iterated over" in a builder-style
pattern to acquire the actual export desired. This required lifetime
trickery for nested instances and some unfortunate API bloat. The major
downside of this approach is that it requires unconditional string
lookups at runtime for exports and additionally does not serve as a
great place to implement the semver-compatible logic of #8395. The goal
of this refactoring is to pave the way to improving this.

The new APIs for loading exports now look a bit more similar to what's
available for core modules. Notably there's a new
`Component::export_index` method which enables performing a string
lookup and returning an index. This index can in turn be passed to
`Instance::get_*` to skip the string lookup when exports are loaded. The
`Instance::exports` API is then entirely removed and dismantled.

The only piece remaining is the ability to load nested exports which is
done through an `Option` parameter to `Component::export_index`. The
way to load a nested instance is now to first lookup the instance with
`None` as this parameter an then the instance itself is `Some` to look
up an export of that instance. This removes the need for a
recursive-style lifetime-juggling API from wasmtime and in theory helps
simplify the usage of loading exports.

* Update `bindgen!` generated structures for exports

This commit updates the output of `bindgen!` to have a different setup
for exports of worlds to handle the changes from the previous commit.
This introduces new `*Pre` structures which are generated alongside the
existing `Guest` structures for example. The `*Pre` versions contain
`ComponentExportIndex` from the previous commit and serve as a path to
accelerating instantiation because all name lookups are skipped.

* Update test expectations for `bindgen!`-generated output

* Review comments

* Fix doc link

show more ...

Add `GetHost` to generated bindings for more flexible linking (#8448)

* Remove unused generated `add_root_to_linker` method

* WIP: bindgen GetHost

* Compile with Rust 1.78+

Use <https://users.rus

Add `GetHost` to generated bindings for more flexible linking (#8448)

* Remove unused generated `add_root_to_linker` method

* WIP: bindgen GetHost

* Compile with Rust 1.78+

Use <https://users.rust-lang.org/t/generic-closure-returns-that-can-capture-arguments/76513/3>
as a guide of how to implement this by making the `GetHost` trait a bit
uglier.

* Add an option to skip `&mut T -> T` impls

Also enable this for WASI crates since they do their own thing with
`WasiView` for now. A future refactoring should be able to remove this
option entirely and switch wasi crates to a new design of `WasiView`.

* Update test expectations

* Review comments

* Undo temporary change

* Handle some TODOs

* Remove no-longer-relevant note

* Fix wasmtime-wasi-http doc link

---------

Co-authored-by: Alex Crichton <[email protected]>

show more ...

Update nightly used in CI and fix warnings (#8416)

Fixes some warnings that nightly Rust has started emitting.

Disable traps by default in `bindgen!` imports (#8310)

By default previously all return types were wrapped in
`wasmtime::Result<T>` to enable any import to return a trap to the wasm
guest. This is a

Disable traps by default in `bindgen!` imports (#8310)

By default previously all return types were wrapped in
`wasmtime::Result<T>` to enable any import to return a trap to the wasm
guest. This is a fair bit of boilerplate, however, and it's easy to
accidentally turn a normal error into a trap. This is additionally
somewhat of a power-user method as many imports probably don't end up
wanting to trap.

This commit adds a new configuration option to `bindgen!`:
`trappable_imports`, and switches the default to `false`. The previous
behavior can be recovered with `trappable_imports: true`.

show more ...

Add a whole lot of semicolons to WIT files (#7159)

To prepare this commit I've run `WIT_REQUIRE_SEMICOLONS=1 ./ci/run-tests.sh`
locally which configures `wit-parser` to generate an error for missing

Add a whole lot of semicolons to WIT files (#7159)

To prepare this commit I've run `WIT_REQUIRE_SEMICOLONS=1 ./ci/run-tests.sh`
locally which configures `wit-parser` to generate an error for missing
semicolons in WIT files. This led me to add quite a few semicolons in
quite a few places in what is going to be the first of a few batches of
semicolons.

CI checks for this cannot be added just yet because the wasi-nn spec is
a submodule which needs to be updated with semicolons before this
repository can require semicolons. Nevertheless that doesn't stop us
from using semicolons in the meantime (yay gradual rollout of changes!)
so I figure this would be good to get in sooner rather than later.

show more ...

Implement support for exported resources in `bindgen!` (#7050)

* Implement support for exported resources in `bindgen!`

This commit updates the `wasmtime::component::bindgen!` to support
exported r

Implement support for exported resources in `bindgen!` (#7050)

* Implement support for exported resources in `bindgen!`

This commit updates the `wasmtime::component::bindgen!` to support
exported resources. Exported resources are themselves always modeled as
`ResourceAny` at runtime and are required to perform dynamic type checks
to ensure that the right type of resource is passed in. Exported
resources have their own `GuestXXX` structure exported to namespace all
of their methods. Otherwise, like imports, there's not a whole lot of
special treatment of exported resources.

* Work around `heck` issue

Looks like `to_snake_case` behaves differently if the `unicode` feature
is enabled or not so bypass that entirely.

show more ...

Enhance `async` configuration of `bindgen!` macro (#6942)

This commit takes a leaf out of `wiggle`'s book to enable bindings
generation for async host functions where only some host functions are
as

Enhance `async` configuration of `bindgen!` macro (#6942)

This commit takes a leaf out of `wiggle`'s book to enable bindings
generation for async host functions where only some host functions are
async instead of all of them. This enhances the `async` key with a few
more options:

async: {
except_imports: ["foo"],
only_imports: ["bar"],
}

This is beyond what `wiggle` supports where either an allow-list or
deny-list can be specified (although only one can be specified). This
can be useful if either the list of sync imports or the list of async
imports is small.

show more ...