Remove need for explicit `Config::async_support` knob (#12371)

* Refactor component model host function definitions

Push the `async`-ness down one layer.

* Remove need for explicit `Config::async

Remove need for explicit `Config::async_support` knob (#12371)

* Refactor component model host function definitions

Push the `async`-ness down one layer.

* Remove need for explicit `Config::async_support` knob

This commit is an attempt to step towards reconciling "old async" and
"new async" in Wasmtime. The old async style is the original async
support in Wasmtime with `call_async`, `func_wrap_async`, etc, where the
main property is that the store is "locked" during an async operation.
Put another way, a store can only execute at most one async operation at
a time. This is in contrast to "new async" support in Wasmtime with the
component-model-async (WASIp3) support, where stores can have more than
one async operation in flight at once.

This commit does not fully reconcile these differences, but it does
remove one hurdle along the way: `Config::async_support`. Since the
beginning of Wasmtime this configuration knob has existed to explicitly
demarcate a config/engine/store as "this thing requires `async` stuff
internally." This has started to make less and less sense over time
where the line between sync and async has become more murky with WASIp3
where the two worlds comingle. The goal of this commit is to deprecate
`Config::async_support` and make the function not actually do anything.

In isolation this can't simply be done, however, because there are many
load-bearing aspects of Wasmtime that rely on this `async_support` knob.
For example once epochs + yielding are enabled it's required that all
Wasm is executed on a fiber lest it hit an epoch and not know how to
yield. That means that this commit is not a simple removal of
`async_support` but instead a refactoring/rearchitecting of how async is
used internally within Wasmtime. The high-level ideas within Wasmtime
now are:

* A `Store` has a "requires async" boolean stored within it.
* All configuration options which end up requiring async, such as
yielding with epochs, turn this boolean on.
* Creation of host functions which use async
(e.g. `func_wrap_{async,concurrent}`) will also turn this option on.
* Synchronous API entrypoints into Wasmtime ensure that this boolean is
disabled.
* Asynchronous APIs are usable at any time.

This means that the concept of an async store vs a sync store is now
gone. All stores are equally capable of executing sync/async, and the
change now is that dynamically some stores will require that async is
used with certain configuration. Additionally all panicking conditions
around `async_support` have been converted to errors instead. All
relevant APIs already returned an error and things are murky enough now
that it's not necessarily trivial to get this right at the embedder
level. In the interest of avoiding panics all detected async mismatches
are now first-class `wasmtime::Error` values.

The end result of this commit is that `Config::async_support` is a
deprecated `#[doc(hidden)]` function that does nothing. While many
internal changes happened as well as having new tests for all this sort
of behavior this is not expected to have a great impact on external
consumers. In general a deletion of `async_support(true)` is in theory
all that's required. This is intended to make it easier to think about
async/sync/etc in the future with WASIp3 and eventually reconcile
`func_wrap_async` and `func_wrap_concurrent` for example. That's left
for future refactorings however.

prtest:full

* Review comments

* Fix CI failures

show more ...

Use `.` instead of `/` for interface members. (#11947)

* Use `.` instead of `/` for interface members.

In the `wasmtime::component::generate` macro, change the syntax for
referencing functions and

Use `.` instead of `/` for interface members. (#11947)

* Use `.` instead of `/` for interface members.

In the `wasmtime::component::generate` macro, change the syntax for
referencing functions and types inside interfaces to use `.` instead
of `/`.

For example, this changes strings like
`wasi:http/types/outgoing-body`
to
`wasi:http/types.outgoing-body`
.

This makes the syntax more consistent with the syntax of
[WIT `use` statements], which use `.` for this purpose.

And, it avoids an incompatibility with the future nested namespaces syntax
([��]), where the `/d` in `a:b/c/d` is for traversing a component export
rather than an interface member.

[WIT `use` statements]: https://github.com/WebAssembly/component-model/blob/main/design/mvp/WIT.md#wit-packages-and-use
[��]: https://github.com/WebAssembly/component-model/blob/main/design/mvp/Explainer.md#gated-features

* Use the new syntax in more places.

* Revert changes to vendored WIT files.

* Revert more changes to vendored files.

* Update syntax in more places.

show more ...

Unify WASIp{2,3} context structures (#11370)

This removes `wasmtime_wasi::p{2,3}::{WasiCtx, WasiCtxBuilder,
WasiView}` in favor of only having `wasmtime_wasi::{WasiCtx,
WasiCtxBuilder, WasiView}` in

Unify WASIp{2,3} context structures (#11370)

This removes `wasmtime_wasi::p{2,3}::{WasiCtx, WasiCtxBuilder,
WasiView}` in favor of only having `wasmtime_wasi::{WasiCtx,
WasiCtxBuilder, WasiView}` instead. Conceptually these revisions of WASI
all provide the same functionality just with a different veneer that the
component model offers, so having only one way to configure host-side
behavior will make it easier to both organize implementations internally
(e.g. more sharing of code) as well as for embedders to configure (only
one context to create/manage).

show more ...

Simplify WASI internal implementations (#11365)

* Simplify WASI internal implementations

This commit migrates the WASIp2 implementation to be closer to the
upcoming WASIp3 implementation in terms o

Simplify WASI internal implementations (#11365)

* Simplify WASI internal implementations

This commit migrates the WASIp2 implementation to be closer to the
upcoming WASIp3 implementation in terms of how things are implemented
internally. Previously the way things worked with WASIp2 is:

* Embedders call `add_to_linker` with `T: WasiView`
* Internally `add_to_linker` is called which creates `WasiImpl<&mut T>`
* All internal implementations were `impl<T> Host for WasiImpl<T> where T: WasiView`
* A forwarding impl of `impl<T: WasiView> WasiView for &mut T` was
required

While this all worked it's a bit complicated for a few reasons:

1. Dealing with generically named structures like `WasiImpl` (or
`IoImpl` or `WasiHttpImpl`) is a bit baroque and not always obvious
as to what's going on.
2. The extra layer of generics in `impl<T> Host for WasiImpl<T>` adds a
layer of conceptual indirection which is non-obvious.
3. Other WASI proposal implementations do not use this strategy and
instead use "view" types or `impl Host for TheType` for example.
4. Internal incantations of `add_to_linker` had to deal with mixtures of
`IoImpl` and `WasiImpl` and aligning everything just right.
5. An extra layer of generics on all impls meant that everything was
generic meaning that `wasmtime-wasi`-the-crate didn't generate much
code, causing longer codegen times for consumers.

The goal of this commit is to migrate towards the style of what WASIp3
is prototyping for how impls are modeled. This is done to increase the
amount of code that can be shared between WASIp2 and WASIp3. This has a
number of benefits such as being easier to understand and also being
more modular where `wasi:clocks` implementations of traits don't require
filesystem context to be present (as is the case today). This in theory
helps a more mix-and-match paradigm of blending together various bits
and pieces of `wasmtime-wasi` implementations.

Concretely the changes made here are:

* `WasiView` no longer inherits from `IoView`, they're unrelated traits
now.
* `WasiView` now returns `WasiViewCtx<'a>` which has `ctx: &'a mut WasiCtx`
and `table: &'a mut ResourceTable`. That means it basically does the
same thing before but in a slightly different fashion.
* Implementations of `Host` traits are now directly for
`WasiCtxView<'_>` and don't involve any generics at all. These are
hopefully easier to understand and also better from a
codegen/compile-time perspective.
* Embedders no longer need to implement `IoView` directly and instead
fold that functionality into `WasiView`.
* `WasiHttpView` no longer inherits from `IoView` and instead has a
direct `fn table` method. Additionally `WasiHttpImpl` no longer embeds
`IoImpl` inside of it.
* Host traits for `wasi:io` are now implemented directly for
`ResourceTable` instead of `IoImpl<T>`.

The immediate goal of this refactoring is to enable more sharing along
the lines of #11362. This was not possible prior because WASIp3 requires
a simultaneous borrow on the table/ctx while the trait hierarchy
previously gave you one-or-the-other. With this new organization it will
be possible to get both at the same time meaning more
structure/contexts/etc can be shared between implementations.

prtest:full

* CI fixes

* More CI fixes

* More CI fixes

show more ...

Redesign function configuration in `bindgen!` (#11328)

* Redesign function configuration in `bindgen!`

This commit is a redesign of how function-level configuration works in
Wasmtime's `bindgen!` m

Redesign function configuration in `bindgen!` (#11328)

* Redesign function configuration in `bindgen!`

This commit is a redesign of how function-level configuration works in
Wasmtime's `bindgen!` macro. The main goal of this redesign is to
better support WASIp3 and component model async functions. Prior to this
redesign there was a mish mash of mechanisms to configure behavior of
imports/exports:

* The `async` configuration could turn everything async, nothing async,
only some imports async, or everything except some imports async.

* The `concurrent_{imports,exports}` keys were required to explicitly
opt-in to component model async signatures and applied to all
imports/exports.

* The `trappable_imports` configuration would indicate a list of imports
allowed to trap and it had special configuration for everything,
nothing, and only a certain list.

* The `tracing` and `verbose_tracing` keys could be applied to either
nothing or all functions.

Overall the previous state of configuration in `bindgen!` was clearly a
hodgepodge of systems that organically grew over time. In my personal
opinion it was in dire need of a refresh to take into account how
component-model-async ended up being implemented as well as
consolidating the one-off systems amongst all of these configuration
keys. A major motivation of this redesign, for example, was to inherit
behavior from WIT files by default. An `async` function in WIT should
not require `concurrent_*` keys to be configured, but rather it should
generate correct bindings by default.

In this commit, all of the above keys were removed. All keys have been
replaced with `imports` and `exports` configuration keys. Each behaves
the same way and looks like so:

bindgen!({
// ...
imports: {
// enable tracing for just this function
"my:local/interface/func": tracing,

// enable verbose tracing for just this function
"my:local/interface/other-func": tracing | verbose_tracing,

// this is blocking in WIT, but generate async bindings for
// it
"my:local/interface/[method]io.block": async,

// like above, but use "concurrent" bindings which have
// access to the store.
"my:local/interface/[method]io.block-again": async | store,

// everything else is, by default, trappable
default: trappable,
},
});

Effectively all the function-level configuration items are now bitflags.
These bitflags are by default inherited from the WIT files itself (e.g.
`async` functions are `async | store` by default). Further configuration
is then layered on top at the desires of the embedder. Supported keys are:

* `async` - this means that a Rust-level `async` function should be
generated. This is either `CallStyle::Async` or
`CallStyle::Concurrent` as it was prior, depending on ...

* `store` - this means that the generated function will have access to
the store on the host. This is only implemented right now for `async |
store` functions which map to `CallStyle::Concurrent`. In the future
I'd like to support just-`store` functions which means that you could
define a synchronous function with access to the store in addition to
an asynchronous function.

* `trappable` - this means that the function returns a
`wasmtime::Result<TheWitBindingType>`. If `trappable_errors` is
applicable then it means just a `Result<TheWitOkType,
TrappableErrorType>` is returned (like before)

* `tracing` - this enables `tracing!` integration for this function.

* `verbose_tracing` - this logs all argument values for this function
(including lists).

* `ignore_wit` - this ignores the WIT-level defaults of the function
(e.g. ignoring WIT `async`).

The way this then works is all modeled is that for any WIT function
being generated there are a set of flags associated with that function.
To calculate the flags the algorithm looks like:

1. Find the first matching rule in the `imports` or `exports` map
depending on if the function is imported or exported. If there is no
matching rule then use the `default` rule if present. This is the
initial set of flags for the function (or empty if nothing was
found).

2. If `ignore_wit` is present, return the flags from step 1. Otherwise
add in `async | store` if the function is `async` in WIT.

The resulting set of flags are then used to control how everything is
generated. For example the same split traits of today are still
generated and it's controlled based on the flags. Note though that the
previous `HostConcurrent` trait was renamed to `HostWithStore` to make
space for synchronous functions in this trait in the future too.

The end result of all these changes is that configuring imports/exports
now uses the exact same selection system as the `with` replacement map,
meaning there's only one system of selecting functions instead of 3.
WIT-level `async` is now respected by default meaning that bindings work
by default without further need to configure anything (unless more
functionality is desired).

One final minor change made here as well is that auto-generated
`instantiate` methods are now always synchronous and an
`instantiate_async` method is unconditionally generated for async mode.
This means that bindings always generate both functions and it's up to
the embedder to choose the appropriate one.

Closes #11246
Closes #11247

* Update expanded test expectations

prtest:full

* Fix the min platform embedding example

* Fix doc tests

* Always generate `*WithStore` traits

This helps when using the `with` mapping since that can always assume
that `HostWithStore` is available in the generated bindings, avoiding
the need to duplicate configuration options.

* Update test expectations

* Review comments

show more ...