fix(bindgen,format): enable wasm encoded to be used in the `bindgen!` macro (#12857)

Remove subtask reparenting (#12570)

This commit updates the implementation of component-model-async
primitives to remove the manual subtask reparenting process. This is
required to fix #12544 at a s

Remove subtask reparenting (#12570)

This commit updates the implementation of component-model-async
primitives to remove the manual subtask reparenting process. This is
required to fix #12544 at a semantic level because a subtask isn't ever
actually reparented, even if its parent exits. The first part of this
change is to remove the `GuestTask::subtasks` field and all relevant
manipulations of it.

This field, however, powered the `TaskExit` abstraction returned from
`call_concurrent`. This commit then subsequently deletes `TaskExit` and
all related infrastructure as it's no longer directly applicable as-is.
The rest of this change is then updating tests/bindings/etc to account
for these two changes.

The main semantic changes related to tests are:

* `wasmtime run`, with and without `--invoke`, no longer waits for all
subtasks. This instead only waits for the main task returning before
exiting. Whether or not this is the correct behavior is under
discussion in WebAssembly/component-model#608

* `wasmtime serve` has been updated to keep the store alive at least
until the response body has been fully transmitted. This is also part
of WebAssembly/component-model#608.

* Some `component-async-tests`-related tests were updated to either
avoid blocking the store as it wasn't needed or yield enough times to
ensure that the test passes.

Closes #12544

prtest:full

show more ...

`wit-bindgen`: Add an option to use `anyhow::Result` instead of `wasmtime::Result` (#12331)

* wit-bindgen: Add an option to use `anyhow::Result` instead of `wasmtime::Result`

* Add option to refere

`wit-bindgen`: Add an option to use `anyhow::Result` instead of `wasmtime::Result` (#12331)

* wit-bindgen: Add an option to use `anyhow::Result` instead of `wasmtime::Result`

* Add option to reference docs

* Rename `anyhow` dev dependency to `anyhow-for-testing`

show more ...

add `task_exit` option to `wasmtime-wit-bindgen` (#11665)

This builds on #11662 by optionally exposing the `TaskExit` return value from
`[Typed]Func::call_concurrent` in the bindings generated for e

add `task_exit` option to `wasmtime-wit-bindgen` (#11665)

This builds on #11662 by optionally exposing the `TaskExit` return value from
`[Typed]Func::call_concurrent` in the bindings generated for exported functions.

Note that the first two commits are shared with #11662.

Fixes #11600

Signed-off-by: Joel Dice <[email protected]>

show more ...

Update wasm-tools crates (#11594)

* Update wasm-tools crates

Keeping up to date

* Match `bindgen!` world selection with `wit-bindgen::generate!`

Redesign function configuration in `bindgen!` (#11328)

* Redesign function configuration in `bindgen!`

This commit is a redesign of how function-level configuration works in
Wasmtime's `bindgen!` m

Redesign function configuration in `bindgen!` (#11328)

* Redesign function configuration in `bindgen!`

This commit is a redesign of how function-level configuration works in
Wasmtime's `bindgen!` macro. The main goal of this redesign is to
better support WASIp3 and component model async functions. Prior to this
redesign there was a mish mash of mechanisms to configure behavior of
imports/exports:

* The `async` configuration could turn everything async, nothing async,
only some imports async, or everything except some imports async.

* The `concurrent_{imports,exports}` keys were required to explicitly
opt-in to component model async signatures and applied to all
imports/exports.

* The `trappable_imports` configuration would indicate a list of imports
allowed to trap and it had special configuration for everything,
nothing, and only a certain list.

* The `tracing` and `verbose_tracing` keys could be applied to either
nothing or all functions.

Overall the previous state of configuration in `bindgen!` was clearly a
hodgepodge of systems that organically grew over time. In my personal
opinion it was in dire need of a refresh to take into account how
component-model-async ended up being implemented as well as
consolidating the one-off systems amongst all of these configuration
keys. A major motivation of this redesign, for example, was to inherit
behavior from WIT files by default. An `async` function in WIT should
not require `concurrent_*` keys to be configured, but rather it should
generate correct bindings by default.

In this commit, all of the above keys were removed. All keys have been
replaced with `imports` and `exports` configuration keys. Each behaves
the same way and looks like so:

bindgen!({
// ...
imports: {
// enable tracing for just this function
"my:local/interface/func": tracing,

// enable verbose tracing for just this function
"my:local/interface/other-func": tracing | verbose_tracing,

// this is blocking in WIT, but generate async bindings for
// it
"my:local/interface/[method]io.block": async,

// like above, but use "concurrent" bindings which have
// access to the store.
"my:local/interface/[method]io.block-again": async | store,

// everything else is, by default, trappable
default: trappable,
},
});

Effectively all the function-level configuration items are now bitflags.
These bitflags are by default inherited from the WIT files itself (e.g.
`async` functions are `async | store` by default). Further configuration
is then layered on top at the desires of the embedder. Supported keys are:

* `async` - this means that a Rust-level `async` function should be
generated. This is either `CallStyle::Async` or
`CallStyle::Concurrent` as it was prior, depending on ...

* `store` - this means that the generated function will have access to
the store on the host. This is only implemented right now for `async |
store` functions which map to `CallStyle::Concurrent`. In the future
I'd like to support just-`store` functions which means that you could
define a synchronous function with access to the store in addition to
an asynchronous function.

* `trappable` - this means that the function returns a
`wasmtime::Result<TheWitBindingType>`. If `trappable_errors` is
applicable then it means just a `Result<TheWitOkType,
TrappableErrorType>` is returned (like before)

* `tracing` - this enables `tracing!` integration for this function.

* `verbose_tracing` - this logs all argument values for this function
(including lists).

* `ignore_wit` - this ignores the WIT-level defaults of the function
(e.g. ignoring WIT `async`).

The way this then works is all modeled is that for any WIT function
being generated there are a set of flags associated with that function.
To calculate the flags the algorithm looks like:

1. Find the first matching rule in the `imports` or `exports` map
depending on if the function is imported or exported. If there is no
matching rule then use the `default` rule if present. This is the
initial set of flags for the function (or empty if nothing was
found).

2. If `ignore_wit` is present, return the flags from step 1. Otherwise
add in `async | store` if the function is `async` in WIT.

The resulting set of flags are then used to control how everything is
generated. For example the same split traits of today are still
generated and it's controlled based on the flags. Note though that the
previous `HostConcurrent` trait was renamed to `HostWithStore` to make
space for synchronous functions in this trait in the future too.

The end result of all these changes is that configuring imports/exports
now uses the exact same selection system as the `with` replacement map,
meaning there's only one system of selecting functions instead of 3.
WIT-level `async` is now respected by default meaning that bindings work
by default without further need to configure anything (unless more
functionality is desired).

One final minor change made here as well is that auto-generated
`instantiate` methods are now always synchronous and an
`instantiate_async` method is unconditionally generated for async mode.
This means that bindings always generate both functions and it's up to
the embedder to choose the appropriate one.

Closes #11246
Closes #11247

* Update expanded test expectations

prtest:full

* Fix the min platform embedding example

* Fix doc tests

* Always generate `*WithStore` traits

This helps when using the `with` mapping since that can always assume
that `HostWithStore` is available in the generated bindings, avoiding
the need to duplicate configuration options.

* Update test expectations

* Review comments

show more ...

Enable the `useless_conversion` Clippy lint (#10838)

* Enable the `useless_conversion` Clippy lint

We've got lots of types in Wasmtime and convert between them quite a
lot, but often over time conv

Enable the `useless_conversion` Clippy lint (#10838)

* Enable the `useless_conversion` Clippy lint

We've got lots of types in Wasmtime and convert between them quite a
lot, but often over time conversions become unnecessary through
refactorings or similar. This will hopefully enable us to clean up some
conversions as they come up to try to have as few as possible ideally.

* Review comments

show more ...

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtim

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtime to the 2024 Edition to keep
up-to-date with Rust idioms and access many of the edition features
exclusive to the 2024 edition.

prtest:full

* Reformat with the 2024 edition

show more ...

async/stream/future support for wasmtime-wit-bindgen (#10044)

* async/stream/future support for wasmtime-wit-bindgen

I've split this out of #9582 to make review easier.

This patch adds async/strea

async/stream/future support for wasmtime-wit-bindgen (#10044)

* async/stream/future support for wasmtime-wit-bindgen

I've split this out of #9582 to make review easier.

This patch adds async/stream/future/error-context support to the host binding
generator, along with placeholder type and function definitions in the
`wasmtime` crate which the generated bindings can refer to. See
https://github.com/dicej/rfcs/blob/component-async/accepted/component-model-async.md#componentbindgen-updates
for the design and rationale.

Note that I've added temporary `[patch.crates-io]` overrides in Cargo.toml until
https://github.com/bytecodealliance/wit-bindgen/pull/1130 and
https://github.com/bytecodealliance/wasm-tools/pull/1978 have been released.

Also note that we emit a `T: 'static` bound for `AsContextMut<Data = T>` when
generating bindings with `concurrent_imports: true`. This is only because
`rustc` insists that the closure we're passing to
`LinkerInstance::func_wrap_concurrent` captures the lifetime of `T` despite my
best efforts to convince it otherwise. Alex and I suspect this is a limitation
in the compiler, and I asked about it on the rust-lang Zulip, but we haven't
been able to determine a workaround so far.

Signed-off-by: Joel Dice <[email protected]>

remove obsolete TODO comment

Signed-off-by: Joel Dice <[email protected]>

make `futures` dep optional

Signed-off-by: Joel Dice <[email protected]>

update `wasm-tools` and `wit-bindgen`

Signed-off-by: Joel Dice <[email protected]>

* run cargo vet

Signed-off-by: Joel Dice <[email protected]>

---------

Signed-off-by: Joel Dice <[email protected]>

show more ...

Upgrade to wasm-tools 220 and wit-bindgen 0.35 releases (#9601)

* component-macro: paths are now an accessor on PackageSourceMap

* wast: WastArg enum is now non-exhaustive

and the error messages w

Upgrade to wasm-tools 220 and wit-bindgen 0.35 releases (#9601)

* component-macro: paths are now an accessor on PackageSourceMap

* wast: WastArg enum is now non-exhaustive

and the error messages were mostly written the wrong way around

* upgrade to wasm-tools 220 release

* supply chain: vet of unicode-width and automatic changes

* github actions: upgrade to cargo-vet 0.10.0

* wasmtime requires ahash feature on hashbrown, a transitive must have been ticking it prior

* upgrade to wit-bindgen 0.35

which uses wasm-tools 220, eliminating the duplicate deps in the lockfile.

* vet wit-bindgen upgrade

* fuzzing: add WasmFeatures::EXTENDED_CONF to expected features

wasm-smith added it in https://github.com/bytecodealliance/wasm-tools/pull/1861

show more ...

Configure WIT feature gates at runtime & implement wasi-cli `exit-with-code` (#9381)

* Remove `features` configuration from component::bindgen! and always emit all unstable members. These features w

Configure WIT feature gates at runtime & implement wasi-cli `exit-with-code` (#9381)

* Remove `features` configuration from component::bindgen! and always emit all unstable members. These features will be gated at runtime.

* Implement wasi-cli's unstable `exit-with-code`

* Add codegen test for unstable features

* Add LinkOptions type and add a parameter to all add_to_linker functions in for worlds/interfaces that use any unstable feature.

* More descriptive test feature names.

* Generate feature gate `if` checks

* Expose `cli-exit-with-code` as CLI option

* Generate bespoke option types per interface and world.

* Add unit test

* Remove exit code restriction on Windows

* Add cli_exit_with_code test

* Use BTreeSet to generate the options in consistent order

* Change wasmtime-wasi's `add_to_linker_(a)sync` signature back to how it was and add new variants that take the option parameters.

* Lift Windows exit code restriction in tests

* Lift Windows exit code restriction

show more ...

bug: bindgen erraneously parses extra comma (#9322)

Co-authored-by: Lochlan Wansbrough <>

feat: component bindgen: add support for multiple wit paths (#9288)

* feat: bindgen: add support for multiple wit paths

* expanded tests

* fmt

---------

Co-authored-by: Lochlan Wansbrough <>

wit-bindgen: Don't trace values containing lists by default. (#9262)

* wit-bindgen: Don't trace values containing lists by default.

`list` values in Wit interfaces can represent things like HTTP bo

wit-bindgen: Don't trace values containing lists by default. (#9262)

* wit-bindgen: Don't trace values containing lists by default.

`list` values in Wit interfaces can represent things like HTTP bodies
which can be very large. To enable tracing without spamming logfiles
with all this data, put printing of values containing `list`s behind
a separate `verbose_tracing` option.

This is a coarse-grained approach; but it seems like a pretty good
default for tracing, and enabling full tracing when one needs it is
straightforward.

In the future, we may want to refine the option by implementing
the `Valuable` trait and using `tracing::field::valuable`, which
could allow us to do things like print non-`list` fields of records
that otherwise contain `list`s.

* Use `option_type_contains_lists` more.

* Hook up `verbose_tracing` to the macro.

* Update expected outputs for tests.

show more ...

Update wasm-tools to 215 (#9053)

Pulling in some changes notably to multi-package `*.wit` files.

prtest:full

component-macro: normalize the path parameter in component bindgen (#8871)

upgrade to wasm-tools 0.211.1 (#8838)

* upgrade to wasm-tools 0.211.1

* code review

* cargo vet: auto imports

* fuzzing: fix wasm-smith changes

* fuzzing: changes for HeapType

* Configure featu

upgrade to wasm-tools 0.211.1 (#8838)

* upgrade to wasm-tools 0.211.1

* code review

* cargo vet: auto imports

* fuzzing: fix wasm-smith changes

* fuzzing: changes for HeapType

* Configure features on `Parser` when parsing

---------

Co-authored-by: Alex Crichton <[email protected]>

show more ...

Overhaul and improve documentation of `bindgen!` (#8727)

This is another take at improving the documentation for `bindgen!` in
Wasmtime. This commit takes a leaf out of the book of
bytecodealliance/

Overhaul and improve documentation of `bindgen!` (#8727)

This is another take at improving the documentation for `bindgen!` in
Wasmtime. This commit takes a leaf out of the book of
bytecodealliance/wit-bindgen#871 to organize the documentation of the
macro a bit more rather than having one giant doc block that can be
difficult to explore. The macro's documentation itself is now mostly a
reference of all the options that can be specified. There is now a new
documentation-only module which serves a few purposes:

* Individual examples are organized per-submodule to be a bit more
digestable.
* Each example has an example of the generated code in addition to the
source code used for each example.
* All examples are tested on CI to compile (none are run).

My hope is that this makes it easier to expand the docs here further
over time with niche features as they arise or with various options that
the macro has. This is one of the lynchpins of Wasmtime's support for
the component model so it seems pretty important to have a good
onboarding experience here.

Along the way I've implemented a few more niche options for the
`bindgen!` macro that I found necessary, such as configuring the
`wasmtime` crate and where it's located.

show more ...

Always generate the same output structure with `bindgen!` (#8721)

Currently with the `bindgen!` macro when the `with` key is used then the
generated bindings are different than if the `with` key was

Always generate the same output structure with `bindgen!` (#8721)

Currently with the `bindgen!` macro when the `with` key is used then the
generated bindings are different than if the `with` key was not used.
Not only for replacement purposes but the generated bindings are missing
two key pieces:

* In the generated `add_to_linker` functions bounds and invocations of
`with`-overridden interfaces are all missing. This means that the
generated `add_to_linker` functions don't actually represent the full
world.

* The generated module hierarchy has "holes" for all the modules that
are overridden. While it's mostly a minor inconvenience it's also easy
enough to generate everything via `pub use` to have everything hooked
up correctly.

After this PR it means that each `bindgen!` macro should, in isolation,
work for any other `bindgen!` macro invocation. It shouldn't be
necessary to weave things together and remember how each macro was
invoked along the way. This is primarily to unblock #8715 which is
running into a case where tcp/udp are generated as sync but their
dependency, `wasi:sockets/network`, is used from an upstream async
version. The generated `add_to_linker` does not compile because tcp/udp
depend on `wasi:sockets/network` isn't added to the linker. After fixing
that it then required more modules to be generated, hence this PR.

show more ...

Update `wit-bindgen`, add `features` to `bindgen!` (#8720)

This commit updates the wit-bindgen family of crates and additionally
adds a `features` key to the `bindgen!` macro. This brings it in line

Update `wit-bindgen`, add `features` to `bindgen!` (#8720)

This commit updates the wit-bindgen family of crates and additionally
adds a `features` key to the `bindgen!` macro. This brings it in line
with `wit-bindgen`'s support which enables usage of the new `@since` and
`@unstable` features of WIT.

show more ...

Add `GetHost` to generated bindings for more flexible linking (#8448)

* Remove unused generated `add_root_to_linker` method

* WIP: bindgen GetHost

* Compile with Rust 1.78+

Use <https://users.rus

Add `GetHost` to generated bindings for more flexible linking (#8448)

* Remove unused generated `add_root_to_linker` method

* WIP: bindgen GetHost

* Compile with Rust 1.78+

Use <https://users.rust-lang.org/t/generic-closure-returns-that-can-capture-arguments/76513/3>
as a guide of how to implement this by making the `GetHost` trait a bit
uglier.

* Add an option to skip `&mut T -> T` impls

Also enable this for WASI crates since they do their own thing with
`WasiView` for now. A future refactoring should be able to remove this
option entirely and switch wasi crates to a new design of `WasiView`.

* Update test expectations

* Review comments

* Undo temporary change

* Handle some TODOs

* Remove no-longer-relevant note

* Fix wasmtime-wasi-http doc link

---------

Co-authored-by: Alex Crichton <[email protected]>

show more ...

bindgen: Commit expanded bindgen output for tests (#8558)

These outputs are checked in and verified to be fresh by a test so that
they can be relied on for code review.

Support additional_derives option in bindgen (#8441)

Signed-off-by: Brian H <[email protected]>

Error when `wasmtime::component::bindgen` has unused with param. (#8371)

Instead of silently dropping unused with parameters on the floor, we
instead error letting the user know that this option is

Error when `wasmtime::component::bindgen` has unused with param. (#8371)

Instead of silently dropping unused with parameters on the floor, we
instead error letting the user know that this option is being ignored.

Fixes https://github.com/bytecodealliance/wasmtime/issues/8304

show more ...

Disable traps by default in `bindgen!` imports (#8310)

By default previously all return types were wrapped in
`wasmtime::Result<T>` to enable any import to return a trap to the wasm
guest. This is a

Disable traps by default in `bindgen!` imports (#8310)

By default previously all return types were wrapped in
`wasmtime::Result<T>` to enable any import to return a trap to the wasm
guest. This is a fair bit of boilerplate, however, and it's easy to
accidentally turn a normal error into a trap. This is additionally
somewhat of a power-user method as many imports probably don't end up
wanting to trap.

This commit adds a new configuration option to `bindgen!`:
`trappable_imports`, and switches the default to `false`. The previous
behavior can be recovered with `trappable_imports: true`.

show more ...