Migrate the `wasmtime` crate to `wasmtime_environ::error::*` (#12231)

* Migrate the `wasmtime` crate to `wasmtime_environ::error::*`

Instead of `anyhow::Error`.

This commit re-exports the `wasmtim

Migrate the `wasmtime` crate to `wasmtime_environ::error::*` (#12231)

* Migrate the `wasmtime` crate to `wasmtime_environ::error::*`

Instead of `anyhow::Error`.

This commit re-exports the `wasmtime_environ::error` as the `wasmtime::error`
module, updates the prelude to include these new error-handling types, redirects
our top-level `wasmtime::{Error, Result}` re-exports to re-export
`wasmtime::error::{Error, Result}`, and updates various use sites that were
directly using `anyhow` to use the new `wasmtime` versions.

This process also required updating the component macro and wit-bindgen macro to
use the new error types instead of `anyhow`.

Part of https://github.com/bytecodealliance/wasmtime/issues/12069

* Replace wasmtime::error::Thing with wasmtime::Thing where it makes sense

* cargo fmt

* Move `crate::error::Thing` to `crate::Thing` where it makes sense

show more ...

update `component-model-async` plumbing (#11123)

* [DO NOT MERGE] update `component-model-async` plumbing

This pulls in the latest Component Model async ABI code from the
`wasip3-prototyping` repo,

update `component-model-async` plumbing (#11123)

* [DO NOT MERGE] update `component-model-async` plumbing

This pulls in the latest Component Model async ABI code from the
`wasip3-prototyping` repo, including various API refactors and spec updates.

This includes all the changes to the `wasmtime` crate from `wasip3-prototyping`
_except_ that the `concurrent` submodule and child submodules contain only
non-functional stubs. For that reason, and the fact that
e.g. `Func::call_async` is now implemented in terms of `Func::call_concurrent`,
most of the component model tests are failing. This commit is not meant to be
merged as-is; a follow-up commit (to be PR'd separately) will contain the real
`concurrent` implementation, at which point the tests will pass again. I'm
splitting these into separate PRs to make review easier.

Signed-off-by: Joel Dice <[email protected]>

* Undo wit-bindgen changes

No longer necessary after other refactors

* Move back to crates.io-based wit-bindgen

* Undo upgrade of http-body-util

(deferred for future PR)

* Add back in arbitrary use of async

Looks like it may have been lost by accident

* Make imports more conventional for Wasmtime

* Some minor changes

* Privatize a component field

* Cut down a bit on #[cfg]

* Undo a no-longer-necessary `pub`

* add doc comments for `{Future,Stream,ErrorContext}Any`

Signed-off-by: Joel Dice <[email protected]>

* rename `concurrent` stub module to `concurrent_disabled`

...and avoid panicking in the stubs.

Signed-off-by: Joel Dice <[email protected]>

* fix test regression

Signed-off-by: Joel Dice <[email protected]>

* revert `call_async` and `post_return_impl` changes

These will need to wait until the `component-model-async` feature is fully
implemented.

Signed-off-by: Joel Dice <[email protected]>

* remove unused struct

Signed-off-by: Joel Dice <[email protected]>

* add `Options::callback` field

This isn't used yet, but will be used when the real `component-model-async`
implementation is merged.

Signed-off-by: Joel Dice <[email protected]>

* Remove no-longer-needed feature

* Trim reexports from Wasmtime

Some of these are no longer needed or can be avoided with small changes.
Some deps are likely needed in the next commit but they'll be best added
there.

* Update test expectations

* More trimming of Cargo.toml

* Defer `*Buffer` traits to next PR

Not needed for this PR I believe.

* Use conventional Wasmtime imports + remove dummy_waker

* Reduce duplication in `*_disabled`

* Remove some unncessary bounds

* Remove some `for<'a>` bounds where unnecessary

* Remove another bound

* Defer more functions to the next PR

`drop_fibers` is different in the next PR, so defer it to then.

* Remove some reexports no longer necessary

Bindings generation changed awhile back so these aren't needed, defer
the implementations to the next PR.

* Remove unnecessary drop

This was already moved to `run_manual_drop_routines`

* Defer a `pub(crate)` to a future PR

* Expand comments in traphandlers

* Defer some types to the next PR

* Update linker documentation

* Add `Send`/`Sync` bounds to `ComponentType`

This commit is extracted from from review of #11123 and #11127. While
not literally present in those PRs it's my own personal conclusion that
it's best to just go ahead and add these bounds at the "base" of the
component trait hierarchy. The current implementation in #11123 adds
bounds in many locations and this would remove the need to add bounds
everywhere and instead have everything inherited through the `Lift` and
`Lower` traits.

This raises the question of: why? The main conclusion that I've reached
leading to this change is that Wasmtime currently will store `R`, a
return value, on the stack during the lowering process back into linear
memory. This might involve allocation, however, meaning that wasm can be
invoked and a context switch could happen. For Wasmtime's `unsafe impl`
of `Send` and `Sync` on fibers to be sound it requires that this
stack-local variable is also `Send` and `Sync` as it's an entirely
user-provided type. Thus I've concluded that for results it's always
required for these to be both `Send` and `Sync` (or at the very least,
`Send`).

Given that I've gone ahead and updated to require both `Send` and `Sync`
for both params and results. This is not expected to actually have any
impact in practice since all primitives are already `Send`/`Sync` (minus
`Rc` impls all removed here) and all `bindgen!`-generated types are
compositions of `Send`/`Sync` primitives meaning that they're also
`Send` and `Sync`.

* Remove some now-unnecessary bounds

* Fix build after #11160

* Remove some now-unnecessary duplicate bounds

* Uncomment test that now works

* Undo accidental doc wrap

* Clarify comment on `Value` types

Don't leave `TODO` in public-facing documentation ideally

* Actually resolve the conflict (forgot to commit)

* Avoid returning boxed futures in APIs

* Defer making constructors more public to a future PR

* Make a method name more conventional

* Refactor `linear_lift_into_from_memory`

* Drop the `max_count` parameter in favor of slicing the `WasmList`
itself. Avoids situations such as what happens if `max_count` is
larger than the length of the list.
* Don't have the default implementation collect to a vector and then
push all that onto a different vector. Instead push each item
individually through `extend`.

* De-indent a block of code added

* Remove unsafety from `prepare_call`

Mostly move the parameters themselves to the closure to avoid raw
pointers/drop/etc.

This will have the consequence of in the future `call_async` is going to
now require `Params: 'static` but that seems more-or-less inevitable at
this point.

* Go back to returning box, alas.

* Apply same treatment to lift function

Make it a closure and reduce some levels of indirection of the various
functions in play.

* Refactor `lower_params` to require less context.

Relax the bounds on the closure specified since it's immediately called
and then additionally take out parameters/captures that the closure can
carry itself.

* Don't pass extraneous `Instance` parameter

This can now be inferred from `Func`.

* Clean up some SAFETY comments

* Generalize the signature of `lift_results`

* Move `lift_results` function to `Func`

Also rename the lift/lower helpers to `with_{lift,lower}_context`

* Remove parameter from `with_lift_context`

Like `with_lower_context` this is fine to capture in the closure passed
in.

* Simplify the dynamic lifting logic

Don't call `with_lift_context` in two locations, only call it once with
a dynamic parameter.

* Refactor away the `Func::lift_results_sync` helper

* Use `with_lift_context` in `call_raw`

* Simplify a call to `Func::call_unchecked_raw`

* Ungate `with_lift_context` to fix non-cm-async build

* Fix compile (bad cherry-pick conflict resolution)

* Use `with_lower_context` in `call_raw`

Trying to unify the async/concurrent paths as much as possible.

* Move params out of `call_raw`

Let closures capture the params, no need to thread it through as an
unnecessary argument.

* Clean up unsafety in `Func::call_raw`

* Accurately mark `call_raw` itself as `unsafe`, then document why
callers should be safe.
* Don't have one large `unsafe` block in `call_raw`, instead split it up
with separate safety comments.

* Move a one-off type definition closer to its use

* Avoid intermediate allocations in dynamic calls

* Simplify a future-return site

* Deduplicate checking parameter count

* Simplify a future invocation with `?`

* Simplify a variable declaration

* Simplify some function signatures

* Remove outdated safety comment

* Refactor to not require `Params: 'static` on `call_async`

* Remove no-longer-necessary SAFETY comment

* Fix typos

* Add a fast-path with no `Box` for sync host functions

Speeds up host calls by ~20% and puts them back on parity with the
beforehand numbers Wasmtime has.

* Synchronize signatures of async/concurrent dynamic calls

Use slices for both instead of vecs for one and slices for the other.
Required some slight rejiggering. Apparently one can solve a closure
problem with another closure, then one surely has no more closure
problems.

* Fix non-cm-async build

---------

Signed-off-by: Joel Dice <[email protected]>
Co-authored-by: Alex Crichton <[email protected]>

show more ...

async/stream/future support for wasmtime-wit-bindgen (#10044)

* async/stream/future support for wasmtime-wit-bindgen

I've split this out of #9582 to make review easier.

This patch adds async/strea

async/stream/future support for wasmtime-wit-bindgen (#10044)

* async/stream/future support for wasmtime-wit-bindgen

I've split this out of #9582 to make review easier.

This patch adds async/stream/future/error-context support to the host binding
generator, along with placeholder type and function definitions in the
`wasmtime` crate which the generated bindings can refer to. See
https://github.com/dicej/rfcs/blob/component-async/accepted/component-model-async.md#componentbindgen-updates
for the design and rationale.

Note that I've added temporary `[patch.crates-io]` overrides in Cargo.toml until
https://github.com/bytecodealliance/wit-bindgen/pull/1130 and
https://github.com/bytecodealliance/wasm-tools/pull/1978 have been released.

Also note that we emit a `T: 'static` bound for `AsContextMut<Data = T>` when
generating bindings with `concurrent_imports: true`. This is only because
`rustc` insists that the closure we're passing to
`LinkerInstance::func_wrap_concurrent` captures the lifetime of `T` despite my
best efforts to convince it otherwise. Alex and I suspect this is a limitation
in the compiler, and I asked about it on the rust-lang Zulip, but we haven't
been able to determine a workaround so far.

Signed-off-by: Joel Dice <[email protected]>

remove obsolete TODO comment

Signed-off-by: Joel Dice <[email protected]>

make `futures` dep optional

Signed-off-by: Joel Dice <[email protected]>

update `wasm-tools` and `wit-bindgen`

Signed-off-by: Joel Dice <[email protected]>

* run cargo vet

Signed-off-by: Joel Dice <[email protected]>

---------

Signed-off-by: Joel Dice <[email protected]>

show more ...

Add async example to bindgen_examples (#9822)

* add async example

* fix bindgen bug to make CI happy

See https://github.com/bytecodealliance/wasmtime/pull/9822#issuecomment-2546851949

* update te

Add async example to bindgen_examples (#9822)

* add async example

* fix bindgen bug to make CI happy

See https://github.com/bytecodealliance/wasmtime/pull/9822#issuecomment-2546851949

* update test expectations

* remove async_trait

show more ...

Expand the set of constructors of `bindgen!`-generated bindings (#9177)

* Expand the set of constructors of `bindgen!`-generated bindings

This commit expands the set of supported constructors for `

Expand the set of constructors of `bindgen!`-generated bindings (#9177)

* Expand the set of constructors of `bindgen!`-generated bindings

This commit expands the set of supported constructors for `bindgen!`
generated structures with the goal of bringing back
`World::new(&mut store, instance)`. This method was removed previously
in a refactoring to add `*Pre` structures but refactoring preexisting
code to use `*Pre` isn't always easy, so these extra generated bindings
provide a smoother migration path for code from before.

* Update test expectations

show more ...

Reduce typo count (#8951)

Refactor and document the wasmtime-wasi-http more (#8861)

* Improve some documentation of the `wasmtime-wasi` crate

Show a few examples of using `with` to point to upstream `wasmtime-wasi`
for bind

Refactor and document the wasmtime-wasi-http more (#8861)

* Improve some documentation of the `wasmtime-wasi` crate

Show a few examples of using `with` to point to upstream `wasmtime-wasi`
for bindings.

* Refactor and document the `wasmtime-wasi-http` more

This commit primarily adds a complete example of using
`wasmtime-wasi-http` to the documentation. Along the way I've done a
number of other refactorings too:

* `bindgen!`-generated `*Pre` structures now implement `Clone`.
* `bindgen!`-generated `*Pre` structures now have an `engine` method.
* `bindgen!`-generated `*Pre` structures now have an `instance_pre` method.
* The structure of `wasmtime-wasi-http` now matches `wasmtime-wasi`,
notably:
* The `proxy` module is removed
* `wasmtime_wasi_http::add_to_linker_{a,}sync` is the top level
add-to-linker function.
* The `bindings` module now contains `Proxy` and `ProxyPre` along with
a `sync` submodule.
* The `bindings` module contains all bindings for `wasi:http` things.
* The `add_only_*` methods are un-hidden and documented.
* Code processing `req` has been simplified by avoiding
decomposing-and-reconstructing a request.
* The `new_incoming_request` method is now generic to avoid callers
having to do boxing/mapping themselves.

* Update expanded macro expectations

* Remove unused import

show more ...

Redesign how component exports work (#8786)

* Un-nest exports in a component

This commit flattens the representation of exports in a component to
make them more easily indexable without forcing tra

Redesign how component exports work (#8786)

* Un-nest exports in a component

This commit flattens the representation of exports in a component to
make them more easily indexable without forcing traversal through the
hierarchy of instance imports/exports to get there.

* Guarantee type information on component exports

Don't have it optional in some cases and present in others, instead
ensure there's type information for all component exports immediately
available.

* Refactor how component instance exports are loaded

This commit is a change to Wasmtime's public API for
`wasmtime::component::Instance` that reorganizes how component exports
are loaded. Previously there was a system where `Instance::exports()`
was called that that was sort of "iterated over" in a builder-style
pattern to acquire the actual export desired. This required lifetime
trickery for nested instances and some unfortunate API bloat. The major
downside of this approach is that it requires unconditional string
lookups at runtime for exports and additionally does not serve as a
great place to implement the semver-compatible logic of #8395. The goal
of this refactoring is to pave the way to improving this.

The new APIs for loading exports now look a bit more similar to what's
available for core modules. Notably there's a new
`Component::export_index` method which enables performing a string
lookup and returning an index. This index can in turn be passed to
`Instance::get_*` to skip the string lookup when exports are loaded. The
`Instance::exports` API is then entirely removed and dismantled.

The only piece remaining is the ability to load nested exports which is
done through an `Option` parameter to `Component::export_index`. The
way to load a nested instance is now to first lookup the instance with
`None` as this parameter an then the instance itself is `Some` to look
up an export of that instance. This removes the need for a
recursive-style lifetime-juggling API from wasmtime and in theory helps
simplify the usage of loading exports.

* Update `bindgen!` generated structures for exports

This commit updates the output of `bindgen!` to have a different setup
for exports of worlds to handle the changes from the previous commit.
This introduces new `*Pre` structures which are generated alongside the
existing `Guest` structures for example. The `*Pre` versions contain
`ComponentExportIndex` from the previous commit and serve as a path to
accelerating instantiation because all name lookups are skipped.

* Update test expectations for `bindgen!`-generated output

* Review comments

* Fix doc link

show more ...

Add `GetHost` to generated bindings for more flexible linking (#8448)

* Remove unused generated `add_root_to_linker` method

* WIP: bindgen GetHost

* Compile with Rust 1.78+

Use <https://users.rus

Add `GetHost` to generated bindings for more flexible linking (#8448)

* Remove unused generated `add_root_to_linker` method

* WIP: bindgen GetHost

* Compile with Rust 1.78+

Use <https://users.rust-lang.org/t/generic-closure-returns-that-can-capture-arguments/76513/3>
as a guide of how to implement this by making the `GetHost` trait a bit
uglier.

* Add an option to skip `&mut T -> T` impls

Also enable this for WASI crates since they do their own thing with
`WasiView` for now. A future refactoring should be able to remove this
option entirely and switch wasi crates to a new design of `WasiView`.

* Update test expectations

* Review comments

* Undo temporary change

* Handle some TODOs

* Remove no-longer-relevant note

* Fix wasmtime-wasi-http doc link

---------

Co-authored-by: Alex Crichton <[email protected]>

show more ...

bindgen: Commit expanded bindgen output for tests (#8558)

These outputs are checked in and verified to be fresh by a test so that
they can be relied on for code review.