Relax panics in async/futures to traps/errors (#12688)

* Relax panics in async/futures to traps/errors

This commit is an admittance that I don't believe we're going to get
to a point where we are c

Relax panics in async/futures to traps/errors (#12688)

* Relax panics in async/futures to traps/errors

This commit is an admittance that I don't believe we're going to get
to a point where we are confident enough in the fuzzing of
component-model-async such that we could confidently say we're
exercising the vast majority of possible panics. Development of
component-model-async has shown a steady trickle of panics over the
course of the development of the feature, and this trend has been
persistent over time as well.

An attempt was made in #12119 to add a fuzzer dedicated to async events
but that didn't actually find anything in development and it has missed
a number of panics present before and discovered after its introduction.
Overall I do not know how to improve the fuzzer to the point that it
would find pretty much all of the existing async-related panics over
time.

To help address this concern of the `concurrent.rs` implementation this
commit goes through and replaces things like `unwrap()`, `assert!`,
`panic!`, and `unreachable!` with an error-producing form. The benefit
of this is that a bug in the implementation is less likely to result in
a panic and instead just results in a non-spec-compliant trap. The
downside of doing this though is that it can become unclear what errors
are "first class traps", or expected to be guest reachable, and which
are expected to be bugs in Wasmtime. To help address this I've performed
a few refactorings here as well.

* Some traps previously present as error strings are now promoted to
using `Trap::Foo` instead. This has some refactoring of the Rust/C
side as well to make it easier to define new variants. Tests were
additionally added for any trap messages that weren't previously
tested as being reachable.

* A new `bail_bug!` macro was added (internally) for Wasmtime. This is
coupled with a concrete `WasmtimeBug` error type (exported as
`wasmtime::WasmtimeBug`). The intention is that `bail!` continues to
be "here's a string and I'm a bit too lazy to make a concrete error"
while `bail_bug!` indicates "this is a bug in wasmtime please report
this if you see it".

The rough vision is that if an error condition is reached, and the system
is not broken in such a way that panicking is required, then `bail_bug!`
can be used to indicate a bug in Wasmtime as opposed to panicking. This
reduces the real-world impact of hitting these scenarios by downgrading a
CVE-worthy `panic!` into a bug-worthy non-spec-compliant trap. Not all
panics are able to be transitioned to this as some are load bearing from
a safety perspective or similar (or indicate something equally broken),
but the vast majority of cases are suitable for "return a trap, lock
down the store, and let destructors take care of everything else".

This change additionally has resulted in API changes for `FutureReader`
and `StreamReader`. For example creation of these types now returns a
`Result` for when the `ResourceTable` is full, for example, instead of
panicking.

* Fix CI build

* Translate `WasmtimeBug` to panics in debug mode

* Review comments

* Refactor some stream methods for fewer panics

show more ...

Remove subtask reparenting (#12570)

This commit updates the implementation of component-model-async
primitives to remove the manual subtask reparenting process. This is
required to fix #12544 at a s

Remove subtask reparenting (#12570)

This commit updates the implementation of component-model-async
primitives to remove the manual subtask reparenting process. This is
required to fix #12544 at a semantic level because a subtask isn't ever
actually reparented, even if its parent exits. The first part of this
change is to remove the `GuestTask::subtasks` field and all relevant
manipulations of it.

This field, however, powered the `TaskExit` abstraction returned from
`call_concurrent`. This commit then subsequently deletes `TaskExit` and
all related infrastructure as it's no longer directly applicable as-is.
The rest of this change is then updating tests/bindings/etc to account
for these two changes.

The main semantic changes related to tests are:

* `wasmtime run`, with and without `--invoke`, no longer waits for all
subtasks. This instead only waits for the main task returning before
exiting. Whether or not this is the correct behavior is under
discussion in WebAssembly/component-model#608

* `wasmtime serve` has been updated to keep the store alive at least
until the response body has been fully transmitted. This is also part
of WebAssembly/component-model#608.

* Some `component-async-tests`-related tests were updated to either
avoid blocking the store as it wasn't needed or yield enough times to
ensure that the test passes.

Closes #12544

prtest:full

show more ...

Yield instead of sleep in component-async-tests (#12567)

* Yield instead of sleep in component-async-tests

Use cooperative yields instead of sleeps to make tests more
deterministic and also avoid t

Yield instead of sleep in component-async-tests (#12567)

* Yield instead of sleep in component-async-tests

Use cooperative yields instead of sleeps to make tests more
deterministic and also avoid them unnecessarily taking up test
parallelism by sleeping. Yielding should have the same effect in terms
of testing by exercising behavior returning `Pending` in futures, so
there's no expected loss in test coverage here.

* Yield fewer times

show more ...

deprecate `[Typed]Func::post_return[_async]` and make them no-ops (#12498)

* deprecate `[Typed]Func::post_return[_async]` and make them no-ops

With the advent of the Component Model concurrency ABI

deprecate `[Typed]Func::post_return[_async]` and make them no-ops (#12498)

* deprecate `[Typed]Func::post_return[_async]` and make them no-ops

With the advent of the Component Model concurrency ABI and it's `task.return`
intrinsic, post-return functions have been informally deprecated and are
expected to be removed for WASI 1.0 and the corresponding stable edition of the
Component Model. Consequently, it does not make sense anymore to require
embedders to explicitly call the post-return function after using
`[Typed]Func::call[_async]`.

As of this commit, `[Typed]Func::post_return[_async]` are no-ops. Instead, the
post-return function is called automatically as part of
`[Typed]Func::call[_async]` if present, which is how
`[Typed]Func::call_concurrent` has worked all along. In addition, this commit
fixes and tests a couple of cases where the task and/or thread was being
disposed of before the post-return function was called.

* address review feedback

* test post-return function in more scenarios

Specifically, I've split the `invoke_post_return` test into multiple tests:

- using `TypedFunc::call`
- using `TypedFunc::call_async` with concurrency support enabled
- using `TypedFunc::call_async` with concurrency support disabled
- using `Func::call_async` with concurrency support disabled
- using `TypedFunc::call_concurrent`

* remove GCC/clang-specific deprecation attribute

This broke the MSVC build.

* bless bindgen output

* remove obsolete post-return functions and fields

Now that post-return calls are handled internally without requiring explicit
action by the embedder, we can avoid unnecessary bookkeeping.

show more ...

Remove need for explicit `Config::async_support` knob (#12371)

* Refactor component model host function definitions

Push the `async`-ness down one layer.

* Remove need for explicit `Config::async

Remove need for explicit `Config::async_support` knob (#12371)

* Refactor component model host function definitions

Push the `async`-ness down one layer.

* Remove need for explicit `Config::async_support` knob

This commit is an attempt to step towards reconciling "old async" and
"new async" in Wasmtime. The old async style is the original async
support in Wasmtime with `call_async`, `func_wrap_async`, etc, where the
main property is that the store is "locked" during an async operation.
Put another way, a store can only execute at most one async operation at
a time. This is in contrast to "new async" support in Wasmtime with the
component-model-async (WASIp3) support, where stores can have more than
one async operation in flight at once.

This commit does not fully reconcile these differences, but it does
remove one hurdle along the way: `Config::async_support`. Since the
beginning of Wasmtime this configuration knob has existed to explicitly
demarcate a config/engine/store as "this thing requires `async` stuff
internally." This has started to make less and less sense over time
where the line between sync and async has become more murky with WASIp3
where the two worlds comingle. The goal of this commit is to deprecate
`Config::async_support` and make the function not actually do anything.

In isolation this can't simply be done, however, because there are many
load-bearing aspects of Wasmtime that rely on this `async_support` knob.
For example once epochs + yielding are enabled it's required that all
Wasm is executed on a fiber lest it hit an epoch and not know how to
yield. That means that this commit is not a simple removal of
`async_support` but instead a refactoring/rearchitecting of how async is
used internally within Wasmtime. The high-level ideas within Wasmtime
now are:

* A `Store` has a "requires async" boolean stored within it.
* All configuration options which end up requiring async, such as
yielding with epochs, turn this boolean on.
* Creation of host functions which use async
(e.g. `func_wrap_{async,concurrent}`) will also turn this option on.
* Synchronous API entrypoints into Wasmtime ensure that this boolean is
disabled.
* Asynchronous APIs are usable at any time.

This means that the concept of an async store vs a sync store is now
gone. All stores are equally capable of executing sync/async, and the
change now is that dynamically some stores will require that async is
used with certain configuration. Additionally all panicking conditions
around `async_support` have been converted to errors instead. All
relevant APIs already returned an error and things are murky enough now
that it's not necessarily trivial to get this right at the embedder
level. In the interest of avoiding panics all detected async mismatches
are now first-class `wasmtime::Error` values.

The end result of this commit is that `Config::async_support` is a
deprecated `#[doc(hidden)]` function that does nothing. While many
internal changes happened as well as having new tests for all this sort
of behavior this is not expected to have a great impact on external
consumers. In general a deletion of `async_support(true)` is in theory
all that's required. This is intended to make it easier to think about
async/sync/etc in the future with WASIp3 and eventually reconcile
`func_wrap_async` and `func_wrap_concurrent` for example. That's left
for future refactorings however.

prtest:full

* Review comments

* Fix CI failures

show more ...

make async tests using `ready` interface more robust (#12360)

This changes the `ready` interface used by `component-async-tests` from:

```
interface ready {
// Set the `ready` state
set-ready:

make async tests using `ready` interface more robust (#12360)

This changes the `ready` interface used by `component-async-tests` from:

```
interface ready {
// Set the `ready` state
set-ready: func(ready: bool);
// Block until `ready` is `true`
when-ready: async func();
}
```
to:
```
interface ready {
resource thing {
constructor();
set-ready: func(ready: bool);
when-ready: async func();
}
}
```

The problem with the original version was that it required global state and thus
caused cross-talk across concurrent tasks. Due to implementation details inside
Wasmtime, the tests worked anyway, but
https://github.com/bytecodealliance/wasmtime/pull/12357 perturbed that and
revealed how fragile tests based on that interface were.

The new version puts the state inside a resource type, allowing each task create
its own instance of that resource type and thereby avoid crosstalk.

show more ...

refactor recursive reentrance checks (#12349)

* refactor recursive reentrance checks

This commit makes a few changes related to recursive reentrance checks, instance
poisoning, etc.:

- Implements

refactor recursive reentrance checks (#12349)

* refactor recursive reentrance checks

This commit makes a few changes related to recursive reentrance checks, instance
poisoning, etc.:

- Implements the more restrictive lift/lower rules described in https://github.com/WebAssembly/component-model/pull/589 such that a component instance may not lower a function lifted by one of its ancestors, nor vice-versa. Any such lower will result in a fused adapter which traps unconditionally, preventing guest-to-guest recursive reentrance without requiring data flow analysis.
- Note that this required updating several WAST tests which were violating the new rule, including some in the `tests/component-model` Git submodule, which I've updated.
- This is handled entirely in the `fact` module now; I've removed the `AlwaysTrap` case previously handled by `wasmtime-cranelift`.
- Removes `FLAG_MAY_ENTER` from `InstanceFlags`. It is no longer needed for guest-to-guest calls due to the above, and for guest-to-host-to-guest calls we can rely on either `FLAG_NEEDS_POST_RETURN` for sync-lifted functions or the `GuestTask` call stack for async-lifted functions.
- Adds a `StoreOpaque::trapped` field which is set when _any_ instance belonging to that store traps, at which point the entire store is considered poisoned, meaning no instance belonging to it may be entered. This prevents indeterminant concurrent task state left over from the trapping instance from leaking into other instances.

Note that this does _not_ include code to push and pop `GuestTask` instances for
guest-to-guest sync-to-sync calls, nor for host-to-guest calls using e.g. the
synchronous `Func::call` API, so certain intrinsics which expect a `GuestTask`
to be present such as `backpressure.inc` will still fail in such cases. I'll
address that in a later PR.

Also note that I made a small change to `wasmtime-wit-bindgen`, adding a `Send`
bound on the `T` type parameter for `store | async` functions. This allowed me
to recursively call `{Typed}Func::call_concurrent` from inside a host function,
and it doesn't have any downsides AFAICT.

Fixes #12128

* bless bindgen expansions

* bless disas tests

* address review feedback

* sync `trap.h` with `trap_encoding.rs`

...and add const assertions to `trap.rs` to help avoid future divergence.

show more ...

Polyfill `ToWasmtimeResult` and add eventually-necessary call sites (#12308)

* Polyfill `ToWasmtimeResult` and add eventually-necessary call sites

This commit polyfills `wasmtime_internal_error::To

Polyfill `ToWasmtimeResult` and add eventually-necessary call sites (#12308)

* Polyfill `ToWasmtimeResult` and add eventually-necessary call sites

This commit polyfills `wasmtime_internal_error::ToWasmtimeResult` in
`wasmtime_environ`, adds the cargo feature plumbing that will eventually connect
to the `"wasmtime_internal_error/anyhow"` cargo feature but for now just
configures this polyfill, and adds uses of the polyfill at all the sites that
will be necessary once we swap out `anyhow` for
`wasmtime_internal_error`. Currently, the polyfill is just an identity function,
because `wasmtime::Result`/`wasmtime_environ::error::Result` is just another
name for `anyhow::Result`. Once we do move away from `anyhow`, however, that
will no longer be the case, and these uses will do the necessary conversion.

My goal with landing this as an incremental commit is to make it so that the
actual commit that does the error crate swap out can be as close to _just_ the
`Cargo.toml` dependency change, without any other code changes as much as
possible. This, in turn, means that swap out should be super easy to revert, if
necessary.

* Add a couple more `.to_wasmtime_result()` invocations in fuzz code

* Revert "Add a couple more `.to_wasmtime_result()` invocations in fuzz code"

This reverts commit b33696e5f63f7eed731ca9dad10fec5e55a550d3.

* Move cranelift fuzz targets back to anyhow

* Fix cargo feature enablement for explorer

show more ...

clean up subtask in `starting` state in `subtask_cancel` (#12322)

https://github.com/bytecodealliance/wasmtime/pull/12304 uncovered a bug which
caused [`wit-bindgen`'s `cancel-import` test to
hang](

clean up subtask in `starting` state in `subtask_cancel` (#12322)

https://github.com/bytecodealliance/wasmtime/pull/12304 uncovered a bug which
caused [`wit-bindgen`'s `cancel-import` test to
hang](https://github.com/bytecodealliance/wit-bindgen/issues/1495).
Specifically, cancelling a subtask that was in a `starting` state turned it into
a zombie that never exited, meaning `TaskExit::block` would block forever
without making progress.

This commit properly disposes of the subtask's thread and sets its `exited`
field to `true`, which allows `subtask_drop` to delete it entirely.
Additionally, I've updated `test_cancel` to call `TaskExit::block` to ensure
this does not regress.

show more ...