make async tests using `ready` interface more robust (#12360)

This changes the `ready` interface used by `component-async-tests` from:

```
interface ready {
// Set the `ready` state
set-ready:

make async tests using `ready` interface more robust (#12360)

This changes the `ready` interface used by `component-async-tests` from:

```
interface ready {
// Set the `ready` state
set-ready: func(ready: bool);
// Block until `ready` is `true`
when-ready: async func();
}
```
to:
```
interface ready {
resource thing {
constructor();
set-ready: func(ready: bool);
when-ready: async func();
}
}
```

The problem with the original version was that it required global state and thus
caused cross-talk across concurrent tasks. Due to implementation details inside
Wasmtime, the tests worked anyway, but
https://github.com/bytecodealliance/wasmtime/pull/12357 perturbed that and
revealed how fragile tests based on that interface were.

The new version puts the state inside a resource type, allowing each task create
its own instance of that resource type and thereby avoid crosstalk.

show more ...

remove `callback_code::POLL` and make `waitable-set.poll` not yield (#12182)

As of https://github.com/WebAssembly/component-model/pull/578, there is no more
`POLL` callback code, and `waitable-set.p

remove `callback_code::POLL` and make `waitable-set.poll` not yield (#12182)

As of https://github.com/WebAssembly/component-model/pull/578, there is no more
`POLL` callback code, and `waitable-set.poll` should not yield. Guest
toolchains can emulate the old code with a combination of `YIELD` and
`waitable-set.poll`.

Note that I've removed the now-redundant copy of `trap-if-block-and-sync.wast`
in favor of the upstream version.

show more ...

Update wasm-tools dependencies (#12031)

* Change separator style

* Update wasm-tools dependencies

* Also update wit-bindgen
* Drop `[async]` name prefixes
* Plumb `async` as part of a function ty

Update wasm-tools dependencies (#12031)

* Change separator style

* Update wasm-tools dependencies

* Also update wit-bindgen
* Drop `[async]` name prefixes
* Plumb `async` as part of a function type

Runtime handling of async functions and new traps are to be implemented
in subsequent commits. This is just getting everything running again.

* Run clang-format, also add test

* Fix some more wast tests

---------

Co-authored-by: Sy Brand <[email protected]>

show more ...

Revamp component model stream/future host API (again) (#11515)

* Revamp component model stream/future host API (again)

This changes the host APIs for dealing with futures and streams from a
"rendez

Revamp component model stream/future host API (again) (#11515)

* Revamp component model stream/future host API (again)

This changes the host APIs for dealing with futures and streams from a
"rendezvous"-style API to a callback-oriented one.

Previously you would create e.g. a `StreamReader`/`StreamWriter` pair and call
their `read` and `write` methods, respectively, and those methods would return
`Future`s that resolved when the operation was matched with a corresponding
`write` or `read` operation on the other end.

With the new API, you instead provide a `StreamProducer` trait implementation
whe creating the stream, whose `produce` method will be called as soon as a read
happens, giving the implementation a chance to respond immediately without
making the reader wait for a rendezvous. Likewise, you can match the read end
of a stream to a `StreamConsumer` to respond immediately to writes. This model
should reduce scheduling overhead and make it easier to e.g. pipe items to/from
`AsyncWrite`/`AsyncRead` or `Sink`/`Stream` implementations without needing to
explicitly spawn background tasks. In addition, the new API provides direct
access to guest read and write buffers for `stream<u8>` operations, enabling
zero-copy operations.

Other changes:

- I've removed the `HostTaskOutput`; we were using it to run extra code with
access to the store after a host task completes, but we can do that more
elegantly inside the future using `tls::get`. This also allowed me to
simplify `Instance::poll_until` a bit.

- I've removed the `watch_{reader,writer}` functionality; it's not needed now
given that the runtime will automatically dispose of the producer or consumer
when the other end of the stream or future is closed -- no need for embedder
code to manage that.

- In order to make `UntypedWriteBuffer` `Send`, I had to wrap its raw pointer
`buf` field in a `SendSyncPtr`.

- I've removed `{Future,Stream}Writer` entirely and moved
`Instance::{future,stream}` to `{Future,Stream}Reader::new`, respectively.

- I've added a bounds check to the beginnings of `Instance::guest_read` and
`Instance::guest_write` so that we need not do it later in
`Guest{Source,Destination}::remaining`, meaning those functions can be
infallible.

Note that I haven't updated `wasmtime-wasi` yet to match; that will happen in
one or more follow-up commits.

Signed-off-by: Joel Dice <[email protected]>

* Add `Accessor::getter`, rename `with_data` to `with_getter`

* fixup bindgen invocation

Signed-off-by: Roman Volosatovs <[email protected]>

* add support for zero-length writes/reads to/from host

I've added a test to cover this; it also tests direct buffer access for
`stream<u8>`, which I realized I forgot to cover earlier. And of course there
was a bug :facepalm:.

Signed-off-by: Joel Dice <[email protected]>

* add `{Destination,Source}::remaining` methods

This can help `Stream{Producer,Consumer}` implementations determine how many
items to write or read, respectively.

Signed-off-by: Joel Dice <[email protected]>

* wasi: migrate sockets to new API

Signed-off-by: Roman Volosatovs <[email protected]>

* tests: read the socket stream until EOF

Signed-off-by: Roman Volosatovs <[email protected]>

* p3-sockets: account for cancellation

Signed-off-by: Roman Volosatovs <[email protected]>

* p3-sockets: mostly ensure byte buffer cancellation-safety

Signed-off-by: Roman Volosatovs <[email protected]>

* p3-filesystem: switch to new API

Signed-off-by: Roman Volosatovs <[email protected]>

* fixup! p3-sockets: mostly ensure byte buffer cancellation-safety

* p3-cli: switch to new API

Signed-off-by: Roman Volosatovs <[email protected]>

* p3: limit maximum buffer size

Signed-off-by: Roman Volosatovs <[email protected]>

* p3-sockets: remove reuseaddr test loop workaround

Signed-off-by: Roman Volosatovs <[email protected]>

* p3: drive I/O in `when_ready`

Signed-off-by: Roman Volosatovs <[email protected]>

* fixup! p3: drive I/O in `when_ready`

* Refine `Stream{Producer,Consumer}` APIs

Per conversations last week with Roman, Alex, and Lann, I've updated these
traits to present a lower-level API based on `poll_{consume,produce}` functions
and have documented the implementation requirements for various scenarios which
have come up in `wasmtime-wasi`, particularly around graceful cancellation. See
the doc comments for those functions for details.

Signed-off-by: Joel Dice <[email protected]>

* being integration of new API

Signed-off-by: Roman Volosatovs <[email protected]>

* update wasi/src/p3/filesystem to use new stream API

This is totally untested so far; I'll run the tests once we have everything else
compiling.

Signed-off-by: Joel Dice <[email protected]>

* update wasi/src/p3/cli to use new stream API

This is totally untested and doesn't even compile yet due to a lifetime issue I
don't have time to address yet. I'll follow up later with a fix.

Signed-off-by: Joel Dice <[email protected]>

* fix: remove `'a` bound on `&self`

Signed-off-by: Roman Volosatovs <[email protected]>

* finish `wasi:sockets` adaptation

Signed-off-by: Roman Volosatovs <[email protected]>

* finish `wasi:cli` adaptation

Note, that this removes the read optimization - let's get the
implementation complete first and optimize later

Signed-off-by: Roman Volosatovs <[email protected]>

* remove redundant loop in sockets

Signed-off-by: Roman Volosatovs <[email protected]>

* wasi: buffer on 0-length reads

Signed-off-by: Roman Volosatovs <[email protected]>

* finish `wasi:filesystem` adaptation

Signed-off-by: Roman Volosatovs <[email protected]>

* remove `MAX_BUFFER_CAPACITY`

Signed-off-by: Roman Volosatovs <[email protected]>

* refactor `Cursor` usage

Signed-off-by: Roman Volosatovs <[email protected]>

* impl Default for VecBuffer

Signed-off-by: Roman Volosatovs <[email protected]>

* refactor: use consistent import styling

Signed-off-by: Roman Volosatovs <[email protected]>

* feature-gate fs Arc accessors

Signed-off-by: Roman Volosatovs <[email protected]>

* Update test expectations

---------

Signed-off-by: Joel Dice <[email protected]>
Signed-off-by: Roman Volosatovs <[email protected]>
Co-authored-by: Alex Crichton <[email protected]>
Co-authored-by: Roman Volosatovs <[email protected]>

show more ...

add Component Model async ABI tests (#11136)

* add Component Model async ABI tests

This pulls in the tests from the `wasip3-prototyping` repo, minus the ones
requiring WASIp3 support in `wasmtime-w

add Component Model async ABI tests (#11136)

* add Component Model async ABI tests

This pulls in the tests from the `wasip3-prototyping` repo, minus the ones
requiring WASIp3 support in `wasmtime-wasi[-http]`, which will be PR'd
separately.

* add audits and exemptions for new `component-async-tests` deps

In order to convince `cargo vet` that we only needed these deps to be
`safe-to-run` (not necessarily `safe-to-deploy`, since it's test code), I've
moved the `wasm-compose` dep to the `dev-dependencies` section of the
`Cargo.toml` file, which required rearranging some code.

I've exempted `wasm-compose` since it's a BA project, and also exempted all but
one of the remaining new deps since they each get well over 10,000 downloads per
day from crates.io. I've audited and certified the remaining dep, `im-rc`,
which came in a bit shy of the 10,000-per-day mark.

Signed-off-by: Joel Dice <[email protected]>

* simplify `component_async_tests::util::sleep`

Signed-off-by: Joel Dice <[email protected]>

---------

Signed-off-by: Joel Dice <[email protected]>

show more ...