Run s390x CI on a native machine (#11711)

The hardware for this is provided by IBM and it's definitely faster than
QEMU. We've still got slow jobs elsewhere but removing this as a blocker
is still n

Run s390x CI on a native machine (#11711)

The hardware for this is provided by IBM and it's definitely faster than
QEMU. We've still got slow jobs elsewhere but removing this as a blocker
is still nice for CI times!

prtest:linux-s390x

show more ...

Enable clippy's `redundant_field_names` lint (#10985)

Seems like a clear enough improvement and also something useful to have
a lint for as refactorings enable/disable this pattern over time, so
thi

Enable clippy's `redundant_field_names` lint (#10985)

Seems like a clear enough improvement and also something useful to have
a lint for as refactorings enable/disable this pattern over time, so
this should help us stay consistent.

show more ...

wasi-sockets: Fix `shutdown` bugs (#9225)

* Rename:
LastWrite -> WriteState,
Done -> Ready,
Waiting -> Writing

* Wrap the TCP streams inside a mutex.
And rename `abort_wait` to `cancel`

* Move shu

wasi-sockets: Fix `shutdown` bugs (#9225)

* Rename:
LastWrite -> WriteState,
Done -> Ready,
Waiting -> Writing

* Wrap the TCP streams inside a mutex.
And rename `abort_wait` to `cancel`

* Move shutdown into the stream types

* Fix Linux-specific peculiarity where `recv` continues to work even after `shutdown(sock, SHUT_RD)` has been called.

* Refactor poll_cancel

* Fix data loss issue.
Calling `shutdown` _after_ data has been accepted by `write`, but _before_ it has been fully flushed, caused the in-flight data to be lost.

* Simplify poll_cancel

* Use tokio::sync::Mutex instead of std::sync::Mutex so I can revert to regular async code

show more ...

wasi-sockets: Add SO_REUSEADDR back in (#7690)

* Restore SO_REUSEADDR.

This inadvertently removed in 8ca80569452908d678066fb34a11d1ceda78e0c7 when switching from `bind_existing_tcp_listener` to `ru

wasi-sockets: Add SO_REUSEADDR back in (#7690)

* Restore SO_REUSEADDR.

This inadvertently removed in 8ca80569452908d678066fb34a11d1ceda78e0c7 when switching from `bind_existing_tcp_listener` to `rustix::net::bind`

* Remove AddressInUse workarounds by generating a random port.

* fmt

* Ignore unused_variables warning

* Prevent spurious test failures by trying again a few times on EADDRINUSE

* Fix grammar in .wit documentation

show more ...

Add a test for allow-ip-name-lookup=no (#7657)

The test simply tries looking up example.com and ensuring it fails with
`PermanentResolverFailure`. Additionally, `PermanentResolverFailure` is
removed

Add a test for allow-ip-name-lookup=no (#7657)

The test simply tries looking up example.com and ensuring it fails with
`PermanentResolverFailure`. Additionally, `PermanentResolverFailure` is
removed as one of the allowed errors in the normal ip name lookup test
as those allowed errors should only be transient lookup errors and
`PermanentResolverFailure` is very much not a transient error.

Signed-off-by: Ryan Levick <[email protected]>

show more ...

wasi-sockets: integration test & internal documentation updates (#7570)

* Implement timeout on all blocking utility methods.

* Expand documentation of compatibility fixes

wasi-sockets: Simplify ip name lookup interface (#7483)

* Refactor ip_name_lookup test

* Update ip-name-lookup::resolve-addresses

- Remove the non-essential parameters
- Lift the restriction again

wasi-sockets: Simplify ip name lookup interface (#7483)

* Refactor ip_name_lookup test

* Update ip-name-lookup::resolve-addresses

- Remove the non-essential parameters
- Lift the restriction against parsing IP addresses. Implementations would still have to parse IP addresses to decide whether or not to return an error

* Deduplicate to_canonical

show more ...

wasi-sockets: Sync up UDP implementation & tests with TCP (#7423)

* Refactor & move utilities. So that UDP can use them too.

* Minor tweak in error code while connecting

* Refactor: move portabili

wasi-sockets: Sync up UDP implementation & tests with TCP (#7423)

* Refactor & move utilities. So that UDP can use them too.

* Minor tweak in error code while connecting

* Refactor: move portability workarounds out of the TCP implementation.

* Sync up UDP implementation and tests
based on the existing TCP implementation & tests.

* Make test less flaky in CI.

show more ...

Rename poll-list to poll, poll-one to pollable.block, and introduce pollable.ready (#7427)

* update wasi:io/poll wit to https://github.com/WebAssembly/wasi-io/pull/54

* put version in wit package n

Rename poll-list to poll, poll-one to pollable.block, and introduce pollable.ready (#7427)

* update wasi:io/poll wit to https://github.com/WebAssembly/wasi-io/pull/54

* put version in wit package name

* implement changes to the wits

* move contents of Host::poll_one to HostPollable::block
* rename Host::poll_list to Host::poll,
* implement HostPollable::ready, using futures::future::poll_immediate

* wit: fix reference to poll-list

* wasi-http wit: fix reference to poll-list

* clocks implementation: ready returns immediately if deadline has past

this is an optimization, but what it really allows us to do is assert
pollable.ready() for a subscribe_duration(0) is ready immediately.

* component adapter: rename poll-list to poll

* test-programs: renames to poll functions

test-programs/src/bin/preview2_sleep.rs in particular now asserts
ready() on a subscribe_duration(0) and a subscribe_instant(now() - 1),
so we have test coverage for ready as well now

* code review

Co-authored-by: Alex Crichton <[email protected]>

---------

Co-authored-by: Alex Crichton <[email protected]>

show more ...

monotonic clock: introduce `duration` type, split `subscribe` (#7358)

* monotonic clock: introduce `duration` type, split `subscribe`

We are introducing a `duration` type because it has a distinct

monotonic clock: introduce `duration` type, split `subscribe` (#7358)

* monotonic clock: introduce `duration` type, split `subscribe`

We are introducing a `duration` type because it has a distinct meaning
from `instant`: an `instant` can only be compared to other `instant`s
from the exact same `monotonic-clock`, whereas a `duration` represents
a duration of time which can be compared to any other duration of time.

The `duration` type is motivated, in part, by a desire to reuse it to
specify durations such as timeouts in other WASI proposals.

Instead of taking a boolean specifying whether the u64 is an absolute or
relative time, `subscribe-instant` takes an `instant` type and
`subscribe-duration` takes a `duration` type.

* wasmtime-wasi: implement changes to monotonic clock interface

* adapter: fixes for subscribe duration and instant

* factor subscribe_to_duration out into a helper function.

* sync wits into wasi-http

* component adapter: fix for interpreting flag for monotonic instant vs duration

* subscribe_instant impl: fix logic for when time is in the past

caught by subtract overflow panic, thankfully

* test-programs: compile against new subscribe instant, duration methods

show more ...

wasi-sockets: Introduce UDP streams (#7243)

* Introduce UDP streams

Introduce new `inbound-datagram-stream` and `outbound-datagram-stream` types and moved `receive` and `send` methods to those resp

wasi-sockets: Introduce UDP streams (#7243)

* Introduce UDP streams

Introduce new `inbound-datagram-stream` and `outbound-datagram-stream` types and moved `receive` and `send` methods to those respectively. These streams are returned by `bind` can be individually subscribed to. This resolves a design issue where a UDP server would end up in a spin loop because `receive` returned EWOULDBLOCK but poll_* always returned immediately because the socket was ready for sending. In this new setup, users can poll each direction separately. Fixes https://github.com/WebAssembly/wasi-sockets/issues/64

Additionally:
- Enable send-like behaviour by making `outbound-datagram::remote-address` optional. Fixes https://github.com/WebAssembly/wasi-sockets/pull/57
- Dropped the `network` parameter from the `connect` call, because `bind` is now _required_ to perform IO.

* Align names with wasi-http

* Revert previous changes to `bind`. Replace `connect` with `stream`

Remove the Mutex again. Instead allow `stream` to be called multiple times, but trap if the previous streams are still active.

* The code block was treated as Rust code.

* Align more closely to wasi-io's input&output-stream

* Use `send` instead of `sendto` on connected sockets.

prtest:full

show more ...

Refactor the test-programs test suite (#7182)

* Refactor the test-programs test suite

This commit is a large refactoring that reorganizes `test-programs` and
how we tests wasms in Wasmtime. Often w

Refactor the test-programs test suite (#7182)

* Refactor the test-programs test suite

This commit is a large refactoring that reorganizes `test-programs` and
how we tests wasms in Wasmtime. Often writing tests requires complicated
interactions with the guest which can't be done via hand-written `*.wat`
syntax and requires a compiler to get engaged. For this purpose Wasmtime
currently has the `crates/test-programs/*` test suite which builds files
from source and then runs the tests. This has been somewhat cumbersome
in the past though and it's not been easy to extend this over time, so
this commit attempts to address this.

The scheme implemented in this PR looks like:

* All wasm test programs live in `crates/test-programs/src/bin/*.rs`.
All of them, no exceptions.

* Wasm tests have shared support located at
`crates/test-programs/src/lib.rs` and its submodules, such as bindings
generation for WASI.

* Wasm tests are built by a new `crates/test-programs/artifacts` crate.
This crate compiles modules and additionally creates components for
all test programs. The crate itself only records the path to these
outputs and a small amount of testing support, but otherwise doesn't
interact with `wasmtime`-the-crate itself.

* All tests in `crates/test-programs/tests/*.rs` have moved. For example
wasi-http tests now live at `crates/wasi-http/tests/*.rs`. Legacy
tests of wasi-common now live at `crates/wasi-common/tests/*.rs`.
Modern tests for preview2 live at `crates/wasi/tests/*.rs`.

* Wasm tests are bucketed based on their filename prefix. For example
`preview1_*` is tested in wasi-common and wasmtime-wasi. The
`preview2_*` prefix is only tested with wasmtime-wasi, however.

* A new `cli_*` prefix is used to execute tests as part of
`tests/all/main.rs`. This is a new submodule in
`tests/all/cli_tests.rs` which executes these components on the
command line. Many old "command" tests were migrated here.

* Helper macros are generated to assert that a test suite is run in its
entirety. This way if a `preview1_*` test is added it's asserted to
get added to both wasi-common and wasmtime-wasi in the various modes
they run tests.

Overall this moved a number of tests around and refactored some edges of
the tests, but this should not lose any tests (except one that wasn't
actually testing anything). Additionally the hope is that it's much
easier to add tests in the future. The process is to add a new file in
`crates/test-programs/src/bin/*.rs` named appropriately. For example a
preview2 executable is `preview2_*` and a CLI tests is `cli_*`. When
building the test suite an error is generated in the appropriate module
then of "please write a test here", and then a test is written in the
same manner as the other tests in the module.

* Remove no-longer-needed fetches

prtest:full

* I'm worried wasi is running low on semicolons

* Add the WASI target in all CI actions

* Add unknown-unknown target on all CI builders too

* Fix building test artifacts under miri

Need to avoid wrappers for these cross-compiled targets

* Break circular dependency for packaging

Don't use the workspace dep for `wasmtime-wasi` since it injects a
version, instead use a `path = '..'` dependency to fool Cargo into
dropping the dependency during the package phase.

* Fix some merge conflicts with tests

* Fix rebase for new tests

* Remove stray comment

* Fix some flaky tests

* Fix network tests in synchronous mode

This commit is an attempt to fix some networking tests in synchronous
mode in our test suite. Currently networking tests don't actually run in
synchronous mode on CI which is why no failures have been surfaced yet,
but the refactoring in #7182 is going to start doing this.

Currently the `udp_sample_application.rs` test blocks infinitely in
synchronous mode for me locally, most of the time. This appears to be an
interaction between how Tokio handles readiness and how we're
entering the event loop. We're effectively entering the Tokio event loop
with a future that's always ready which ends up starving Tokio of
otherwise performing its background work such as updating flags for
readiness of reading/writing.

The fix here is to add a yield at the start of an `in_tokio` block which
is used in synchronous mode. This is a kludge fix but the intention is
to enable Tokio to have a chance to update readiness flags and process
events from epoll/kqueue/etc.

An additional fix to this issue is WebAssembly/wasi-sockets#64 where the
test is waiting on `READABLE` or `WRITABLE`, but in this specific case
it should only wait on `READABLE`. If it waited on just this then that
would also fix this issue. Nevertheless having a `yield_now` is expected
to have little-to-no overhead and otherwise fix this edge case of an
always-ready future.

* Fix passing empty arguments on the CLI

* Add another blocking accept

* Update crates/test-programs/src/bin/api_proxy.rs

Co-authored-by: Trevor Elliott <[email protected]>

---------

Co-authored-by: Trevor Elliott <[email protected]>

show more ...