Handle `uextend` in conditional trap lowerings on x64 (#11044)

* Handle uextend in conditional trap lowerings on x64

* Logging and rebuilding for test programs and debug tests

* Forward srcloc deb

Handle `uextend` in conditional trap lowerings on x64 (#11044)

* Handle uextend in conditional trap lowerings on x64

* Logging and rebuilding for test programs and debug tests

* Forward srcloc debug info to conditional trap instructions from conditional branches to unconditional traps

show more ...

Handle defined shared memories in dwarf processing (#8750)

This commit resolves an assert in the dwarf generating of core wasm
modules when the module has a defined linear memory which is flagged
`s

Handle defined shared memories in dwarf processing (#8750)

This commit resolves an assert in the dwarf generating of core wasm
modules when the module has a defined linear memory which is flagged
`shared`. This is represented slightly differently in the `VMContext`
than owned memories that aren't `shared`, and looks more like an
imported memory. With support in #8740 it's now much easier to support
this.

Closes #8652

show more ...

Add some support for imported memories to generated DWARF (#8740)

This is more-or-less a prerequisite for #8652 and extends the generated
dwarf with expressions to not only dereference owned memorie

Add some support for imported memories to generated DWARF (#8740)

This is more-or-less a prerequisite for #8652 and extends the generated
dwarf with expressions to not only dereference owned memories but
additionally imported memories which involve some extra address
calculations to be emitted in the dwarf.

show more ...

Remove some usage of UnsafeCell in the adapter (#7117)

* Fix compile warnings in preview1 adapter

This isn't part of the "main build" so it's not inheriting the "deny
warnings" flag passed in CI, s

Remove some usage of UnsafeCell in the adapter (#7117)

* Fix compile warnings in preview1 adapter

This isn't part of the "main build" so it's not inheriting the "deny
warnings" flag passed in CI, so fixup some warnings which have leaked
through over time.

* Remove some usage of `UnsafeCell` in the adapter

Move to abstractions such as `OnceCell` from the standard library as
well as `RefCell` to avoid some unsafety. This shouldn't have any
practical ramifications on the adapter, mostly just trying to help
over-time maintenance.

* Deny test-programs warnings in CI

show more ...

Wasi-http: support inbound requests (proxy world) (#7091)

* Move the incoming_handler impl into http_impl

* Remove the incoming handler -- we need to use it as a guest export

* Start adding a test

Wasi-http: support inbound requests (proxy world) (#7091)

* Move the incoming_handler impl into http_impl

* Remove the incoming handler -- we need to use it as a guest export

* Start adding a test-programs test for the server side of wasi-http

* Progress towards running a server test

* Implement incoming-request-method

* Validate outparam value

* Initial incoming handler test

* Implement more of the incoming api

* Finish the incoming api implementations

* Initial cut at `wasmtime serve`

* fix warning

* wasmtime-cli: invoke ServeCommand, and add enough stuff to the linker to run trivial test

* fix warnings

* fix warnings

* argument parsing: allow --addr to specify sockaddr

* rustfmt

* sync wit definitions between wasmtime-wasi and wasmtime-wasi-http

* cargo vet: add an import config and wildcard audit for wasmtime-wmemcheck

* cargo vet: audit signal-hook-registry

* Remove duplicate add_to_linker calls for preview2 interfaces

prtest:full

* Add a method to finish outgoing responses

Co-authored-by: Adam Foltzer <[email protected]>
Co-authored-by: Pat Hickey <[email protected]>

* Mark the result of the incoming_{request,response}_consume methods as own

* Explicit versions for http-body and http-body-util

* Explicit `serve` feature for the `wasmtime serve` command

* Move the spawn outside of the future returned by `ProxyHandler::call`

* Review feedback

---------

Co-authored-by: Trevor Elliott <[email protected]>
Co-authored-by: Adam Foltzer <[email protected]>

show more ...

Update in-tree wit-bindgen to 0.11.0 (#6947)

This only affects tests and the adapter itself, but not in any breaking
way. The tests for wasi-http are reorganized to be commands which is
also require

Update in-tree wit-bindgen to 0.11.0 (#6947)

This only affects tests and the adapter itself, but not in any breaking
way. The tests for wasi-http are reorganized to be commands which is
also required to not have any exports currently since wit-bindgen for
Rust guests doesn't support generating bindings in one crate and
exporting in another.

show more ...

Implement the `tcp` interface of wasi-sockets. (#6837)

* Implement the `tcp` interface of wasi-sockets.

Implement the `tcp`, `tcp-create-socket`, and `network` interfaces of
wasi-sockets.

* Minor

Implement the `tcp` interface of wasi-sockets. (#6837)

* Implement the `tcp` interface of wasi-sockets.

Implement the `tcp`, `tcp-create-socket`, and `network` interfaces of
wasi-sockets.

* Minor cleanups.

* Update to the latest upstream wasi-sockets.

* Address review feedback.

* Handle zero-length reads and writes, and other cleanups.

* Fix compilation on macOS.

* Fix compilation on Windows.

* Update all the copies of wasi-socket wit files.

* Sync up more wit files.

* Fix the errno code for non-blocking `connect` on Windows.

prtest:full

* Tolerate `NOTCONN` errors when cleaning up with `shutdown`.

* Simplify the polling mechanism.

This requires an updated tokio for `Interest::ERROR`.

* Downgrade to tokio 1.29.1 for now.

* Move `tcp_state` out of the `Arc`.

* `accept` doesn't need a write lock.

* Remove `tcp_state`'s `RwLock`.

show more ...

feat: align wasi-http with component linker (#6195)

* feat: align wasi-http with component linker

* feat(wasi-http): allow bidirectional stream

* feat(wasi-http): clean up children when dropping r

feat: align wasi-http with component linker (#6195)

* feat: align wasi-http with component linker

* feat(wasi-http): allow bidirectional stream

* feat(wasi-http): clean up children when dropping resource

* chore: update based on feedback

* chore: replace wasi http context references

* chore: fix logical issue with outgoing body stream

* chore: use thread-safe reference-counting pointer

* chore: cleanup resources using table

* fix(wasi-preview1-component-adapter): sync command extended wit

* fix(wasi-preview1-component-adapter): sync command extended wit

* fix(wasmtime-wasi): sync wit for http types

* chore: refactor using wasmtime-wasi crate

fix(wasi-http): misconfiguration in wasmtime linkers

chore: keep streams details

chore: fix wasi http tests

* chore: use pollable from wasmtime-wasi

* chore: update wasi http linker for module

* chore: update test programs for wasi http

* fix(wasi-http): ensure proper errors are surfaced

* chore: split wasi http tests into individual files

* chore: ensure protocol error is mapped correctly

* chore: disable temporarily wasi http in wasmtime cli

* chore: comment out wasi http in wasmtime cli

* chore(ci): ensure wit definitions in sync

* feat(wasi-http): generate async host binding

* chore: make wasi http tests async

* chore: update ci workflow based on suggestion

Co-authored-by: Pat Hickey <[email protected]>

* feat(wasmtime-wasi): update logging world to latest

* feat(wasmtime): update proxy world to latest

* feat(wasmtime-wasi): add back command extended world

* fix(wasi-http): sync wit definitions

* chore: update tests with latest wit definitions

* Update src/commands/run.rs

* Update src/commands/run.rs

* Update src/commands/run.rs

* Update src/commands/run.rs

* Update src/commands/run.rs

* Update src/commands/run.rs

* Update src/commands/run.rs

* Update src/commands/run.rs

* Update src/commands/run.rs

* chore: fix formatting

* Ignore flaky test

* chore: fix compilation error for riscv64 arch

* Avoid `cp -T` on macos

Adding prtest:full to ensure that we've seen a successful build before
queuing.

* Don't build the wasi-http test programs for the native target

* Debug the wit consistency check

* Update streams.wit in wasi-http

* Mark the component outbound_request_post test flaky

* Disable flaky wasi-http-tests on windows only

* Use diff instead of rm/cp/git diff

* Disable more tests on windows

---------

Co-authored-by: Eduardo Rodrigues <[email protected]>
Co-authored-by: Pat Hickey <[email protected]>
Co-authored-by: Trevor Elliott <[email protected]>

show more ...

Update adapter build (#6573)

* Update adapter build

* Rename the binary artifact to `wasi_snapshot_preview1.wasm` and update
build scripts to account for this.
* Update documentation to mention d

Update adapter build (#6573)

* Update adapter build

* Rename the binary artifact to `wasi_snapshot_preview1.wasm` and update
build scripts to account for this.
* Update documentation to mention difference between reactor/command
builds.

Closes #6569

* More renaming

show more ...

Re-enable wasi-http for recent wit bindgen changes. Renable tests. (#6495)

* Re-enable wasi-http

* Address comments.

Use Rust 1.69.0 instead of 1.70.0 (#6502)

* Use Rust 1.69.0 instead of 1.70.0

Try to fix some recent CI breakage

prtest:full

* Don't force stable for test-programs

Update Wasmtime for upcoming WIT changes (#6390)

* Update Wasmtime for upcoming WIT changes

This PR integrates bytecodealliance/wasm-tools#1027 into Wasmtime. The
main changes here are:

* WIT synt

Update Wasmtime for upcoming WIT changes (#6390)

* Update Wasmtime for upcoming WIT changes

This PR integrates bytecodealliance/wasm-tools#1027 into Wasmtime. The
main changes here are:

* WIT syntax is updated with WebAssembly/component-model#193
* Generated bindings in the `bindgen!` macro have been updated to
reflect the new structure of WIT.
* The accepted component model binary format has been updated to account
for changes.

This PR disables wasi-http tests and the on-by-default feature because
the WIT syntax has been updated but the submodule containing the WITs
has not been updated yet so there's no way to get that building
temporarily. Once that's updated then this can be reenabled.

* Update wasmtime-wasi crate with new WIT

* Add wit-bindgen override for the updated version

* Officially disable wasi-http tests/building

* Move test-reactor WIT into the main WIT files

Don't store duplicates with the rest of the WASI WIT files we have.

* Remove adapter's copy of WIT files

* Disable default features for wit-bindgen

* Plumb disabling wasi-http tests a bit more

* Fix reactor tests and adapter build

* Remove no-longer-needed feature

* Update adapter verification script

* Back out some wasi-http hacks

* Update vet and some dependency sources

* Move where wit-bindgen comes from

Make it a more "official" location which is also less likely to be
accidentally deleted in the future.

* Don't document wasi-http-tests

show more ...

add command and reactor tests

move the command and reactor tests into test-programs

Refactor test-programs to build modules and components (#6385)

* wasi-tests and wasi-http-tests no longer have their own workspace

* wasi-tests: fix warnings

* rewrite the test-programs build.rs t

Refactor test-programs to build modules and components (#6385)

* wasi-tests and wasi-http-tests no longer have their own workspace

* wasi-tests: fix warnings

* rewrite the test-programs build.rs to generate {package}_modules.rs and _components.rs

The style is cribbed from preview2-prototying repo, but I ended up
refactoring it a bit.

* better escaping should help with windows?

* long form cap-std-sync and tokio test suites

* convert wasi-http test

* fixes, comments

* apply cargo fmt to whole workspace

* bump test-programs and wasi-http-tests to all use common dependency versions

wit-bindgen 0.6.0 and wit-component 0.7.4

* add new audits

* cargo vet prune

* package and supply chain updates to fix vulnerabilities

h2 upgraded from 0.3.16 -> 0.3.19 to fix vulnerability
tempfile upgraded from 0.3.3 -> 0.3.5 to eliminate dep on vulnerable
remove_dir_all

* deny: temporarily allow duplicate wasm-encoder, wasmparser, wit-parser

prtest:full

* convert more dependencies to { workspace = true }

Alex asked me to do thsi for wit-component and wit-bindgen, and I found
a few more (cfg-if, tempfile, filecheck, anyhow...

I also reorganized the workspace dependencies section to make the ones
our team maintains more clearly separated from our external
dependencies.

* test-programs build: ensure that the user writes a #[test] for each module, component

* fix build of wasi-tests on windows

* misspelled macos

* mark wasi-tests crate test=false so we dont try building it natively...

* mark wasi-http-tests test=false as well

* try getting the cargo keys right

* just exclude wasi-tests and wasi-http-tests in run-tests.sh

* interesting paths fails on windows

* misspelling so nice i did it twice

* new cargo deny exception: ignore all of wit-bindgen's dependencies

* auto-import wildcard vets

show more ...

Begin implementation of wasi-http (#5929)

* Integrate experimental HTTP into wasmtime.

* Reset Cargo.lock

* Switch to bail!, plumb options partially.

* Implement timeouts.

* Remove generated fil

Begin implementation of wasi-http (#5929)

* Integrate experimental HTTP into wasmtime.

* Reset Cargo.lock

* Switch to bail!, plumb options partially.

* Implement timeouts.

* Remove generated files & wasm, add Makefile

* Remove generated code textfile

* Update crates/wasi-http/Cargo.toml

Co-authored-by: Eduardo de Moura Rodrigues <[email protected]>

* Update crates/wasi-http/Cargo.toml

Co-authored-by: Eduardo de Moura Rodrigues <[email protected]>

* Extract streams from request/response.

* Fix read for len < buffer length.

* Formatting.

* types impl: swap todos for traps

* streams_impl: idioms, and swap todos for traps

* component impl: idioms, swap all unwraps for traps, swap all todos for traps

* http impl: idiom

* Remove an unnecessary mut.

* Remove an unsupported function.

* Switch to the tokio runtime for the HTTP request.

* Add a rust example.

* Update to latest wit definition

* Remove example code.

* wip: start writing a http test...

* finish writing the outbound request example

havent executed it yet

* better debug output

* wasi-http: some stubs required for rust rewrite of the example

* add wasi_http tests to test-programs

* CI: run the http tests

* Fix some warnings.

* bump new deps to latest releases (#3)

* Add tests for wasi-http to test-programs (#2)

* wip: start writing a http test...

* finish writing the outbound request example

havent executed it yet

* better debug output

* wasi-http: some stubs required for rust rewrite of the example

* add wasi_http tests to test-programs

* CI: run the http tests

* bump new deps to latest releases

h2 0.3.16
http 0.2.9
mio 0.8.6
openssl 0.10.48
openssl-sys 0.9.83
tokio 1.26.0

---------

Co-authored-by: Brendan Burns <[email protected]>

* Update crates/test-programs/tests/http_tests/runtime/wasi_http_tests.rs

* Update crates/test-programs/tests/http_tests/runtime/wasi_http_tests.rs

* Update crates/test-programs/tests/http_tests/runtime/wasi_http_tests.rs

* wasi-http: fix cargo.toml file and publish script to work together (#4)

unfortunately, the publish script doesn't use a proper toml parser (in
order to not have any dependencies), so the whitespace has to be the
trivial expected case.

then, add wasi-http to the list of crates to publish.

* Update crates/test-programs/build.rs

* Switch to rustls

* Cleanups.

* Merge switch to rustls.

* Formatting

* Remove libssl install

* Fix tests.

* Rename wasi-http -> wasmtime-wasi-http

* prtest:full

Conditionalize TLS on riscv64gc.

* prtest:full

Fix formatting, also disable tls on s390x

* prtest:full

Add a path parameter to wit-bindgen, remove symlink.

* prtest:full

Fix tests for places where SSL isn't supported.

* Update crates/wasi-http/Cargo.toml

---------

Co-authored-by: Eduardo de Moura Rodrigues <[email protected]>
Co-authored-by: Pat Hickey <[email protected]>
Co-authored-by: Pat Hickey <[email protected]>

show more ...

Remove dependency on `more-asserts` (#4408)

* Remove dependency on `more-asserts`

In my recent adventures to do a bit of gardening on our dependencies I
noticed that there's a new major version

Remove dependency on `more-asserts` (#4408)

* Remove dependency on `more-asserts`

In my recent adventures to do a bit of gardening on our dependencies I
noticed that there's a new major version for the `more-asserts` crate.
Instead of updating to this though I've opted to instead remove the
dependency since I don't think we heavily lean on this crate and
otherwise one-off prints are probably sufficient to avoid the need for
pulling in a whole crate for this.

* Remove exemption for `more-asserts`

show more ...

test-programs: test wasi-tokio

cap-primitives bump fixes windows bugs

new cap-std apis fix some windows tests!

fix for fd_readdir test on linux, and symlink_create / nofollow_errors

remove virtfs - it is not suitable for use

upstream fixes are pending for symlink_create & nofollow_errors on windows

this windows test should be fixed...

path_rename should work on windows with TESTCONFIG

doh