feat(p3): implement wasi:tls (#12834)* Split off p2-specific bits into submodule* Vendor the 0.3.0-draft WIT files* Host traits scaffolding* Rename p2 test* Work around bug in `tokio-native
feat(p3): implement wasi:tls (#12834)* Split off p2-specific bits into submodule* Vendor the 0.3.0-draft WIT files* Host traits scaffolding* Rename p2 test* Work around bug in `tokio-native-tls`* Create error type* Implement p3Co-authored-by: Roman Volosatovs <[email protected]>* Fix test on Windows' SChannelSame reason as described in https://github.com/bytecodealliance/wasmtime/pull/11064* Satisfy clippy* Fix typo---------Co-authored-by: Roman Volosatovs <[email protected]>
show more ...
Debugging: add integration test with LLDB and some minor tweaks. (#12856)This PR adds:- An integration-test that runs LLDB against the Wasmtime CLI to verify basic debugging functionality, simi
Debugging: add integration test with LLDB and some minor tweaks. (#12856)This PR adds:- An integration-test that runs LLDB against the Wasmtime CLI to verify basic debugging functionality, similar to the existing native-debug tests.- A CI job that runs the above in CI.- Some minor tweaks to the gdbstub debugger design: - Rather than the initial single-step to get to the first Wasm instruction where module(s) will be instantiated into the store and visible to the debugger, we pre-register modules with the store eagerly. This avoids the slightly hacky flow and also is a preparation step for `wasmtime serve` debugging, where we can't single-step into execution eagerly (because execution doesn't start at all until an HTTP request arrives). - Add a separate message-printing path for "debugger info messages", allowing us to print the "debugger is listening on <PORT>" message without inheriting stderr for the whole debugger component environment. This message is necessary for the above integration test (it parses the message to determine when the debuggee is ready).
Debugging: add the debug-main world. (#12756)* Debugging: add the debug-main world.This PR "draws the rest of the owl" for the debug-mainworld (bytecodealliance/rfcs#45). This includes a WIT wor
Debugging: add the debug-main world. (#12756)* Debugging: add the debug-main world.This PR "draws the rest of the owl" for the debug-mainworld (bytecodealliance/rfcs#45). This includes a WIT world that hostsdebug components that have access to "host debug powers" via adebugging API, and the ability to load such a debug-component and giveit control of the main program as a debuggee when using `wasmtimerun`.The WIT is namespaced to `bytecodealliance:wasmtime` and is slightlyaspirational in places: for example, the host does not yet implementinjection of early return values or exception-throws. I intend to fillout a series of TODO issues once this all lands to track followup("post-MVP") work.This PR does not include any debug components. I separately have agdbstub component, with which I tested and co-developed this host-sideimplementation. My plan is to land it in a followup PR as a componentthat will be embedded in/shipped with the Wasmtime CLI and availableunder an easy-to-use CLI option. Once we have that gdbstub component,we can also implement end-to-end integration tests that boot up LLDBand run through an expected interaction. (Separately, thoseintegration tests will require a release of wasi-sdk to ship an LLDBbinary that we can use.) As such, there are no real tests in this PR:interesting behaviors only really occur with a full end-to-end flow.The integration with the CLI is a little awkward (we internally buildanother `wasmtime run` command that invokes the debug component, andtie it together with the debuggee via a special `invoke_debugger` API;this seemed less bad than reworking all of the WASI setup to be morereusable). Happy to take more ideas here.* Review feedback.* Review feedback.* Review feedback: update vendor-wit.sh.* Review feedback: -Ddebugger-arg= -> -Darg=.* Review feedback.* Review feedback.* Review feedback: factor host.rs into several submodules.* Review feedback: rename Debugger to Debuggee on host side.* Review feedback: split inherit_stdin_stdout, and add corresponding options for the debug component.* Review feedback.* Review feedback.* Add simple debug-component tests.* Add wasm32-wasip2 target in a few places in CI* Cargo vets for wstd dependency.* Add wasm32-wasip2 in more places* fix debug-component test dependence on componentization byte offsets* Review feedback.* Fix cancel-safety of EventFuture.* Fix: Interrupted events should only occur after interrupt(), not on every epoch yield.* Review feedback.* Review feedback: strip down WASI imports in debugger world.* fold debugger test component back into wasip1 + adapter test artifact compilation flow
Fix two security advisories. (#12652)* Fix two security advisories.This commit contains merged fixes for two security advisories inWasmtime:* GHSA-852m-cvvp-9p4w* GHSA-243v-98vx-264hThis in
Fix two security advisories. (#12652)* Fix two security advisories.This commit contains merged fixes for two security advisories inWasmtime:* GHSA-852m-cvvp-9p4w* GHSA-243v-98vx-264hThis introduces new knobs to Wasmtime to limit the scope of resourcesthat WASI implementations will allocate on behalf of guests. Unlikebackports to 41.0.x-and-prior these knobs all have default values whichare considered reasonable for hosts if they don't further tune them. Thefollowing CLI knobs have been added:* `-Smax-resources` - limits the total component-model resources a guest can allocate in a table* `-Shostcall-fuel` - a broad limit which enforces that at most this amount of data will be copied from the guest to the host in any one API call (e.g. `string` values can't be too big, `list<string>` can't be quadratic, etc). This fuel is reset on each host function call.* `-Smax-random-size` - the maximal size of the return value of the `get-random-bytes` and `get-insecure-random-bytes` WASI functions.* `-Smax-http-fields-size` - a limit on the size of `wasi:http` `fields` values to avoid infinitely buffering data within the host.The `http` crate has additionally been updated to avoid a panic whenadding too many headers to a `fields` object.Co-authored-by: Mark Bundschuh <[email protected]>Co-authored-by: Pat Hickey <[email protected]>Co-authored-by: Joel Dice <[email protected]>* CI fixes* Run rustfmt* Fix wasi-common build* Fix tests on 32-bit* Fix nightly test expectationsprtest:full---------Co-authored-by: Mark Bundschuh <[email protected]>Co-authored-by: Pat Hickey <[email protected]>Co-authored-by: Joel Dice <[email protected]>
Fix component-async fuzzer on OSS-Fuzz (#12400)On OSS-Fuzz the fuzzers are built in one place and run in another, so`include_bytes!` is needed to get the wasm into the final binary. Thisis done w
Fix component-async fuzzer on OSS-Fuzz (#12400)On OSS-Fuzz the fuzzers are built in one place and run in another, so`include_bytes!` is needed to get the wasm into the final binary. Thisis done with a layer of macros to avoid an `include_bytes!` for all wasmprograms which would make the generated Rust metadata a bit... large.
Add a new fuzzer focused on component-model-async events (#12119)* Add a new fuzzer focused on component-model-async eventsThis commit adds a new fuzzer mode to the `misc` fuzzer of Wasmtimewhic
Add a new fuzzer focused on component-model-async events (#12119)* Add a new fuzzer focused on component-model-async eventsThis commit adds a new fuzzer mode to the `misc` fuzzer of Wasmtimewhich is focused on async events and interleavings of components usingthe component-model-async proposal. This fuzzer works by having aprecompiled guest program which serves as the component to run. Thisprecompiled component has a custom `fuzz.wit` which is used to interfacewith the fuzzer itself. The fuzzer is then a fuzz-generated sequence ofcommands to send to the component which verifies that everythingexecutes correctly, has no panics, etc.This fuzzer intends to stress async communication and taskinfrastructure with component-model-async. Notably this does not stresslifting/lowering or arbitrary type signatures. This does, however,permute all of the following:* Guest/host interactions (also guest/guest, host/host, etc).* Async functions, both ready and pending.* Future operations: reads, writes, cancellation, transfers, etc.* Stream operations: reads, writes, cancellation, transfers, etc.This is all throwing into a large "soup" and then asserted to workcorrectly. There's a few gotchas here and there for how this fuzzer isdesigned, such as some events requiring "yield N times to await thisevent happening". This is required because Wasmtime is allowed tonon-deterministically select between a number of "ready events" and whatto dispatch.This is not intended to be a one-size-fits-all fuzzer forcomponent-model-async. The recent enhancements to the `component_api`fuzzer are intended to complement this fuzzer in terms of what'sstressed where internally.* Review comments
Use the same `cc` extension for C++ files (#11881)* Use the same `cc` extension for C++ filesRename a few stragglers using `*.cpp` to `*.cc` to have consistentsyntax throughout the repository.
Use the same `cc` extension for C++ files (#11881)* Use the same `cc` extension for C++ filesRename a few stragglers using `*.cpp` to `*.cc` to have consistentsyntax throughout the repository.* Fix another reference* Rename more extensions
Rename test programs (#11828)* exclude DS_Store files from git diffs* "preview" -> "p" prefix in test programs* fix test building* prefix http tests as p2_http* rename more tests* rename
Rename test programs (#11828)* exclude DS_Store files from git diffs* "preview" -> "p" prefix in test programs* fix test building* prefix http tests as p2_http* rename more tests* rename another file* get more tests to pass* fix build.rs* finish renaming tests* undo DS_Store addition* fix remaining naming issues* debug print on failing test* debug again* again* remove debug annotations* prefix cli_serve_sleep* final touches
Update and refactor how wasi-testsuite is run (#11560)* Update and refactor how wasi-testsuite is runThis commit updates the `WebAssembly/wasi-testuite` submodule in thisrepository which hasn't
Update and refactor how wasi-testsuite is run (#11560)* Update and refactor how wasi-testsuite is runThis commit updates the `WebAssembly/wasi-testuite` submodule in thisrepository which hasn't been updated once in the past 2 years. This thenadditionally refactors how tests are run:* A new top-level test is added at `tests/wasi.rs` similar to `tests/wast.rs`.* The top-level test uses `libtest_mimic` to enable running tests in parallel.* Using `libtest_mimic` enables running a test-at-a-time.* Using `libtest_mimic` enables seeing a list of failures instead of just the first failure.* Component versions of tests are run in addition to core wasm versions of tests.* Minor updates are made to the adapter and wasmtime-wasi to get some tests passing.* The list of allowed failures is updated with a `FIXME`-per-test of what's remaining.prtest:full* Fix in-tree tests with new behavior* Add Windows exemptions* Fix tests with wasi-common
feat: begin wasip3 implementation (#11221)* ci: add subdir supportSigned-off-by: Roman Volosatovs <[email protected]>* feat: extract common `WasiCtxBuilder`Signed-off-by: Roman Volosatov
feat: begin wasip3 implementation (#11221)* ci: add subdir supportSigned-off-by: Roman Volosatovs <[email protected]>* feat: extract common `WasiCtxBuilder`Signed-off-by: Roman Volosatovs <[email protected]>* chore: vendor p3 WITSigned-off-by: Roman Volosatovs <[email protected]>* feat: begin wasip3 implementationSigned-off-by: Roman Volosatovs <[email protected]>* chore(wasip3): remove now-redundant async stubsSigned-off-by: Roman Volosatovs <[email protected]>* test(wasip3): link wasip2Signed-off-by: Roman Volosatovs <[email protected]>* refactor: `allow` -> `expect`Signed-off-by: Roman Volosatovs <[email protected]>* chore: add bindgen `tracing` issue refref https://github.com/bytecodealliance/wasmtime/issues/11245Signed-off-by: Roman Volosatovs <[email protected]>* chore: adapt to `Accessor` API changes- hide `CommandPre`, since it is currently unusable https://github.com/bytecodealliance/wasmtime/issues/11249- use `Command::new` directly in examples, since `instantiate_async` does not provide a way to call an exportSigned-off-by: Roman Volosatovs <[email protected]>* doc: add a link to p3 `add_to_linker`Signed-off-by: Roman Volosatovs <[email protected]>---------Signed-off-by: Roman Volosatovs <[email protected]>
add Component Model async ABI tests (#11136)* add Component Model async ABI testsThis pulls in the tests from the `wasip3-prototyping` repo, minus the onesrequiring WASIp3 support in `wasmtime-w
add Component Model async ABI tests (#11136)* add Component Model async ABI testsThis pulls in the tests from the `wasip3-prototyping` repo, minus the onesrequiring WASIp3 support in `wasmtime-wasi[-http]`, which will be PR'dseparately.* add audits and exemptions for new `component-async-tests` depsIn order to convince `cargo vet` that we only needed these deps to be`safe-to-run` (not necessarily `safe-to-deploy`, since it's test code), I'vemoved the `wasm-compose` dep to the `dev-dependencies` section of the`Cargo.toml` file, which required rearranging some code.I've exempted `wasm-compose` since it's a BA project, and also exempted all butone of the remaining new deps since they each get well over 10,000 downloads perday from crates.io. I've audited and certified the remaining dep, `im-rc`,which came in a bit shy of the 10,000-per-day mark.Signed-off-by: Joel Dice <[email protected]>* simplify `component_async_tests::util::sleep`Signed-off-by: Joel Dice <[email protected]>---------Signed-off-by: Joel Dice <[email protected]>
Avoid running wasm-opt when compiling test programs (#11075)wasm-opt will strip DWARF debug info, which causes a test failure fordebug::lldb::dwarf_codegen_optimized_wasm_optimized.clang automat
Avoid running wasm-opt when compiling test programs (#11075)wasm-opt will strip DWARF debug info, which causes a test failure fordebug::lldb::dwarf_codegen_optimized_wasm_optimized.clang automatically runs wasm-opt if it is found in PATH and if optimizationis enabled. clang-20 has a --no-wasm-opt option, but that doesn't workfor wasi-sdk-25.
Update nightly used in CI (#10957)A new lint was added to rustc so this updates the nightly used in CI andthen additionally fixes the lints that are firing.
Consolidate building guest programs for tests (#10654)* Consolidate building guest programs for testsThis commit folds all C/C++ tests for debuginfo into the same locationas Rust programs compil
Consolidate building guest programs for tests (#10654)* Consolidate building guest programs for testsThis commit folds all C/C++ tests for debuginfo into the same locationas Rust programs compiled to wasm. That means that all of`tests/all/debug/testsuite/*` is moved to`crates/test-programs/src/bin/*`. Additionally the hardcoded per-testconfiguration in `crates/test-programs/artifacts/build.rs` was removedin favor of in-test configuration at the top of the file. This meansthat all precompiled binaries are removed from the tree for DWARFtesting.Various offsets were updated, a test was dropped as it wasn't used, andsome commands/directives were updated to account for this adjustment.prtest:debug* Adjust a lint setting
[DI] Fix live range tracking off-by-one confusions (#10570)* Dump blocks in the VL table* Add a test* Work around #10572 in tests* [DI] Fix live range tracking off-by-one confusionsHow thin
[DI] Fix live range tracking off-by-one confusions (#10570)* Dump blocks in the VL table* Add a test* Work around #10572 in tests* [DI] Fix live range tracking off-by-one confusionsHow things used to work w.r.t. instruction indices (IIs):1) In lowering: - Reversed order: IIs represented "before IP"s. - Block args were defined one instruction too late, but this issue was masked due to how RA allocates, at least in simple examples. - Execution order: IIs represented "after IP"s.2) In RA: - IIs represented "before IP"s. - Notice the mismatch.3) In emit: - RA directions w.r.t. the explicit ProgPoint positions were not respected and always treated as "after".How things work after this change:1) In lowering: - Reversed order: IIs represent "after IP"s. - Execution order: IIs represent "before IP"s.2) In RA: - No change; mismatch fixed.3) In emit: - ProgPoint positions now respected.This fixes various "silent bad debug info" issues.
Bump MSRV to 1.84.0 (#10520)* Bump MSRV to 1.84.0Coupled with today's release of 1.86.0* Fix tests on windows
Update some dependencies on their major version tracks (#10425)* Update rustix to 1.0.x* Bump itertools to its latest version* Update base64 to its latest version* Update wit-bindgen to its l
Update some dependencies on their major version tracks (#10425)* Update rustix to 1.0.x* Bump itertools to its latest version* Update base64 to its latest version* Update wit-bindgen to its latest version* Update v8 to its latest version on crates.ioJust keeping up-to-date* Update capstone dependency to its latest version* Update libtest-mimic to its latest version* Update cargo-metadata dependency* Update thiserror dependency to latest* Update bytesize dependency* Drop getrandom dependency from test-programsFavor using `wasi::random_get` instead for now.* Fix deny.toml syntax* Fix merge conflict* Downgrade v8 to respect MSRV* Fix compile on windows
[DWARF] Fix the loclist to exprloc optimization (#10400)* Add a test* Fix the loclist -> exprloc optimizationThe expression must be valid over the entire parent scope.
Initial implementation of Wasi-tls (Transport Layer Security) (#10249)* Initial implementation of wasi-tlsThis crate provides the Wasmtime host implementation for the [wasi-tls] API.The [wasi-t
Initial implementation of Wasi-tls (Transport Layer Security) (#10249)* Initial implementation of wasi-tlsThis crate provides the Wasmtime host implementation for the [wasi-tls] API.The [wasi-tls] world allows WebAssembly modules to perform SSL/TLS operations,such as establishing secure connections to servers. TLS often relies on other wasi networking systemsto provide the stream so it will be common to enable the [wasi:cli] world as well with the networking features enabled.The initial implemntation is using rustls.Signed-off-by: James Sturtevant <[email protected]>* Remove configuration object for nowSigned-off-by: James Sturtevant <[email protected]>* Update cargo patch to use temp branchSigned-off-by: James Sturtevant <[email protected]>* Rename tcp streams to wasistreams to be more genericSigned-off-by: James Sturtevant <[email protected]>* gate the wasi-tls ctx behind a featureSigned-off-by: James Sturtevant <[email protected]>* cleanup and clippy fixesSigned-off-by: James Sturtevant <[email protected]>* Fix issue when another pollable cancelsSigned-off-by: James Sturtevant <[email protected]>* prtest:fullSigned-off-by: James Sturtevant <[email protected]>* Skip test on riscv64/s390xSigned-off-by: James Sturtevant <[email protected]>* Drop debug info to support tests on pulley based platformsSigned-off-by: James Sturtevant <[email protected]>* Update signature of `close-notify`* Use draft versionSigned-off-by: James Sturtevant <[email protected]>* Remove patchesSigned-off-by: James Sturtevant <[email protected]>* Ungate tls on riscv64 and s390x* Un-gate wais-http on riscv64/s390x as well* Add wasmtime-wasi-tls to publish list* Add wasmtime-wasi-tls to public API crate list* Revert some changes to Cargo.lock---------Signed-off-by: James Sturtevant <[email protected]>Signed-off-by: James Sturtevant <[email protected]>Co-authored-by: badeend <[email protected]>Co-authored-by: Alex Crichton <[email protected]>
[DWARF] Basic infrastructure for compiling test assets from source (#10193)* CI* Source* Remove generic.wasm from source control
Improve rebuild detection of test-programs (#10303)This commit improves the logic of detecting when to rebuild the`test-programs` artifacts used during test by parsing the `*.d` filesthat Cargo e
Improve rebuild detection of test-programs (#10303)This commit improves the logic of detecting when to rebuild the`test-programs` artifacts used during test by parsing the `*.d` filesthat Cargo emits as part of its compilation and using that as the`cargo:rerun-if-changed` directive. This not only includes what waspreviously depended on but additionally includes features such as `path`dependencies which might temporarily be used during development.
Rename wasi-runtime-config to wasi-config (#9404)* Rename wasi-runtime-config to wasi-config* Remove audit-as-cratesio entryThis crate no longer exists on crates.io so cargo-vet is failing---
Rename wasi-runtime-config to wasi-config (#9404)* Rename wasi-runtime-config to wasi-config* Remove audit-as-cratesio entryThis crate no longer exists on crates.io so cargo-vet is failing---------Co-authored-by: Alex Crichton <[email protected]>
Add `rust-version.workspace = true` to all crates (#9112)Right now this is only on some crates such as `wasmtime` itself and`wasmtime-cli`, but by applying it to all crates it helps with versions
Add `rust-version.workspace = true` to all crates (#9112)Right now this is only on some crates such as `wasmtime` itself and`wasmtime-cli`, but by applying it to all crates it helps with versionselection of those using just Cranelift for example.
Implement wasi-keyvalue (#8983)* Implement wasi-keyvalue* Allow preset data for In-Memory provider, rename allow_hosts to allow_redis_hosts* Add vets---------Co-authored-by: Alex Crichton <
Implement wasi-keyvalue (#8983)* Implement wasi-keyvalue* Allow preset data for In-Memory provider, rename allow_hosts to allow_redis_hosts* Add vets---------Co-authored-by: Alex Crichton <[email protected]>
Rename the `wasm32-wasi` target to `wasm32-wasip1` (#8867)This rename is happening in upstream Rust and should be in enough placesnow.prtest:full
12