dcd65446 - support WASIp3 instance reuse in `wasmtime serve` (#11807)

Mon, 20 Oct 2025 14:55:48 +0000

support WASIp3 instance reuse in `wasmtime serve` (#11807)* support WASIp3 instance reuse in `wasmtime serve`This is a draft implementation of WASIp3 instance reuse. Previously, we usedone store and one instance per request for both p2 and p3 handlers, discardingthe store and instance immediately after the request was handled. Now weattempt to reuse p3 instances and their stores instead, both serially andconcurrently.Advantages of reuse:- Higher throughput: We can amortize the time spent creating and disposing of instances and stores over a number of requests.- Lower memory and address space usage: Concurrent instance reuse allows us to handle more requests with fewer instances, especially when the handler is I/O bound.- Developers will presumably need to make changes to their applications anyway when migrating from p2 to p3, so this is an ideal time to change the default from no reuse to (some) reuse, assuming we want to do it at all.Disadvantages of reuse:- Reduced isolation: When the same instance is used to handle multiple requests, the blast radius of bugs in the guest is larger; a trap can affect unrelated requests, and a security flaw could leak state across requests.- Host implementation complexity: There's more code to manage and more configuration knobs to tune.- Guest implementation complexity: You can't just stash state in a global variable and call it a day. Similarly, shortcuts like using a leaking GC require additional thought.Given the tradeoffs, we'll definitely need to provide components a way to optout of reuse if desired. Seehttps://github.com/WebAssembly/wasi-http/issues/190 for related discussion.Implementation details:I've moved `ProxyHandler` from `wasmtime_cli::commands::serve` to`wasmtime_wasi_http::handler` (so it can be reused by custom embedders),abstracted away the `serve`-specific parts, and added support for instance reusevia a new `ProxyHandler::push` function. The `push` function takes a work itemrepresenting an incoming request and dispatches it to a dynamically sized poolof worker tasks, each with its own store and `wasi:http/handler@0.3.x` instance.Worker tasks accept work items as capacity allows until they either reach amaximum total reuse count or hit an idle timeout, at which point they exit.Performance:I've run a few benchmarks using `wasmtime-serve-rps.sh` and[hello-wasip3-http](https://github.com/dicej/hello-wasip3-http/blob/main/src/lib.rs)(a simple "hello, world" request handler). Highlights:- 45% more requests per second with instance reuse enabled vs. disabled- 64% fewer concurrent instances required for an I/O-bound handler - Here I added a 40ms delay to the "hello, world" handler to simulate I/OTODOs:- Support host-enforced request timeouts. Note that we can't simply use the epoch-based trap-on-timeout approach we've traditionally used since a given instance may be responsible for a number of concurrent requests. This will require adding a public API to the `wasmtime` crate for cleanly cancelling a guest task without affecting other ones.- Add CLI options for the configuration knobs (e.g. max request count per instance, idle timeout, etc.)- Use guest signals like `backpressure`, explicit `wasi:cli/exit/exit`, https://github.com/WebAssembly/wasi-http/issues/190, etc. to drive reuse decisionsSigned-off-by: Joel Dice * additional instance reuse supportThis addresses a couple of TODO items from my previous instance reuse PR:- Support request timeouts by gracefully shutting down the worker as soon as any request hits a timeout, letting any other pending tasks finish or time out before actually dropping the instance.- Convert the previously hard-coded config options to CLI options.- Enable profiling when instance reuse is enabled.Signed-off-by: Joel Dice * start worker tasks more aggressivelyThis improves `wasmtime serve` performance when instance reuse is enabled by:- spawning a worker task any time there are no idle workers available- considering a worker to be unavailable until it has finished initializing itselfSigned-off-by: Joel Dice * revert unecessary wasmtime-serve-rps.sh changesSigned-off-by: Joel Dice * enable WASIp2 instance reuseSigned-off-by: Joel Dice * add clarifying comments about when and why we start worker tasksSigned-off-by: Joel Dice * remove duplicated code in `ProxyHandler::spawn`By bypassing the task queue when there are no already-available workers, I'venarrowed the performance gap between the "fast path" code and the normal path toabout 2%, which is close enough that we can remove the duplicated code.Signed-off-by: Joel Dice * ensure timeouts enforced if event loop stallsThis fixes an issue such that timeouts weren't being reliably enforced when theguest either busy loops for an extended time or makes a sync call to a hostfunction that takes exclusive access to the `Store` and blocks for a while. Ineither of those cases, the `StoreContextMut::run_concurrent` event loop willstall, being unable to make progress while the task fiber monopolizes the`Store`. In this case, we must enforce timeouts _outside_ the event loop. Seethe new code comments for details on how that is done.This also tweaks `wasmtime serve` stderr/stdout output a bit to address`cli_tests::cli_serve_*` test regressions.Signed-off-by: Joel Dice * add reuse and timeout tests to `cli_tests`This adds several new integration tests which exercise various aspects of`wasmtime serve`'s instance reuse feature, including timeouts for p2-style(synchronous) sleeps and p3-style (asynchronous) sleeps.Signed-off-by: Joel Dice * fix foreach_api breakageSigned-off-by: Joel Dice * Minor cleanups to `handler.rs`* Don't use `FutureExt`, use `async { .. }` instead.* Don't use `T`, instead use `S::StoreData`.* Shift an `unreachable!()` to be closer to the "source"* add more comments to `handler.rs`These comments draw attention to a few of the more subtle aspects of the codeand the invariants it is upholding.Signed-off-by: Joel Dice * add docs regarding the `req_id` parameterSigned-off-by: Joel Dice * add comments regarding atomic load/store orderingsThis also changes one of the `SeqCst` orderings to `Relaxed`.Signed-off-by: Joel Dice ---------Signed-off-by: Joel Dice Co-authored-by: Alex Crichton List of files: /wasmtime-44.0.1/crates/test-programs/src/bin/p2_cli_serve_hello_world.rs

Changes in p2_cli_serve_hello_world.rs

dcd65446 - support WASIp3 instance reuse in `wasmtime serve` (#11807)