Several fixes to debugging infrastructure: component vs. module PCs and gdbstub wasm module names. (#12901)* Debugging: fix module-relative vs component-relative PCs and unique library names.Two
Several fixes to debugging infrastructure: component vs. module PCs and gdbstub wasm module names. (#12901)* Debugging: fix module-relative vs component-relative PCs and unique library names.Two bugfixes for guest debugging with components:1. Convert component-relative source locations to module-relative PCs in the frame table. The guest-debug API presents a core-Wasm view where components are deconstructed into individual modules, so all PCs must be module-relative. This adds a `wasm_module_offset` field to `ModuleTranslation` and `FuncEnvironment`, set during component translation, and subtracts it in `debug_tags()`.2. Give unique names to "library" entries in the gdbstub XML response. LLDB's DynamicLoader deduplicates by name, so using "wasm" for all modules caused only the first to be loaded.* Debugging: add ModulePC and ComponentPC newtypes for Wasm PC offsets.Introduce `ModulePC` (module-relative) and `ComponentPC`(component-relative) newtype wrappers around u32 Wasm bytecodeoffsets. These replace raw u32 values throughout the frame table,breakpoint, and debug systems to prevent confusion between the twooffset spaces.* Debugging: add regression test for component module-relative PCs.
show more ...
Debugging: add integration test with LLDB and some minor tweaks. (#12856)This PR adds:- An integration-test that runs LLDB against the Wasmtime CLI to verify basic debugging functionality, simi
Debugging: add integration test with LLDB and some minor tweaks. (#12856)This PR adds:- An integration-test that runs LLDB against the Wasmtime CLI to verify basic debugging functionality, similar to the existing native-debug tests.- A CI job that runs the above in CI.- Some minor tweaks to the gdbstub debugger design: - Rather than the initial single-step to get to the first Wasm instruction where module(s) will be instantiated into the store and visible to the debugger, we pre-register modules with the store eagerly. This avoids the slightly hacky flow and also is a preparation step for `wasmtime serve` debugging, where we can't single-step into execution eagerly (because execution doesn't start at all until an HTTP request arrives). - Add a separate message-printing path for "debugger info messages", allowing us to print the "debugger is listening on <PORT>" message without inheriting stderr for the whole debugger component environment. This message is necessary for the above integration test (it parses the message to determine when the debuggee is ready).
fix: correct various typos (#12807)Signed-off-by: Ho Kim <[email protected]>
Debugging: add the debug-main world. (#12756)* Debugging: add the debug-main world.This PR "draws the rest of the owl" for the debug-mainworld (bytecodealliance/rfcs#45). This includes a WIT wor
Debugging: add the debug-main world. (#12756)* Debugging: add the debug-main world.This PR "draws the rest of the owl" for the debug-mainworld (bytecodealliance/rfcs#45). This includes a WIT world that hostsdebug components that have access to "host debug powers" via adebugging API, and the ability to load such a debug-component and giveit control of the main program as a debuggee when using `wasmtimerun`.The WIT is namespaced to `bytecodealliance:wasmtime` and is slightlyaspirational in places: for example, the host does not yet implementinjection of early return values or exception-throws. I intend to fillout a series of TODO issues once this all lands to track followup("post-MVP") work.This PR does not include any debug components. I separately have agdbstub component, with which I tested and co-developed this host-sideimplementation. My plan is to land it in a followup PR as a componentthat will be embedded in/shipped with the Wasmtime CLI and availableunder an easy-to-use CLI option. Once we have that gdbstub component,we can also implement end-to-end integration tests that boot up LLDBand run through an expected interaction. (Separately, thoseintegration tests will require a release of wasi-sdk to ship an LLDBbinary that we can use.) As such, there are no real tests in this PR:interesting behaviors only really occur with a full end-to-end flow.The integration with the CLI is a little awkward (we internally buildanother `wasmtime run` command that invokes the debug component, andtie it together with the debuggee via a special `invoke_debugger` API;this seemed less bad than reworking all of the WASI setup to be morereusable). Happy to take more ideas here.* Review feedback.* Review feedback.* Review feedback: update vendor-wit.sh.* Review feedback: -Ddebugger-arg= -> -Darg=.* Review feedback.* Review feedback.* Review feedback: factor host.rs into several submodules.* Review feedback: rename Debugger to Debuggee on host side.* Review feedback: split inherit_stdin_stdout, and add corresponding options for the debug component.* Review feedback.* Review feedback.* Add simple debug-component tests.* Add wasm32-wasip2 target in a few places in CI* Cargo vets for wstd dependency.* Add wasm32-wasip2 in more places* fix debug-component test dependence on componentization byte offsets* Review feedback.* Fix cancel-safety of EventFuture.* Fix: Interrupted events should only occur after interrupt(), not on every epoch yield.* Review feedback.* Review feedback: strip down WASI imports in debugger world.* fold debugger test component back into wasip1 + adapter test artifact compilation flow
Debugger: a few updates to the debugger crate. (#12684)This PR upstreams a few changes from my WIP debug-component branch tothe async-wrapper in the debugger crate:- The inner debuggee body take
Debugger: a few updates to the debugger crate. (#12684)This PR upstreams a few changes from my WIP debug-component branch tothe async-wrapper in the debugger crate:- The inner debuggee body takes a future that need not be `'static`; this is important later in the integration.- Two race conditions fixed: (i) wait for initial debuggee body startup before sending a `with_store` query; and (ii) allow `with_store` queries after receiving a `Finished` message, which may happen when processing final debugger commands after the program completes.- As a result of some shuffling around future lifetimes and other type-system head-banging, as well as the termination race-condition fixed, the `Store<T>` is not transferred to the debuggee body and back via the `JoinHandle`; rather it is passed back in a message upon completion.
Debugging: refactor stack frame cursor into frame handle abstraction. (#12566)* Debugging: refactor stack frame cursor into frame handle abstraction.This addresses some of the issues described #1
Debugging: refactor stack frame cursor into frame handle abstraction. (#12566)* Debugging: refactor stack frame cursor into frame handle abstraction.This addresses some of the issues described #12486: we need the abilityto keep a handle to a stack frame as long as execution is frozen, andkeep multiple of these handles around, alongside the `Store`, withoutany handle directly holding a borrow of the store.The frame handles work by means of an "execution version" scheme: theidea is that whenever any execution resumes in a given store, allhandles to existing frames could be invalidated, but if no suchexecution occurs, all handles should still be valid. A tuple of(globally unique for process lifetime) store ID, and execution versionwithin that store, should be sufficient to uniquely identify anyfrozen-stack period during execution. This accomplishes cheap handleinvalidation without the need to track existing handles.This PR also implements a cache of parsed frame-table data. Previouslythis was lazily parsed by the cursor as it walked up a stack, but withmultiple handles hanging around, and with handles meant to be cheap tohold and clone, and with handles being invalidated eagerly, it makesmuch more sense to persist this parsed metadata at the `Store` level.(It cannot persist at the `Engine` level because PCs are local perstore.)* Re-bless disas tests (offsets in VMStoreContext changed).* Handle invalidation tests.* Review comments, and make API return `Result`s rather than panic'ing on stale handles.* Review feedback.* Doc-comment link fix.* Review feedback.* cfg-gate Activation method to `debug` feature only.* Fix unused-import warning in no-debug cfg.* Fix doc link (again, after rename from latest feedback).
Remove need for explicit `Config::async_support` knob (#12371)* Refactor component model host function definitionsPush the `async`-ness down one layer.* Remove need for explicit `Config::async
Remove need for explicit `Config::async_support` knob (#12371)* Refactor component model host function definitionsPush the `async`-ness down one layer.* Remove need for explicit `Config::async_support` knobThis commit is an attempt to step towards reconciling "old async" and"new async" in Wasmtime. The old async style is the original asyncsupport in Wasmtime with `call_async`, `func_wrap_async`, etc, where themain property is that the store is "locked" during an async operation.Put another way, a store can only execute at most one async operation ata time. This is in contrast to "new async" support in Wasmtime with thecomponent-model-async (WASIp3) support, where stores can have more thanone async operation in flight at once.This commit does not fully reconcile these differences, but it doesremove one hurdle along the way: `Config::async_support`. Since thebeginning of Wasmtime this configuration knob has existed to explicitlydemarcate a config/engine/store as "this thing requires `async` stuffinternally." This has started to make less and less sense over timewhere the line between sync and async has become more murky with WASIp3where the two worlds comingle. The goal of this commit is to deprecate`Config::async_support` and make the function not actually do anything.In isolation this can't simply be done, however, because there are manyload-bearing aspects of Wasmtime that rely on this `async_support` knob.For example once epochs + yielding are enabled it's required that allWasm is executed on a fiber lest it hit an epoch and not know how toyield. That means that this commit is not a simple removal of`async_support` but instead a refactoring/rearchitecting of how async isused internally within Wasmtime. The high-level ideas within Wasmtimenow are:* A `Store` has a "requires async" boolean stored within it.* All configuration options which end up requiring async, such as yielding with epochs, turn this boolean on.* Creation of host functions which use async (e.g. `func_wrap_{async,concurrent}`) will also turn this option on.* Synchronous API entrypoints into Wasmtime ensure that this boolean is disabled.* Asynchronous APIs are usable at any time.This means that the concept of an async store vs a sync store is nowgone. All stores are equally capable of executing sync/async, and thechange now is that dynamically some stores will require that async isused with certain configuration. Additionally all panicking conditionsaround `async_support` have been converted to errors instead. Allrelevant APIs already returned an error and things are murky enough nowthat it's not necessarily trivial to get this right at the embedderlevel. In the interest of avoiding panics all detected async mismatchesare now first-class `wasmtime::Error` values.The end result of this commit is that `Config::async_support` is adeprecated `#[doc(hidden)]` function that does nothing. While manyinternal changes happened as well as having new tests for all this sortof behavior this is not expected to have a great impact on externalconsumers. In general a deletion of `async_support(true)` is in theoryall that's required. This is intended to make it easier to think aboutasync/sync/etc in the future with WASIp3 and eventually reconcile`func_wrap_async` and `func_wrap_concurrent` for example. That's leftfor future refactorings however.prtest:full* Review comments* Fix CI failures
Migrate debugger to `wasmtime::error` (#12261)
Debugger: add top-level crate and async wrapper allowing for a "debugger environment". (#12183)* Debugger: add top-level crate and async wrapper allowing for a "debug environment".The debug suppo
Debugger: add top-level crate and async wrapper allowing for a "debugger environment". (#12183)* Debugger: add top-level crate and async wrapper allowing for a "debug environment".The debug support in Wasmtime so far is structured around asynccallbacks that occur at certain kinds of events, like breakpoints. Thisis a suitable foundation but makes for an awkward implementation of atop-level debugger implementation, which likely has an event loopdealing with user commands (via a UI or a protocol connection) andexpects to perform actions such as "run until next breakpoint".This PR introduces a new crate that wraps a `Store` in a `Debugger`.This wrapper embodies an inner async body that can perform whateveractions it likes on the `Store` that is passed back in. This inner bodyis spawned as an async task. The debugger wrapper registers its own`DebugHandler` callback that communicates with the outside world viabidirectional command/response queues.On the "outside", the `Debugger` presents an interface suitable forinserting into a debug protocol server or UI: an async method that runsuntil next event and returns that event, and a method that permitsquerying or modifying the store whenever the `run` method is notexecuting. The latter operates by sending a closure over the queue,because the `Store` must continue to be owned by the async task that is(still) running and suspended in async callbacks.Right now, this is exercised only via a few unit tests, but the intentis to next build up the "top half" of the debugger using thisabstraction, e.g. by running a gdbstub protocol server (likely as a Wasmcomponent in a "debug-main WIT world" -- RFC needed for this).Also, when we eventually move debugging over to native use of`run_concurrent`, this paradigm should remain mostly unchanged at thislevel of API: there can still be an object that has an async method thatruns and yields the next event, and there can still be a method thattakes a closure that can operate (within its scope only) on the `Store`.A few warts that I could use feedback on:- Cancelation safety is weird. Fibers panic when dropped before execution of their body completes, and this seems to mean that we can't allow a `Debugger` to drop early (or at least, the `tokio::test` unit test that owns the runtime that runs the async task to finish before the debugged body completes!). If there is a better way to handle cancelation safety here, I'm all ears.- It's not clear to me if the boxed-closure-and-`Any` approach to providing access to the `Store` is the best we can do, but I suspect it is.* Cancel safety!* Add new crate to publish.rs script.* Review feedback.* Review feedback: state diagram.* Update after merge from main: new DebugEvent for epoch yields.* Make Debugger drop-safe by making debug event callbacks compatible with fiber teardown.* CI fix on `debugger` crate manifest.- Do not link to non-existent README in new crate.- Remove a few other attributes that our internal crates don't have (now the set of attributes at the top level is the same as for the new error crate).