Several fixes to debugging infrastructure: component vs. module PCs and gdbstub wasm module names. (#12901)

* Debugging: fix module-relative vs component-relative PCs and unique library names.

Two

Several fixes to debugging infrastructure: component vs. module PCs and gdbstub wasm module names. (#12901)

* Debugging: fix module-relative vs component-relative PCs and unique library names.

Two bugfixes for guest debugging with components:

1. Convert component-relative source locations to module-relative PCs
in the frame table. The guest-debug API presents a core-Wasm view
where components are deconstructed into individual modules, so all
PCs must be module-relative. This adds a `wasm_module_offset` field
to `ModuleTranslation` and `FuncEnvironment`, set during component
translation, and subtracts it in `debug_tags()`.

2. Give unique names to "library" entries in the gdbstub XML response.
LLDB's DynamicLoader deduplicates by name, so using "wasm" for all
modules caused only the first to be loaded.

* Debugging: add ModulePC and ComponentPC newtypes for Wasm PC offsets.

Introduce `ModulePC` (module-relative) and `ComponentPC`
(component-relative) newtype wrappers around u32 Wasm bytecode
offsets. These replace raw u32 values throughout the frame table,
breakpoint, and debug systems to prevent confusion between the two
offset spaces.

* Debugging: add regression test for component module-relative PCs.

show more ...

Debugging: add debug-tags to instrumented trap sites so we actually get PCs on traps. (#12802)

This was not exposed earlier by (i) lack of handling of trap events in
the initial version of the gdbst

Debugging: add debug-tags to instrumented trap sites so we actually get PCs on traps. (#12802)

This was not exposed earlier by (i) lack of handling of trap events in
the initial version of the gdbstub component in #12771, and (ii) lack
of asserting some value for the PC on the top frame in the debug-event
test for traps. We got the PC for the last opcode in the function body
previously because, with no debug tags on the trapping path that calls
raise() (sunk to the bottom of the machine code body as cold code), we
scanned backward for the last tag metadata and found that
instead. Adding metadata according to the current source location when
emitting traps fixes this for all trapping events.

show more ...

Debugging: PC in a frame at a callsite should be the return address, not the call. (#12750)

* Debugging: PC in a frame at a callsite should be the return address, not the call.

In working out why a

Debugging: PC in a frame at a callsite should be the return address, not the call. (#12750)

* Debugging: PC in a frame at a callsite should be the return address, not the call.

In working out why a `finish` command in
LLDB-attached-to-Wasmtime-via-gdbstub wasn't working, I discovered that
our current debugging APIs, when presenting info from a frame suspended
at a callsite up the stack, present the current PC as *at* the call
instruction, rather than *past it* (at the return address). The latter
is conventional on all real ISAs, and is hence what the debugger
expects.

This PR makes the most straightforward fix: the debug tuple attached to
the call, and hence the metadata read out by the debug frame walker, now
encodes the PC of the next opcode. This is sufficient to fix `finish`
within LLDB.

An alternative I considered, and prototyped, is also worth mentioning:
one might see the argument for allowing a debugger to see the callsite
that invoked the next frame, and separately, see the return address
(i.e., both pieces of information are useful). In [an alternative
branch], there is a new table in the debug frame info metadata giving
the size of each callsite, so the debug frame-handle API can present a
`get-return-address` accessor on a `frame` resource alongside `get-pc`.
Ultimately I opted not to go with this because it has more overhead and
complexity and a *concrete* use-case wasn't forthcoming to me, but I'm
happy to reconsider if someone wants that instead.

[an alternative branch]: https://github.com/cfallin/wasmtime/tree/debugger-return-address-separate

* Fix return PC in Pulley test.

* Fix disas test.

show more ...

Cranelift: upgrade to regalloc2 0.14.0 and use static/constant `MachineEnv`s. (#12596)

* Update to new regalloc2 with constant MachineEnv.

* Re-bless Cranelift filetests.

* Re-bless Wasmtime disas

Cranelift: upgrade to regalloc2 0.14.0 and use static/constant `MachineEnv`s. (#12596)

* Update to new regalloc2 with constant MachineEnv.

* Re-bless Cranelift filetests.

* Re-bless Wasmtime disas tests.

* Update to RA2 0.14.0.

* Review feedback.

* cargo-vet update.

show more ...

Debugging: refactor stack frame cursor into frame handle abstraction. (#12566)

* Debugging: refactor stack frame cursor into frame handle abstraction.

This addresses some of the issues described #1

Debugging: refactor stack frame cursor into frame handle abstraction. (#12566)

* Debugging: refactor stack frame cursor into frame handle abstraction.

This addresses some of the issues described #12486: we need the ability
to keep a handle to a stack frame as long as execution is frozen, and
keep multiple of these handles around, alongside the `Store`, without
any handle directly holding a borrow of the store.

The frame handles work by means of an "execution version" scheme: the
idea is that whenever any execution resumes in a given store, all
handles to existing frames could be invalidated, but if no such
execution occurs, all handles should still be valid. A tuple of
(globally unique for process lifetime) store ID, and execution version
within that store, should be sufficient to uniquely identify any
frozen-stack period during execution. This accomplishes cheap handle
invalidation without the need to track existing handles.

This PR also implements a cache of parsed frame-table data. Previously
this was lazily parsed by the cursor as it walked up a stack, but with
multiple handles hanging around, and with handles meant to be cheap to
hold and clone, and with handles being invalidated eagerly, it makes
much more sense to persist this parsed metadata at the `Store` level.
(It cannot persist at the `Engine` level because PCs are local per
store.)

* Re-bless disas tests (offsets in VMStoreContext changed).

* Handle invalidation tests.

* Review comments, and make API return `Result`s rather than panic'ing on stale handles.

* Review feedback.

* Doc-comment link fix.

* Review feedback.

* cfg-gate Activation method to `debug` feature only.

* Fix unused-import warning in no-debug cfg.

* Fix doc link (again, after rename from latest feedback).

show more ...

Debug: implement breakpoints and single-stepping. (#12133)

* Debug: implement breakpoints and single-stepping.

This is a PR that puts together a bunch of earlier pieces (patchable
calls in #12061 a

Debug: implement breakpoints and single-stepping. (#12133)

* Debug: implement breakpoints and single-stepping.

This is a PR that puts together a bunch of earlier pieces (patchable
calls in #12061 and #12101, private copies of code in #12051, and all
the prior debug event and instrumentation infrastructure) to implement
breakpoints in the guest debugger.

These are implemented in the way we have planned in #11964: each
sequence point (location prior to a Wasm opcode) is now a patchable call
instruction, patched out (replaced with NOPs) by default. When patched
in, the breakpoint callsite calls a trampoline with the `patchable` ABI
which then invokes the `breakpoint` hostcall. That hostcall emits the
debug event and nothing else.

A few of the interesting bits in this PR include:
- Implementations of "unpublish" (switch permissions back to read/write
from read/execute) for mmap'd code memory on all our platforms.
- Infrastructure in the frame-tables (debug info) metadata producer and
parser to record "breakpoint patches".
- A tweak to the NOP metadata packaged with the `MachBuffer` to allow
multiple NOP sizes. This lets us use one 5-byte NOP on x86-64, for
example (did you know x86-64 had these?!) rather than five 1-byte
NOPs.

This PR also implements single-stepping with a global-per-`Store` flag,
because at this point why not; it's a small additional bit of logic to
do *all* patches in all modules registered in the `Store` when that flag
is enabled.

A few realizations for future work:
- The need for an introspection API available to a debugger to see the
modules within a component is starting to become clear; either that,
or the "module and PC" location identifier for a breakpoint switches
to a "module or component" sum type. Right now, the tests for this
feature use only core modules. Extending to components should not
actually be hard at all, we just need to build the API for it.
- The interaction between inlining and `patchable_call` is interesting:
what happens if we inline a `patchable_call` at a `try_call` callsite?
Right now, we do *not* update the `patchable_call` to a `try_call`,
because there is no `patchable_try_call`; this is fine in the Wasmtime
embedding in practice because we never (today!) throw exceptions from
a breakpoint handler. This does suggest to me that maybe we should
make patchability a property of any callsite, and allow try-calls to
be patchable too (with the same restriction about no return values as
the only restriction); but happy to discuss that one further.

* Add missing debug.wat disas test.

* Review feedback.

* Fix comment on `CodeMemory::text_mut`.

* Review feedback.

* Review feedback: abort process on failure to re-apply executable permissions.

* Implement icache flush for aarch64.

This appears to be necessary as we otherwise see a failure in CI on
macOS/aarch64 that is consistent with patched-in breakpoint calls still
being incorrectly cached after we remove them and republish the code.

There is a longstanding issue in #3310 tracking proper icache coherence
handling on aarch64. We implemented this for Linux with the `membarrier`
syscall but never did so for macOS. Maybe this is the first point at
which it matters, because code was always loaded at new addresses (hence
did not have coherence issues because nothing would have been cached)
previously.

prtest:full

* Review feedback: use `next_multiple_of`.

show more ...

Debugging: force hostcall-based-traps when guest-debugging is enabled. (#12052)

* Debugging: force hostcall-based-traps when guest-debugging is enabled.

We manually configure this for tests now but

Debugging: force hostcall-based-traps when guest-debugging is enabled. (#12052)

* Debugging: force hostcall-based-traps when guest-debugging is enabled.

We manually configure this for tests now but this change formalizes the
dependence on hostcall-based traps that we outlined in #11964.

* Re-bless tests.

* Review feedback.

show more ...

Wasmtime: implement debug instrumentation and basic host API to examine runtime state. (#11769)

* Wasmtime: implement debug instrumentation and basic host API to examine runtime state.

This PR impl

Wasmtime: implement debug instrumentation and basic host API to examine runtime state. (#11769)

* Wasmtime: implement debug instrumentation and basic host API to examine runtime state.

This PR implements ideas from the [recent RFC] to serve as the basis
for Wasm (guest) debugging: it adds a stackslot to each function
translated from Wasm, stores to replicate Wasm VM state in the
stackslot as the program runs, and metadata to describe the format of
that state and allow reading it out at runtime.

As an initial user of this state, this PR adds a basic "stack view"
API that, from host code that has been called from Wasm, can examine
Wasm frames currently on the stack and read out all of their locals
and stack slots.

Note in particular that this PR does not include breakpoints,
watchpoints, stepped execution, or any sort of user interface for any
of this; it is only a foundation.

This PR still has a few unsatisfying bits that I intend to address:

- The "stack view" performs some O(n) work when the view is initially
taken, computing some internal data per frame. This is forced by the
current design of `Backtrace`, which takes a closure and walks that
closure over stack frames eagerly (rather than work as an
iterator). It's got some impressive iterator-chain stuff going on
internally, so refactoring it to the latter approach might not
be *too* bad, but I haven't tackled it yet.

A O(1) stack view, that is, one that does work only for frames as
the host API is used to walk up the stack, is desirable because some
use-cases may want to quickly examine e.g. only the deepest
frame (say, running with a breakpoint condition that needs to read a
particular local's value after each step).

- It includes a new `Config::compiler_force_inlining()` option that is
used only for testing that we get the correct frames after
inlining. I couldn't get the existing flags to work on a Wasmtime
config level and suspect there may be an existing bug there; I will
try to split out a fix for it.

This PR renames the existing `debug` option to `native_debug`, to
distinguish it from the new approach.

[recent RFC]: https://github.com/bytecodealliance/rfcs/pull/44

* Update to new APIs on Cranelift side.

* Test update.

* Adjust objdump printing of InstPos on frame progpoints; and adjust progpoint collapsing.

* Convert to iterator form.

* Fix path in native-debug tests (debug -> native_debug rename).

* Enforce that `debug_instrumentation` can only be enabled when feature is enabled.

* Add missing assert.

* Use builtin knob for forcing intra-module inlining instead.

* Review feedback:

- Make StackView own the current frame rather than handing it out. This
prevents the current frame (`FrameView`) from walking away and hiding
somewhere it shouldn't, to be used unsoundly later.
- Assert no-GC during stack walk.

* Merge debug-instrumentation hooks on FuncEnvironment into before/after hooks.

* Review feedback: avoid downcasting funcs twice.

* Add debug feature to `wasmtime` crate's defaults.

* Review feedback: u32s for local and stack indices in debug host API.

* Use *const u8 as stack pointers and `with_exposed_provenance` in debug API.

* Remove some `srcloc` plumbing in Wasm translator.

* Rename native-debug back to debug, and make the new thing "guest debugging".

* rustfmt in debugging test.

* fix disas test after guest-debug CLI option rename.

* Review feedback: no separate debug-instrumentation hooks on FuncEnvironment.

* Review feedback: update doc comment on `Config::guest_debug`.

* Review feedback: rename `generate_debuginfo` to `debug_native` in tunables.

* Review feedback: miscellaneous comments.

* Review comment: fix wording in safety conditions.

* revert wasi-common submodule update

* Properly root values in debug frame slots.

Fixes #11841.

* Fix non-`debug`-feature build.

* Review feedback: naming.

* Ignore tests that compile modules in miri.

show more ...