Several fixes to debugging infrastructure: component vs. module PCs and gdbstub wasm module names. (#12901)

* Debugging: fix module-relative vs component-relative PCs and unique library names.

Two

Several fixes to debugging infrastructure: component vs. module PCs and gdbstub wasm module names. (#12901)

* Debugging: fix module-relative vs component-relative PCs and unique library names.

Two bugfixes for guest debugging with components:

1. Convert component-relative source locations to module-relative PCs
in the frame table. The guest-debug API presents a core-Wasm view
where components are deconstructed into individual modules, so all
PCs must be module-relative. This adds a `wasm_module_offset` field
to `ModuleTranslation` and `FuncEnvironment`, set during component
translation, and subtracts it in `debug_tags()`.

2. Give unique names to "library" entries in the gdbstub XML response.
LLDB's DynamicLoader deduplicates by name, so using "wasm" for all
modules caused only the first to be loaded.

* Debugging: add ModulePC and ComponentPC newtypes for Wasm PC offsets.

Introduce `ModulePC` (module-relative) and `ComponentPC`
(component-relative) newtype wrappers around u32 Wasm bytecode
offsets. These replace raw u32 values throughout the frame table,
breakpoint, and debug systems to prevent confusion between the two
offset spaces.

* Debugging: add regression test for component module-relative PCs.

show more ...

Migrate the Wasmtime CLI to `wasmtime::error` (#12295)

* Migrate wasmtime-cli to `wasmtime::error`

* migrate benches to `wasmtime::error` as well

* Remove new usage of anyhow that snuck in

Debug: implement breakpoints and single-stepping. (#12133)

* Debug: implement breakpoints and single-stepping.

This is a PR that puts together a bunch of earlier pieces (patchable
calls in #12061 a

Debug: implement breakpoints and single-stepping. (#12133)

* Debug: implement breakpoints and single-stepping.

This is a PR that puts together a bunch of earlier pieces (patchable
calls in #12061 and #12101, private copies of code in #12051, and all
the prior debug event and instrumentation infrastructure) to implement
breakpoints in the guest debugger.

These are implemented in the way we have planned in #11964: each
sequence point (location prior to a Wasm opcode) is now a patchable call
instruction, patched out (replaced with NOPs) by default. When patched
in, the breakpoint callsite calls a trampoline with the `patchable` ABI
which then invokes the `breakpoint` hostcall. That hostcall emits the
debug event and nothing else.

A few of the interesting bits in this PR include:
- Implementations of "unpublish" (switch permissions back to read/write
from read/execute) for mmap'd code memory on all our platforms.
- Infrastructure in the frame-tables (debug info) metadata producer and
parser to record "breakpoint patches".
- A tweak to the NOP metadata packaged with the `MachBuffer` to allow
multiple NOP sizes. This lets us use one 5-byte NOP on x86-64, for
example (did you know x86-64 had these?!) rather than five 1-byte
NOPs.

This PR also implements single-stepping with a global-per-`Store` flag,
because at this point why not; it's a small additional bit of logic to
do *all* patches in all modules registered in the `Store` when that flag
is enabled.

A few realizations for future work:
- The need for an introspection API available to a debugger to see the
modules within a component is starting to become clear; either that,
or the "module and PC" location identifier for a breakpoint switches
to a "module or component" sum type. Right now, the tests for this
feature use only core modules. Extending to components should not
actually be hard at all, we just need to build the API for it.
- The interaction between inlining and `patchable_call` is interesting:
what happens if we inline a `patchable_call` at a `try_call` callsite?
Right now, we do *not* update the `patchable_call` to a `try_call`,
because there is no `patchable_try_call`; this is fine in the Wasmtime
embedding in practice because we never (today!) throw exceptions from
a breakpoint handler. This does suggest to me that maybe we should
make patchability a property of any callsite, and allow try-calls to
be patchable too (with the same restriction about no return values as
the only restriction); but happy to discuss that one further.

* Add missing debug.wat disas test.

* Review feedback.

* Fix comment on `CodeMemory::text_mut`.

* Review feedback.

* Review feedback: abort process on failure to re-apply executable permissions.

* Implement icache flush for aarch64.

This appears to be necessary as we otherwise see a failure in CI on
macOS/aarch64 that is consistent with patched-in breakpoint calls still
being incorrectly cached after we remove them and republish the code.

There is a longstanding issue in #3310 tracking proper icache coherence
handling on aarch64. We implemented this for Linux with the `membarrier`
syscall but never did so for macOS. Maybe this is the first point at
which it matters, because code was always loaded at new addresses (hence
did not have coherence issues because nothing would have been cached)
previously.

prtest:full

* Review feedback: use `next_multiple_of`.

show more ...

Wasmtime: implement debug instrumentation and basic host API to examine runtime state. (#11769)

* Wasmtime: implement debug instrumentation and basic host API to examine runtime state.

This PR impl

Wasmtime: implement debug instrumentation and basic host API to examine runtime state. (#11769)

* Wasmtime: implement debug instrumentation and basic host API to examine runtime state.

This PR implements ideas from the [recent RFC] to serve as the basis
for Wasm (guest) debugging: it adds a stackslot to each function
translated from Wasm, stores to replicate Wasm VM state in the
stackslot as the program runs, and metadata to describe the format of
that state and allow reading it out at runtime.

As an initial user of this state, this PR adds a basic "stack view"
API that, from host code that has been called from Wasm, can examine
Wasm frames currently on the stack and read out all of their locals
and stack slots.

Note in particular that this PR does not include breakpoints,
watchpoints, stepped execution, or any sort of user interface for any
of this; it is only a foundation.

This PR still has a few unsatisfying bits that I intend to address:

- The "stack view" performs some O(n) work when the view is initially
taken, computing some internal data per frame. This is forced by the
current design of `Backtrace`, which takes a closure and walks that
closure over stack frames eagerly (rather than work as an
iterator). It's got some impressive iterator-chain stuff going on
internally, so refactoring it to the latter approach might not
be *too* bad, but I haven't tackled it yet.

A O(1) stack view, that is, one that does work only for frames as
the host API is used to walk up the stack, is desirable because some
use-cases may want to quickly examine e.g. only the deepest
frame (say, running with a breakpoint condition that needs to read a
particular local's value after each step).

- It includes a new `Config::compiler_force_inlining()` option that is
used only for testing that we get the correct frames after
inlining. I couldn't get the existing flags to work on a Wasmtime
config level and suspect there may be an existing bug there; I will
try to split out a fix for it.

This PR renames the existing `debug` option to `native_debug`, to
distinguish it from the new approach.

[recent RFC]: https://github.com/bytecodealliance/rfcs/pull/44

* Update to new APIs on Cranelift side.

* Test update.

* Adjust objdump printing of InstPos on frame progpoints; and adjust progpoint collapsing.

* Convert to iterator form.

* Fix path in native-debug tests (debug -> native_debug rename).

* Enforce that `debug_instrumentation` can only be enabled when feature is enabled.

* Add missing assert.

* Use builtin knob for forcing intra-module inlining instead.

* Review feedback:

- Make StackView own the current frame rather than handing it out. This
prevents the current frame (`FrameView`) from walking away and hiding
somewhere it shouldn't, to be used unsoundly later.
- Assert no-GC during stack walk.

* Merge debug-instrumentation hooks on FuncEnvironment into before/after hooks.

* Review feedback: avoid downcasting funcs twice.

* Add debug feature to `wasmtime` crate's defaults.

* Review feedback: u32s for local and stack indices in debug host API.

* Use *const u8 as stack pointers and `with_exposed_provenance` in debug API.

* Remove some `srcloc` plumbing in Wasm translator.

* Rename native-debug back to debug, and make the new thing "guest debugging".

* rustfmt in debugging test.

* fix disas test after guest-debug CLI option rename.

* Review feedback: no separate debug-instrumentation hooks on FuncEnvironment.

* Review feedback: update doc comment on `Config::guest_debug`.

* Review feedback: rename `generate_debuginfo` to `debug_native` in tunables.

* Review feedback: miscellaneous comments.

* Review comment: fix wording in safety conditions.

* revert wasi-common submodule update

* Properly root values in debug frame slots.

Fixes #11841.

* Fix non-`debug`-feature build.

* Review feedback: naming.

* Ignore tests that compile modules in miri.

show more ...

Wasmtime/Cranelift: carry "FP to SP offset" in exception data, and use it in stackwalk. (#11500)

* Wasmtime/Cranelift: carry "FP to SP offset" in exception data, and use it in stackwalk.

Currently

Wasmtime/Cranelift: carry "FP to SP offset" in exception data, and use it in stackwalk. (#11500)

* Wasmtime/Cranelift: carry "FP to SP offset" in exception data, and use it in stackwalk.

Currently Wasmtime unwinds stack frames to look for exception handlers
by walking frames one-by-one, following the FP chain as usual, and
assuming that *these frames are contiguous*: that is, that the SP in
any given frame (bottom of that frame) is immediately above the FP of
the next lower frame, plus the FP/return address pair (e.g. 16 bytes).
This allows us to get the SP for any given frame in addition to FP. We
need SP for two reasons:

- To look up dynamic context, to match Wasm tag instances for handlers
against the thrown tag;
- To actually set SP when we resume, if we do resume to a handler in
this frame.

This logic *almost but not quite* worked: I had forgotten that in our
tail-call ABI, we need to clean up incoming stack args in the
callee (because only the final callee in a parade of tail-calling
functions that reuse the same stack frame location knows how many args
it has, not the original caller). This implies that there is an
"incoming args area" *above* the FP/return address pair. Thus, frames
are not necessarily contiguous by the above definition.

In #11489 we see a case where a function of signature `(func)`
tail-calls one of `(func (param i32 i32 i32 i32 i32))`, which on
x86-64 (with four arg registers left for Wasm) is sufficient to create
incoming stack args, which then trips up the unwinder, reading a bogus
vmctx and segfaulting.

The most reasonable solution seems to be to embed the SP-to-FP offset
in the exception metadata itself, so from only the FP (which is
totally robust -- we rely on the FP chain for multiple kinds of
stack-walking) we can get the SP, allowing us to read dynamic context
and to reset SP during resume.

This PR does just that. Technically, in our ABI, the SP-to-FP offset
is constant for an entire function, but it was simpler in the
exception metadata to encode this per callsite instead (there is no
other notion of "per-function" data, only "per-callsite", so it would
be a separate binary search).

Fixes #11489.

prtest:full

* Review feedback.

show more ...

WebAssembly exception-handling support. (#11326)

* WebAssembly exception-handling support.

This PR introduces support for the [Wasm exception-handling proposal],
which introduces a conventional try

WebAssembly exception-handling support. (#11326)

* WebAssembly exception-handling support.

This PR introduces support for the [Wasm exception-handling proposal],
which introduces a conventional try/catch mechanism to WebAssembly. The
PR supports modules that use `try_table` to register handlers for a
lexical scope; and provides `throw` and `throw_ref` that allocate (in
the first case) and throw exception objects.

This PR builds on top of the work in #10510 for Cranelift-level
exception support, #10919 for an unwinder, and #11230 for exception
objects built on top of GC, in addition a bunch of smaller fix and
enabling PRs around those.

[Wasm exception-handling proposal]: https://github.com/WebAssembly/exception-handling/

prtest:full

* Permit UnwindToWasm to have unused fields in Pulley builds (for now).

* Resolve miri-caught reborrowing issue.

* Ignore exceptions tests in miri for now (Pulley not supported).

* Use wasmtime_test on exceptions tests.

* Get tests passing on pulley platforms

* Add a check to `supports_host` for the generated test and assert
failure also when that is false.
* Remove `pulley_unsupported` test as it falls out of `#[wasmtime_test]`
* Remove `exceptions_store` helper as it falls out of `#[wasmtime_test]`
* Remove miri annotations as they fall out of `#[wasmtime_test]`

* Remove dead import

* Skip some unsupported tests entirely in `#[wasmtime_test]`

If the selected compiler doesn't support the host at all then there's no
need to run it. Actually running it could misinterpret `CraneliftNative`
as "run with pulley" otherwise, so avoid such false negatives.

* Cranelift: dynamic contexts: account for outgoing-args area.

---------

Co-authored-by: Alex Crichton <[email protected]>

show more ...

Classify more functions as trampolines in `objdump` (#10925)

Changes from wasip3-prototyping and peeling them off into this repo.

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtim

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtime to the 2024 Edition to keep
up-to-date with Rust idioms and access many of the edition features
exclusive to the 2024 edition.

prtest:full

* Reformat with the 2024 edition

show more ...

Apply argument extensions to component intrinsics too (#10540)

* Apply argument extensions to component intrinsics too

This commit fixes the definition of host signatures for component
builtins to

Apply argument extensions to component intrinsics too (#10540)

* Apply argument extensions to component intrinsics too

This commit fixes the definition of host signatures for component
builtins to work in the same way that core builtins do which is to use
platform-specific sign-extension-rules by default. This fixes a
regression found in the wasip3-prototyping repository where new
component intrinsics didn't have this set and subsequently were failing
in riscv64 CI.

* Add some tests

show more ...

Use a `clap` helper to reduce parsing in `objdump` (#10440)

TIL about `ValueEnum`.

Add `--stack-maps` to `wasmtime objdump` (#10439)

* Add `--stack-maps` to `wasmtime objdump`

Follow-up to #10404 and #10405

* Enable traps/stack maps by default in objdump

Add a `wasmtime objdump` subcommand (#10405)

This commit adds an `objdump` subcommand to the `wasmtime` CLI. Like all
other subcommands this can be disabled for a more minimal build of the
CLI as we

Add a `wasmtime objdump` subcommand (#10405)

This commit adds an `objdump` subcommand to the `wasmtime` CLI. Like all
other subcommands this can be disabled for a more minimal build of the
CLI as well. The purpose of this subcommand is to provide a
Wasmtime-specific spin on the venerable native `objdump` itself. Notably
this brings Wasmtime-specific knowledge for filtering functions, showing
Wasmtime metadata, etc.

This command is intended to look like `objdump` roughly but also has
configurable output with various flags and things that can be printed.
For now the main Wasmtime additions are showing the address map
section, stack map section, and trap section of a `*.cwasm` file.

This new subcommand replaces the infrastructure of the `disas` test
suite, and now that test suite uses `wasmtime objdump` to generate test
expectations. Additionally the subcommand replaces the Pulley `objdump`
example as a more full-featured objdump that also works natively with
Pulley.

The hope is that if we add more binary metadata in the future (such as
unwinding tables) that can be relatively easily added here for
exploration as well. Otherwise this is mostly just a developer
convenience for Wasmtime developers as well and hopefully doesn't cost
too much in maintenance burden.

Closes #10336

show more ...