Debugging: add debugger support for `wasmtime serve`. (#12859)

This adopts a simple solution to #12776: it takes the "instance reuse"
paradigm to the extreme, instantiating exactly one instance and

Debugging: add debugger support for `wasmtime serve`. (#12859)

This adopts a simple solution to #12776: it takes the "instance reuse"
paradigm to the extreme, instantiating exactly one instance and
serializing all requests into that one instance. This allows the
debugger component to operate on one `Store`, setting breakpoint state
and presenting its execution to the attached debugger as a single
program execution and minimizing impedance mismatches.

This also adds an integration test that runs an existing wasi-http
test component under the debugger.

show more ...

Several fixes to debugging infrastructure: component vs. module PCs and gdbstub wasm module names. (#12901)

* Debugging: fix module-relative vs component-relative PCs and unique library names.

Two

Several fixes to debugging infrastructure: component vs. module PCs and gdbstub wasm module names. (#12901)

* Debugging: fix module-relative vs component-relative PCs and unique library names.

Two bugfixes for guest debugging with components:

1. Convert component-relative source locations to module-relative PCs
in the frame table. The guest-debug API presents a core-Wasm view
where components are deconstructed into individual modules, so all
PCs must be module-relative. This adds a `wasm_module_offset` field
to `ModuleTranslation` and `FuncEnvironment`, set during component
translation, and subtracts it in `debug_tags()`.

2. Give unique names to "library" entries in the gdbstub XML response.
LLDB's DynamicLoader deduplicates by name, so using "wasm" for all
modules caused only the first to be loaded.

* Debugging: add ModulePC and ComponentPC newtypes for Wasm PC offsets.

Introduce `ModulePC` (module-relative) and `ComponentPC`
(component-relative) newtype wrappers around u32 Wasm bytecode
offsets. These replace raw u32 values throughout the frame table,
breakpoint, and debug systems to prevent confusion between the two
offset spaces.

* Debugging: add regression test for component module-relative PCs.

show more ...

Handle OOM in the rest of Wasmtime's non-component, -async, -compilation APIs (#12858)

* Handle OOM in more places in the public API

A bunch of random places:

* Add: `Trap::try_new` to handle OOM

Handle OOM in the rest of Wasmtime's non-component, -async, -compilation APIs (#12858)

* Handle OOM in more places in the public API

A bunch of random places:

* Add: `Trap::try_new` to handle OOM while creating traps
* Use: `TryVec` inside `Func::call_impl_do_call` and `wasm_val_raw_storage` to
hold the args and rets
* Add: `Instance::try_exports` for iterating over an instance's exports while
handling OOM
* `Linker:try_get`, like `Linker::get` but handling OOM
* `Linker:try_get_by_import`, like `Linker::get_by_import` but handling OOM
* Use `try_new` to box things in `SharedMemory::new`
* Use `TryVec` instead of `Vec` in our dynamic tables

* Add OOM tests for most of Wasmtime's public API

Excludes component-, async-, and compilation-related APIs.

* address review feedback

* fix test compilation

* fix c-api

show more ...

fix: correct various typos (#12807)

Signed-off-by: Ho Kim <[email protected]>

Debugging: allow breakpoints to be set at "function start" by slipping forward to first opcode. (#12791)

LLDB, when instructed to `break main`, looks at the DWARF metadata for
`main` and finds its P

Debugging: allow breakpoints to be set at "function start" by slipping forward to first opcode. (#12791)

LLDB, when instructed to `break main`, looks at the DWARF metadata for
`main` and finds its PC range, then sets a breakpoint at the first
PC. This is reasonable behavior for native ISAs! That PC better be a
real instruction!

On Wasm, however, (i) toolchains typically emit the PC range as
*including* the *locals count*, a leb128 value that precedes the first
opcode and any types of locals; (ii) our gdbstub component that
bridges LLDB to our debug APIs (#12771) only supports *exact* PCs for
breakpoints, so when presented with a PC that does not actually point
to an opcode, setting the breakpoint is effectively a no-op. There
will always be a difference of at least 1 byte between the
start-of-function offset and first-opcode offset (for a leb128 of `0`
for no locals), so a breakpoint "on" a function will never work.

I initially prototyped a fix that adds a sequence point at the start
of every function (which, again, is *guaranteed* to be distinct from
the first opcode), and the branch is [here], but I didn't like the
developer experience: this meant that when a breakpoint at a function
start fired, LLDB had a weird interstitial state where no line-number
applied.

The behavior that would be closer in line with "native" debug
expectations is that we add a bit of fuzzy-ish matching: setting a
breakpoint at function start should break at the first opcode, even if
that's a few (or many) bytes later. There are two options here:
special-case function start, or generally change the semantics of our
breakpoint API so that "add breakpoint at `pc`" means "add breakpoint
at next opcode at or after `pc`". I opted for the latter in this PR
because it's more consistent.

The logic is a little subtle because we're effectively defining an
n-to-1 mapping with this "snap-to-next" behavior, so we have to
refcount each breakpoint (consider setting a breakpoint at function
start *and* at the first opcode, then deleting them, one at a time). I
believe the result is self-consistent, even if a little more
complicated now. And, importantly, with #12771 on top of this change,
it produces the expected behavior for the (very simple!) debug script
"`b main`; `continue`".

[here]: https://github.com/cfallin/wasmtime/tree/breakpoint-at-func-start

show more ...

Debugging: apply single-stepping patches to modules instantiated after setting changes. (#12663)

We currently do a store-wide state-change on all registered modules
when the "single stepping" flag c

Debugging: apply single-stepping patches to modules instantiated after setting changes. (#12663)

We currently do a store-wide state-change on all registered modules
when the "single stepping" flag changes, patching in or out all
breakpoints. However, this design didn't account for modules
registered with the store *after* single-stepping is enabled (and new
modules may be registered any time an instantiation occurs). In
particular this is problematic when a debugger, e.g., sets the
single-step flag right at the beginning of execution of a "host main
function" that calls Wasm, before the main instantiation occurs.

This PR threads through the breakpoint state in the registration path,
with the narrow waist at `ModuleRegistry::register` which is the only
place where modules are added to the `LoadedCode`.

show more ...

Debugging: avoid re-patching code on single-step update when state doesn't change. (#12638)

When the setter for the single-step flag is given an `enable` boolean
that matches the existing state (i.e

Debugging: avoid re-patching code on single-step update when state doesn't change. (#12638)

When the setter for the single-step flag is given an `enable` boolean
that matches the existing state (i.e., enabling when already enabled or
disabling when already disabled), we should not loop through and
re-patch all breakpoint patch sites. This is an optimization, not
required for correctness, but a rather obvious optimization to make (and
important when building higher-level APIs for e.g. a single-step command
that always set the flag).

show more ...

Debugging: add `debug_all_modules()` / `debug_all_instances()` accessors on `Store`. (#12637)

* Debugging: add `debug_all_modules()` / `debug_all_instances()` accessors on `Store`.

When writing a d

Debugging: add `debug_all_modules()` / `debug_all_instances()` accessors on `Store`. (#12637)

* Debugging: add `debug_all_modules()` / `debug_all_instances()` accessors on `Store`.

When writing a debugger top-half using Wasmtime's debug APIs, it is
essential to be able to list all moules, and all instances, within a
`Store`. This PR adds those APIs.

The way in which the debug APIs are placed on
Store/StoreContextMut/Caller/etc is also refactored to be more uniform.

* ignore new test with compilation in miri

show more ...

miri: add guest-debugging, including frame accesses. (#12575)

* miri: add guest-debugging, including frame accesses.

The fix to `vm_store_context` provenance in `record_unwind`/`unwind`
is a little

miri: add guest-debugging, including frame accesses. (#12575)

* miri: add guest-debugging, including frame accesses.

The fix to `vm_store_context` provenance in `record_unwind`/`unwind`
is a little weird to me. I was seeing mut access to the
`vm_store_context` via `store.vm_store_context_mut()` in
`record_unwind` (before this diff) then access via the previously
saved raw pointer in the `CallThreadState`, which was registered as
invalid and I believe is indeed invalid. This was only manifesting
when setting `Config::guest_debug`, even without the frame-handle
accesses added here. I didn't dig into the exact diff in codegen or
runtime behavior that caused this but in any case, accessing
`vm_store_context` via these two different paths (with one mut)
appears to be unsound in any case. The fix here is to set the unwind
state via the raw pointer in `CallThreadState` since that's the only
path that the subsequent `unwind` has access to.

Unrelated but useful: `ci/miri-provenance.test.sh` now accepts
`MIRI_RUST_VERSION=+nightly` or whatnot, which is nice for running
locally (I keep `stable` as my default toolchain).

* Revert MIRI_RUST_VERSION in the CI script and add note about `rustup run` instead.

* Add a bit more usage of debug API to Pulley provenance test.

* Switch to using `Store`-derived `VMStoreContext` where available and re-deriving the raw pointer in `CallThreadState`.

prtest:full

show more ...

Cranelift/Wasmtime/Pulley/Debugging: use little-endian mode to spill/reload vectors in guest-debugging slot and ABI clobbers. (#12585)

* Cranelift/Wasmtime/Pulley/Debugging: use little-endian mode t

Cranelift/Wasmtime/Pulley/Debugging: use little-endian mode to spill/reload vectors in guest-debugging slot and ABI clobbers. (#12585)

* Cranelift/Wasmtime/Pulley/Debugging: use little-endian mode to spill/reload vectors in guest-debugging slot and ABI clobbers.

When running Pulley on an s390x (or other big-endian) host, and enabling
guest-debugging instrumentation, a very strange confluence of events
occurs:

- Pulley uses "native endian" of the host by default for loads and
stores.
- Patchable calls to debug hooks use the `preserve_all` ABI, which
spills all registers in the trampoline adapter (callee in this ABI),
including vector registers.
- Saving vector-typed locals/operand stack values to the debugger state
slot also uses vector stores.
- All of these stores were thus big-endian on big-endian hosts.
- Pulley's bytecode only supports little-endian vector loads/stores.

We were thus hitting an assert in Pulley codegen (the Cranelift
backend) when encountering a `VStore` VCode instruction with a
big-endian mode.

This PR makes two changes that avoid this issue:

- The ABI code for Pulley is careful to specify little-endian mode
explicitly for any vector load/store.
- The debug instrumentation code is refactored to use little-endian
explicitly for vector types *only*.
- (Why not for all types? Because we GC-root GC ref values, and
these need to be provided to the collector as mutable storage
cells, so need to be in native endianness.)

Test will come as part of #12575 incorporating a
Pulley-with-guest-debugging test and running on s390x amongst our
platforms.

prtest:full

* Review feedback.

* Re-bless Cranelift tests (explicit `little` flags on `preserve_all` tests).

show more ...

Debugging: refactor stack frame cursor into frame handle abstraction. (#12566)

* Debugging: refactor stack frame cursor into frame handle abstraction.

This addresses some of the issues described #1

Debugging: refactor stack frame cursor into frame handle abstraction. (#12566)

* Debugging: refactor stack frame cursor into frame handle abstraction.

This addresses some of the issues described #12486: we need the ability
to keep a handle to a stack frame as long as execution is frozen, and
keep multiple of these handles around, alongside the `Store`, without
any handle directly holding a borrow of the store.

The frame handles work by means of an "execution version" scheme: the
idea is that whenever any execution resumes in a given store, all
handles to existing frames could be invalidated, but if no such
execution occurs, all handles should still be valid. A tuple of
(globally unique for process lifetime) store ID, and execution version
within that store, should be sufficient to uniquely identify any
frozen-stack period during execution. This accomplishes cheap handle
invalidation without the need to track existing handles.

This PR also implements a cache of parsed frame-table data. Previously
this was lazily parsed by the cursor as it walked up a stack, but with
multiple handles hanging around, and with handles meant to be cheap to
hold and clone, and with handles being invalidated eagerly, it makes
much more sense to persist this parsed metadata at the `Store` level.
(It cannot persist at the `Engine` level because PCs are local per
store.)

* Re-bless disas tests (offsets in VMStoreContext changed).

* Handle invalidation tests.

* Review comments, and make API return `Result`s rather than panic'ing on stale handles.

* Review feedback.

* Doc-comment link fix.

* Review feedback.

* cfg-gate Activation method to `debug` feature only.

* Fix unused-import warning in no-debug cfg.

* Fix doc link (again, after rename from latest feedback).

show more ...

Forbid rustdoc warnings in CI (#12420)

* Forbid rustdoc warnings in CI

This commit corrects our handling of rustdoc flags in CI to ensure that
warnings indeed fire. Additionally this changes our fl

Forbid rustdoc warnings in CI (#12420)

* Forbid rustdoc warnings in CI

This commit corrects our handling of rustdoc flags in CI to ensure that
warnings indeed fire. Additionally this changes our flags to pass
`-Dwarnings` to ensure that we have warning-free doc builds when all
features are enabled at least.

There were quite a lot of preexisting issues to fix, so this
additionally goes through and fixes all the warnings that cropped up.

* Update nightly toolchain again

prtest:full

* Update another nightly

* Fix a warning in generated code

show more ...

Reduce duplication amongst `debug_*` helpers (#12386)

Deduplicate the check for `guest_debug`, validity checks, and conversion
from `Export` to `Extern`. No functional change here, just a
refactorin

Reduce duplication amongst `debug_*` helpers (#12386)

Deduplicate the check for `guest_debug`, validity checks, and conversion
from `Export` to `Extern`. No functional change here, just a
refactoring.

show more ...

Debugging: provide access to private (non-exported) entities. (#12367)

* Debugging: provide access to private (non-exported) entities.

A debugger will need to access all entities (globals, tables,

Debugging: provide access to private (non-exported) entities. (#12367)

* Debugging: provide access to private (non-exported) entities.

A debugger will need to access all entities (globals, tables, memories),
even those that are not exported, in order to provide a full debugging
experience: for example, a developer who has a debugger attached to a
Wasm component will expect to be able to see data in its memory.

Historically we have been very careful in Wasmtime to provide access to
Wasm instances' entities only as the Wasm type system allows -- that is,
only if they are exported. However, debugging is privileged -- in the
same way that a native host debugger has `ptrace` and can view
everything about the debuggee, we need to provide APIs for seeing
through the encapsulation boundary.

To ensure that this "violation of encapsulation" is scoped only to the
extent needed for the legitimate need (debugging), this API is
dynamically available only when `guest_debug` is configured true for a
given engine. Otherwise, the accessor returns `None`.

I opted not to provide a full introspection API that enumerates all of
the entities as the debugger should already have access to the debuggee
module and be able to enumerate the entities. Thus, the API only
provides a host-API handle when asking for an entity by index in a given
instance's index space.

* Review feedback.

* Fix tests on 32-bit build (where threads and thus shared memory are not supported)

show more ...

Migrate the `wasmtime` crate to `wasmtime_environ::error::*` (#12231)

* Migrate the `wasmtime` crate to `wasmtime_environ::error::*`

Instead of `anyhow::Error`.

This commit re-exports the `wasmtim

Migrate the `wasmtime` crate to `wasmtime_environ::error::*` (#12231)

* Migrate the `wasmtime` crate to `wasmtime_environ::error::*`

Instead of `anyhow::Error`.

This commit re-exports the `wasmtime_environ::error` as the `wasmtime::error`
module, updates the prelude to include these new error-handling types, redirects
our top-level `wasmtime::{Error, Result}` re-exports to re-export
`wasmtime::error::{Error, Result}`, and updates various use sites that were
directly using `anyhow` to use the new `wasmtime` versions.

This process also required updating the component macro and wit-bindgen macro to
use the new error types instead of `anyhow`.

Part of https://github.com/bytecodealliance/wasmtime/issues/12069

* Replace wasmtime::error::Thing with wasmtime::Thing where it makes sense

* cargo fmt

* Move `crate::error::Thing` to `crate::Thing` where it makes sense

show more ...

Debug: clarify safety around ActivationBacktrace::new. (#12191)

* Debug: clarify safety around ActivationBacktrace::new.

As per [this discussion], we decided that we would handle potential
frame-cu

Debug: clarify safety around ActivationBacktrace::new. (#12191)

* Debug: clarify safety around ActivationBacktrace::new.

As per [this discussion], we decided that we would handle potential
frame-cursor invalidation unsafety (with a hypothetical future
frame-editing debugger API) by putting `unsafe` on that future
hypothetical API rather than on the cursor construction. With the APIs
available today on the `Store`, there is no way to invalidate the frame
cursor while within its lifetime-bounded scope (with lifetime tied to
the `Store` that is passed into the hostcall creating the cursor), so
the API today should be completely safe. This PR makes it so.

[this discussion]: https://github.com/bytecodealliance/wasmtime/pull/12176#discussion_r2636431864

* store_mut is safe as well.

show more ...

Debugging: add event for epoch yields. (#12194)

This will almost certainly be needed to implement "Ctrl-C" behavior for
a gdbstub server or other user-facing debugger top-half, so the user can
inter

Debugging: add event for epoch yields. (#12194)

This will almost certainly be needed to implement "Ctrl-C" behavior for
a gdbstub server or other user-facing debugger top-half, so the user can
interrupt the debuggee and return control to the debugger.

show more ...

Debugging: allow frame cursor to visit all activations. (#12176)

* Debugging: allow frame cursor to visit all activations.

In the initial design for the `DebugFrameCursor`, I was concerned
about th

Debugging: allow frame cursor to visit all activations. (#12176)

* Debugging: allow frame cursor to visit all activations.

In the initial design for the `DebugFrameCursor`, I was concerned
about the effects of host async on the stability of visiting earlier
activations (see also the discussion of async combinators in #11896).
The basic hypothesized problem was that when Wasm calls host-code
calls Wasm, the sequence of activations on the stack is not even
stable between async polls; so any debugger hook, which is an async
function, should not be allowed to hold a frame cursor across a yield
point since it could become invalidated if the next poll stacks up the
activations differently.

In further conversations it's become clear that this is not actually a
possibility, for the simple reason that the inner re-entrant
activations into the same store take full ownership (mutably reborrow)
that store, and that mut reborrow becomes part of the future; so the
exact chain of activations will remain in the same sequence when
re-polled. Said another way, it is impossible at any given level of
async host-code to create *more than one* future that re-enters the
same store and somehow poll those in different orders at different
times. The worst that a host-code async combinator can do is drop the
future that re-enters the store. This drops and invalidates whatever
frames a cursor held over a yield might be referencing, but it *also*
drops the async invocation of the debugger hook itself, and due to
lifetimes the cursor cannot escape that hook, so everything is still
sound.

This PR thus updates the `DebugFrameCursor` to visit all
activations. I've generalized the backtrace code a bit to enable this,
and built an internal `StoreBacktrace` that is an iterator over all
activations associated with the store.

At the `DebugFrameCursor` (public API) level, the two basic choices
were to present a sentinel for host frame(s) explicitly and make all
Wasm-specific accessors return `Option<T>`, or skip over host
frames. I opted for the latter, with `move_to_parent()` returning an
enum value now that indicates whether it moved to a new activation.

A note regarding the *async* component ABI: once debugging is possible
within a `run_concurrent` environment, it will again be the case that
a single frame cursor should see only one activation, because
each (re)-entry into the store becomes a new task, if my understanding
is correct. At that time, we should build an API that lets the
debugger see the activation for each task separately. That's a simpler
model ultimately, and it will be nice when we move to it, but as long
as we have the sync component ABI with async host code and the ability
to stack activations as we do today, we need to provide the debugger
this visibility.

(Aside: why does the debugger *need* to see more than one activation?
In addition to presenting a weird and incoherent view of the world to
the user if we don't, it is also necessary to implement the
"next" (step-over) debugger action, because otherwise a call to a host
function that re-enters the store may lead to a state with fewer, but
completely disjoint, stack frames on the "one latest activation" from
which it's not possible to reason about whether we've left the
called-into function yet.)

* Review feedback.

* cargo fmt.

show more ...

Debug: implement breakpoints and single-stepping. (#12133)

* Debug: implement breakpoints and single-stepping.

This is a PR that puts together a bunch of earlier pieces (patchable
calls in #12061 a

Debug: implement breakpoints and single-stepping. (#12133)

* Debug: implement breakpoints and single-stepping.

This is a PR that puts together a bunch of earlier pieces (patchable
calls in #12061 and #12101, private copies of code in #12051, and all
the prior debug event and instrumentation infrastructure) to implement
breakpoints in the guest debugger.

These are implemented in the way we have planned in #11964: each
sequence point (location prior to a Wasm opcode) is now a patchable call
instruction, patched out (replaced with NOPs) by default. When patched
in, the breakpoint callsite calls a trampoline with the `patchable` ABI
which then invokes the `breakpoint` hostcall. That hostcall emits the
debug event and nothing else.

A few of the interesting bits in this PR include:
- Implementations of "unpublish" (switch permissions back to read/write
from read/execute) for mmap'd code memory on all our platforms.
- Infrastructure in the frame-tables (debug info) metadata producer and
parser to record "breakpoint patches".
- A tweak to the NOP metadata packaged with the `MachBuffer` to allow
multiple NOP sizes. This lets us use one 5-byte NOP on x86-64, for
example (did you know x86-64 had these?!) rather than five 1-byte
NOPs.

This PR also implements single-stepping with a global-per-`Store` flag,
because at this point why not; it's a small additional bit of logic to
do *all* patches in all modules registered in the `Store` when that flag
is enabled.

A few realizations for future work:
- The need for an introspection API available to a debugger to see the
modules within a component is starting to become clear; either that,
or the "module and PC" location identifier for a breakpoint switches
to a "module or component" sum type. Right now, the tests for this
feature use only core modules. Extending to components should not
actually be hard at all, we just need to build the API for it.
- The interaction between inlining and `patchable_call` is interesting:
what happens if we inline a `patchable_call` at a `try_call` callsite?
Right now, we do *not* update the `patchable_call` to a `try_call`,
because there is no `patchable_try_call`; this is fine in the Wasmtime
embedding in practice because we never (today!) throw exceptions from
a breakpoint handler. This does suggest to me that maybe we should
make patchability a property of any callsite, and allow try-calls to
be patchable too (with the same restriction about no return values as
the only restriction); but happy to discuss that one further.

* Add missing debug.wat disas test.

* Review feedback.

* Fix comment on `CodeMemory::text_mut`.

* Review feedback.

* Review feedback: abort process on failure to re-apply executable permissions.

* Implement icache flush for aarch64.

This appears to be necessary as we otherwise see a failure in CI on
macOS/aarch64 that is consistent with patched-in breakpoint calls still
being incorrectly cached after we remove them and republish the code.

There is a longstanding issue in #3310 tracking proper icache coherence
handling on aarch64. We implemented this for Linux with the `membarrier`
syscall but never did so for macOS. Maybe this is the first point at
which it matters, because code was always loaded at new addresses (hence
did not have coherence issues because nothing would have been cached)
previously.

prtest:full

* Review feedback: use `next_multiple_of`.

show more ...

Debug: create private code memories per store when debugging is enabled. (#12051)

* Debug: create private code memories per store when debugging is enabled.

This will allow patching code to implem

Debug: create private code memories per store when debugging is enabled. (#12051)

* Debug: create private code memories per store when debugging is enabled.

This will allow patching code to implement e.g. breakpoints. (That is,
for now the copies are redundant, but soon they will not be.)

This change follows the discussion [here] and offline to define a few
types that better encapsulate the distinction we want to enforce.
Basically, there is almost never a bare `CodeMemory`; they are always
wrapped in an `EngineCode` or `StoreCode`, the latter being a per-store
instance of the former. Accessors are moved to the relevant place so
that, for example, one cannot get a pointer to a Wasm function's body
without being in the context of a `Store` where the containing module
has been registered. The registry then returns a `ModuleWithCode` that
boxes up a `Module` reference and `StoreCode` together for cases where
we need both the metadata from the module and the raw code to derive
something.

The only case where we return raw code pointers to the `EngineCode`
directly have to do with Wasm-to-array trampolines: in some cases, e.g.
`InstancePre` pre-creating data structures with references to host
functions, it breaks our expected performance characteristics to make
the function pointers store-specific. This is fine as long as the
Wasm-to-array trampolines never bake in direct calls to Wasm functions;
the strong invariant is that Wasm functions never execute from
`EngineCode` directly. Some parts of the component runtime would also
have to be substantially refactored if we wanted to do away with this
exception.

The per-`Store` module registry is substantially refactored in this PR.
I got rid of the modules-without-code distinction (the case where a
module only has trampolines and no defined functions still works fine),
and organized the BTreeMaps to key on start address rather than end
address, which I find a little more intuitive (one then queries with the
dual to the range -- 0-up-to-PC and last entry found).

[here]: https://github.com/bytecodealliance/wasmtime/pull/12051#pullrequestreview-3493711812

* Review feedback: do not assume a reasonable code alignment; error when it cannot be known

* Review feedback: fail properly in profiler when we are cloning code

* Fix guest-profiler C API.

* Review feedback: make private-code representation impossible in non-debugging-support builds.

* Add TODO comment referencing issue for cloning only .text.

* clang-format

* Review feedback: add back Component::image_range.

* Review feedback: error on registering profiling metadata when debug is enabled.

* rustfmt

* Remove early bail on profiling-data registration when debugging is enabled: this always happens so we cannot error out.

show more ...

Debugging: add a debugger callback mechanism to handle debug events. (#11895)

* Debugging: add a debugger callback mechanism to handle debug events.

This PR adds a notion of "debug events", and a m

Debugging: add a debugger callback mechanism to handle debug events. (#11895)

* Debugging: add a debugger callback mechanism to handle debug events.

This PR adds a notion of "debug events", and a mechanism in Wasmtime to
associate a "debug handler" with a store such that the handler is
invoked as-if it were an async hostcall on each event. The async
handler owns the store while its future exists, so the whole "world"
(within the store) is frozen and the handler can examine any state it
likes with a `StoreContextMut`.

Note that this callback-based scheme is a compromise: eventually, we
would like to have a native async API that produces a stream of events,
as sketched in #11826 and in [this branch]. However, the async approach
implemented naively (that is, with manual fiber suspends and with state
passed on the store) suffers from unsoundness in the presence of dropped
futures. Alex, Nick and I discussed this extensively and agreed that the
`Accessor` mechanism is the right way to allow for a debugger to have
"timesliced"/"shared" access to a store (only when polled/when an event
is delivered), but we will defer that for now, because it requires
additional work (mainly, converting existing async yield points in the
runtime to "give up" the store with the `run_concurrent` mechanism).
I'll file a followup issue to track that. The idea is that we can
eventually build that when ready, but the API we provide to a debugger
component can remain unchanged; only this plumbing and the glue to the
debugger component will be reworked.

With this scheme based on callbacks, we expect that one should be able
to implement a debugger using async channels to communicate with the
callback. The idea is that there would be a protocol where the callback
sends a debug event to the debugger main loop elsewhere in the executor
(e.g., over a Tokio channel or other async channel mechanism), and when
the debugger wants to allow execution to continue, it sends a "continue"
message back. In the meantime, while the world is paused, the debugger
can send messages to the callback to query the `StoreContextMut` it has
and read out state. This indirection/proxying of Store access is
necessary for soundness: again, teleporting the Store out may look like
it almost works ("it is like a mutable reborrow on a hostcall") except
in the presence of dropped futures with sandwiched Wasm->host->Wasm
situations.

This PR implements debug events for a few cases that can be caught
directly in the runtime, e.g., exceptions and traps raised just before
re-entry to Wasm. Other kinds of traps, such as those normally
implemented by host signals, require additional work (as in #11826) to
implement "hostcall injection" on signal reception; and breakpoints will
be built on top of that. The point of this PR is only to get the initial
plumbing in place for events.

[this branch]: https://github.com/cfallin/wasmtime/tree/wasmtime-debug-async

* Add some more tests.

* Review feedback: comment updates, and make `debug` feature depend on `async`.

* Review feedback: debug-hook setter requires guest debugging to be enabled.

* Review feedback: ThrownException event; handle block_on errors; explicitly list UnwindState cases.

* Add comment about load-bearing Send requirement.

* Fix no-unwind build.

* Review feedback: pass in hostcall error messages while keeping the trait object-safe.

Co-authored-by: Alex Crichton <[email protected]>

* Ignore divide-trapping test on Pulley for now.

---------

Co-authored-by: Alex Crichton <[email protected]>

show more ...

Debug API: switch `DebugFrameCursor` over to monomorphization on T, and provide `AsContextMut`. (#11873)

* Debug API: switch `DebugFrameCursor` over to monomorphization on T, and provide `AsContextM

Debug API: switch `DebugFrameCursor` over to monomorphization on T, and provide `AsContextMut`. (#11873)

* Debug API: switch `DebugFrameCursor` over to monomorphization on T, and provide `AsContextMut`.

This permits use of ordinary store accessors, such as GC object
accessors, while viewing the stack. In #11835 and related discussion,
we've concluded that (i) it is safe to provide this mutable access,
including e.g. ability to do a GC, because debugger access comes only
at points that are morally like hostcalls (right now, literally with a
`Caller`, and eventually with debug yields); and (ii) we will need
this in order to provide the full range of expected debugger
functionality. Even before mutation-during-debugging comes into play,
with the API before this change, it was not possible to read GC
objects at all (because all accessors take a mutable store).

* Review feedback.

* Fix some disable-feature builds.

show more ...

Wasmtime: implement debug instrumentation and basic host API to examine runtime state. (#11769)

* Wasmtime: implement debug instrumentation and basic host API to examine runtime state.

This PR impl

Wasmtime: implement debug instrumentation and basic host API to examine runtime state. (#11769)

* Wasmtime: implement debug instrumentation and basic host API to examine runtime state.

This PR implements ideas from the [recent RFC] to serve as the basis
for Wasm (guest) debugging: it adds a stackslot to each function
translated from Wasm, stores to replicate Wasm VM state in the
stackslot as the program runs, and metadata to describe the format of
that state and allow reading it out at runtime.

As an initial user of this state, this PR adds a basic "stack view"
API that, from host code that has been called from Wasm, can examine
Wasm frames currently on the stack and read out all of their locals
and stack slots.

Note in particular that this PR does not include breakpoints,
watchpoints, stepped execution, or any sort of user interface for any
of this; it is only a foundation.

This PR still has a few unsatisfying bits that I intend to address:

- The "stack view" performs some O(n) work when the view is initially
taken, computing some internal data per frame. This is forced by the
current design of `Backtrace`, which takes a closure and walks that
closure over stack frames eagerly (rather than work as an
iterator). It's got some impressive iterator-chain stuff going on
internally, so refactoring it to the latter approach might not
be *too* bad, but I haven't tackled it yet.

A O(1) stack view, that is, one that does work only for frames as
the host API is used to walk up the stack, is desirable because some
use-cases may want to quickly examine e.g. only the deepest
frame (say, running with a breakpoint condition that needs to read a
particular local's value after each step).

- It includes a new `Config::compiler_force_inlining()` option that is
used only for testing that we get the correct frames after
inlining. I couldn't get the existing flags to work on a Wasmtime
config level and suspect there may be an existing bug there; I will
try to split out a fix for it.

This PR renames the existing `debug` option to `native_debug`, to
distinguish it from the new approach.

[recent RFC]: https://github.com/bytecodealliance/rfcs/pull/44

* Update to new APIs on Cranelift side.

* Test update.

* Adjust objdump printing of InstPos on frame progpoints; and adjust progpoint collapsing.

* Convert to iterator form.

* Fix path in native-debug tests (debug -> native_debug rename).

* Enforce that `debug_instrumentation` can only be enabled when feature is enabled.

* Add missing assert.

* Use builtin knob for forcing intra-module inlining instead.

* Review feedback:

- Make StackView own the current frame rather than handing it out. This
prevents the current frame (`FrameView`) from walking away and hiding
somewhere it shouldn't, to be used unsoundly later.
- Assert no-GC during stack walk.

* Merge debug-instrumentation hooks on FuncEnvironment into before/after hooks.

* Review feedback: avoid downcasting funcs twice.

* Add debug feature to `wasmtime` crate's defaults.

* Review feedback: u32s for local and stack indices in debug host API.

* Use *const u8 as stack pointers and `with_exposed_provenance` in debug API.

* Remove some `srcloc` plumbing in Wasm translator.

* Rename native-debug back to debug, and make the new thing "guest debugging".

* rustfmt in debugging test.

* fix disas test after guest-debug CLI option rename.

* Review feedback: no separate debug-instrumentation hooks on FuncEnvironment.

* Review feedback: update doc comment on `Config::guest_debug`.

* Review feedback: rename `generate_debuginfo` to `debug_native` in tunables.

* Review feedback: miscellaneous comments.

* Review comment: fix wording in safety conditions.

* revert wasi-common submodule update

* Properly root values in debug frame slots.

Fixes #11841.

* Fix non-`debug`-feature build.

* Review feedback: naming.

* Ignore tests that compile modules in miri.

show more ...

Another batch of dependency updates (#11832)

* Another batch of dependency updates

Bringing some deps in `Cargo.toml` up-to-date with their latest versions
along the same lines as #11820 to avoid d

Another batch of dependency updates (#11832)

* Another batch of dependency updates

Bringing some deps in `Cargo.toml` up-to-date with their latest versions
along the same lines as #11820 to avoid deps getting too stale/old.

Code-wise this updates `anyhow` which enables preexisting Clippy
warnings to check more code, so those warnings are fixed here as well.

prtest:full

* Run rustfmt

show more ...

Rely on `core::error::Error` (#9702)

* Rely on `core::error::Error`

With Wasmtime's new MSRV at 1.81 this means that `core::error::Error` is
available which means that in `no_std` mode the `Error`

Rely on `core::error::Error` (#9702)

* Rely on `core::error::Error`

With Wasmtime's new MSRV at 1.81 this means that `core::error::Error` is
available which means that in `no_std` mode the `Error` trait can be
used. This has been integrated into `anyhow::Error` already upstream and
means that we can remove our own local hacks such as the `Err2Anyhow` trait.

This commit removes the `Err2Anyhow` trait and all usage, going back to
idiomatic Rust error propagation and conversion even in the `no_std`
world. This should make code more portable by default and remove some
weird idioms we had for supporting this.

prtest:full

* Add some trusted vets

* Audit object crate update

* Disable backtraces on CI

show more ...