sys/miri: fix `Mmap::from_file` to seek to start of file. (#12572)

The constructor takes a `&File` but `&File` still allows mutation of
the underlying state, e.g., the seek position. It turns out th

sys/miri: fix `Mmap::from_file` to seek to start of file. (#12572)

The constructor takes a `&File` but `&File` still allows mutation of
the underlying state, e.g., the seek position. It turns out that this
matters when we try to `Mmap::from_file` more than once, e.g. when
enabling guest debugging and thus mapping private copies of code.

No test yet as this will be covered by upcoming miri coverage of guest
debugging.

show more ...

Debug: implement breakpoints and single-stepping. (#12133)

* Debug: implement breakpoints and single-stepping.

This is a PR that puts together a bunch of earlier pieces (patchable
calls in #12061 a

Debug: implement breakpoints and single-stepping. (#12133)

* Debug: implement breakpoints and single-stepping.

This is a PR that puts together a bunch of earlier pieces (patchable
calls in #12061 and #12101, private copies of code in #12051, and all
the prior debug event and instrumentation infrastructure) to implement
breakpoints in the guest debugger.

These are implemented in the way we have planned in #11964: each
sequence point (location prior to a Wasm opcode) is now a patchable call
instruction, patched out (replaced with NOPs) by default. When patched
in, the breakpoint callsite calls a trampoline with the `patchable` ABI
which then invokes the `breakpoint` hostcall. That hostcall emits the
debug event and nothing else.

A few of the interesting bits in this PR include:
- Implementations of "unpublish" (switch permissions back to read/write
from read/execute) for mmap'd code memory on all our platforms.
- Infrastructure in the frame-tables (debug info) metadata producer and
parser to record "breakpoint patches".
- A tweak to the NOP metadata packaged with the `MachBuffer` to allow
multiple NOP sizes. This lets us use one 5-byte NOP on x86-64, for
example (did you know x86-64 had these?!) rather than five 1-byte
NOPs.

This PR also implements single-stepping with a global-per-`Store` flag,
because at this point why not; it's a small additional bit of logic to
do *all* patches in all modules registered in the `Store` when that flag
is enabled.

A few realizations for future work:
- The need for an introspection API available to a debugger to see the
modules within a component is starting to become clear; either that,
or the "module and PC" location identifier for a breakpoint switches
to a "module or component" sum type. Right now, the tests for this
feature use only core modules. Extending to components should not
actually be hard at all, we just need to build the API for it.
- The interaction between inlining and `patchable_call` is interesting:
what happens if we inline a `patchable_call` at a `try_call` callsite?
Right now, we do *not* update the `patchable_call` to a `try_call`,
because there is no `patchable_try_call`; this is fine in the Wasmtime
embedding in practice because we never (today!) throw exceptions from
a breakpoint handler. This does suggest to me that maybe we should
make patchability a property of any callsite, and allow try-calls to
be patchable too (with the same restriction about no return values as
the only restriction); but happy to discuss that one further.

* Add missing debug.wat disas test.

* Review feedback.

* Fix comment on `CodeMemory::text_mut`.

* Review feedback.

* Review feedback: abort process on failure to re-apply executable permissions.

* Implement icache flush for aarch64.

This appears to be necessary as we otherwise see a failure in CI on
macOS/aarch64 that is consistent with patched-in breakpoint calls still
being incorrectly cached after we remove them and republish the code.

There is a longstanding issue in #3310 tracking proper icache coherence
handling on aarch64. We implemented this for Linux with the `membarrier`
syscall but never did so for macOS. Maybe this is the first point at
which it matters, because code was always loaded at new addresses (hence
did not have coherence issues because nothing would have been cached)
previously.

prtest:full

* Review feedback: use `next_multiple_of`.

show more ...

Replace setjmp/longjmp usage in Wasmtime (#11592)

Since Wasmtime's inception it's used the `setjmp` and `longjmp`
functions in C to implement handling of traps. While this solution was
easy to imple

Replace setjmp/longjmp usage in Wasmtime (#11592)

Since Wasmtime's inception it's used the `setjmp` and `longjmp`
functions in C to implement handling of traps. While this solution was
easy to implement, relatively portable, and performant enough, there are
a number of downsides that have evolved over time to make this an
unattractive approach in the long run:

* Using `setjmp` fundamentally requires using C because Rust does not
understand a function that returns twice. It's fundamentally unsound
to invoke `setjmp` in Rust meaning that Wasmtime has forever needed a
C compiler configured and set up to build. This notably means that
`cargo check` cannot check other targets easily.

* Using `longjmp` means that Rust function frames are unwound on the
stack without running destructors. This is a dangerous operation of
which we get no protection from the compiler about. Both frames
entering wasm and frames exiting wasm are all skipped. Absolutely
minimizing this has been beneficial for portability to platforms such
as Pulley.

* Currently the no_std implementation of Wasmtime requires embedders to
provide `wasmtime_{setjmp,longjmp}` which is a thorn in the side of
what is otherwise a mostly entirely independent implementation of
Wasmtime.

* There is a performance floor to using `setjmp` and `longjmp`. Calling
`setjmp` requires using C but Wasmtime is otherwise written in Rust
meaning that there's a Rust->C->Rust->Wasm boundary which
fundamentally can't be inlined without cross-language LTO which is
difficult to configure.

* With the implementation of the WebAssembly exceptions proposal
Wasmtime now has two means of unwinding the stack. Ideally Wasmtime
would only have one, and the more general one is the method of
exceptions.

* Jumping out of a signal handler on Unix is tricky business. While
we've made it work it's generally most robust of the signal handler
simply returns which it now does.

With all of that in mind the purpose of this commit is to replace the
setjmp/longjmp mechanism of handling traps with the recently implemented
support for exceptions in Cranelift. That is intended to resolve all of
the above points in one swoop.

One point in particular though that's nice about setjmp/longjmp is that
unwinding the stack on a trap is an O(1) operation. For situations such
as stack overflow that's a particularly nice property to have as we can
guarantee embedders that traps are a constant time (albeit somewhat
expensive with signals) operation. Exceptions naively require unwinding
the entire stack, and although frame pointers mean we're just traversing
a linked list I wanted to preserve the O(1) property here nonetheless.
To achieve this a solution is implemented where the array-to-wasm
(host-to-wasm) trampolines setup state in `VMStoreContext` so looking up
the current trap handler frame is an O(1) operation. Namely the sp/fp/pc
values for a `Handler` are stored inline.

Implementing this feature required supporting
relocations-to-offsets-in-functions which was not previously supported
by Wasmtime. This required Cranelift refactorings such as #11570, #11585,
and #11576. This then additionally required some more refactoring in
this commit which was difficult to split out as it otherwise wouldn't be
tested.

Apart from the relocation-related business much of this change is about
updating the platform signal handlers to use exceptions instead of
longjmp to return. For example on Unix this means updating the
`ucontext_t` with register values that the handler specifies. Windows
involves updating similar contexts, and macOS mach ports ended up not
needing too many changes.

In terms of overall performance the relevant benchmark from this
repository, compared to before this commit, is:

sync/no-hook/core - host-to-wasm - typed - nop
time: [10.552 ns 10.561 ns 10.571 ns]
change: [−7.5238% −7.4011% −7.2786%] (p = 0.00 < 0.05)
Performance has improved.

Closes #3927
cc #10923

prtest:full

show more ...

Add support for the Linux PAGEMAP_SCAN ioctl (#11372)

* WIP: use the pagemap_scan ioctl to selectively reset an instance's dirty pages

* Less hacky, and supporting tables, too

* Bugfixes

* WIP: m

Add support for the Linux PAGEMAP_SCAN ioctl (#11372)

* WIP: use the pagemap_scan ioctl to selectively reset an instance's dirty pages

* Less hacky, and supporting tables, too

* Bugfixes

* WIP: memcpy instead of pread

* Refactor support for pagemap

* Don't hold a raw pointer to the original data, plumb through an `Arc`
to have a safe reference instead.
* Update pagemap bindings to latest version of abstraction written.
* Include pagemap-specific tests.
* Use a `PageMap` structure created once-per-pool instead of a
static-with-a-file.
* Refactor to use the "pagemap path" unconditionally which blends in the
keep_resident bits.

* Improve safety documentation

prtest:full

* Fix some lints

* Skip ioctl tests when it's not supported

* Fix a memory leak by moving impls around

* Fix no vm build

* Review comments

* Add more pagemap-specific documentation

* Add more docs, refactor implementation slightly

* Improve `category_*` docs

Basically forward to the Linux kernel source itself.

* Fix compile

* Make pagemap integration resilient across forks

* Fix non-pooling-allocator-build

* Fix portability issues of new test

* Actually use config on macos

---------

Co-authored-by: Till Schneidereit <[email protected]>

show more ...

Deny `unsafe_op_in_unsafe_fn` in `wasmtime::runtime::vm` (#11312)

* Deny `unsafe_op_in_unsafe_fn` in `wasmtime::runtime::vm`

Slowly expanding this lint to more of the crate.

prtest:full

* Fix lin

Deny `unsafe_op_in_unsafe_fn` in `wasmtime::runtime::vm` (#11312)

* Deny `unsafe_op_in_unsafe_fn` in `wasmtime::runtime::vm`

Slowly expanding this lint to more of the crate.

prtest:full

* Fix lints in custom module

* Fix some lints with miri

* Fix non-VM build

* Fix arm windows

show more ...