Use `TryBTreeMap` in the store's `ModuleRegistry` (#12846)

Do not allocate for `wasmtime::Module::imports` iterator (#12682)

* Do not allocate for `wasmtime::Module::imports` iterator

* Revert "Do not allocate for `wasmtime::Module::imports` iterator"

Thi

Do not allocate for `wasmtime::Module::imports` iterator (#12682)

* Do not allocate for `wasmtime::Module::imports` iterator

* Revert "Do not allocate for `wasmtime::Module::imports` iterator"

This reverts commit d10da005f9105358fbe71b8610e89a67844b8726.

* review feedback

show more ...

Handle OOM in `Module::deserialize` (#12673)

Part of https://github.com/bytecodealliance/wasmtime/issues/12069

Debugging: add unique-within-store IDs for every entity. (#12645)

When building an introspection API for use by a debugger, we need a
way to expose *identity*: that is, to give some way of knowing t

Debugging: add unique-within-store IDs for every entity. (#12645)

When building an introspection API for use by a debugger, we need a
way to expose *identity*: that is, to give some way of knowing that a
given `Memory`, `Instance`, etc. is *this* one and not *that* one. Our
handle types variously have either `Eq` implementations or
e.g. `Module::same` for the ones that wrap an `Arc` under-the-covers;
but that's not enough to allow a debugger to e.g. build a hashmap for
whatever metadata that it might expose via whatever debugging
protocol.

For maximal generality and flexibility, we should expose the unique
IDs that already more-or-less exist: for instances, that is their
instance ID directly; for entities owned by instances, we can build a
`u64` with the instance ID in the upper 32 bits and the defined-index
in the lower 32 bits. IDs for all entities except modules are
unique-within-a-Store (and this is all that is needed); IDs for
modules happen to reuse the `CompiledModuleId` and so are
unique-within-an-Engine.

I've opted to name these `debug_index_within_store` to scope the
feature and intended use-case clearly, but if there's a desire, I
could easily rename them to simply `index`. I shied away from that
here because I didn't want to give a notion that these indices are
somehow canonical or correspond to some order or other.

show more ...

Debugging: preserve original Wasm bytecode inside of compiled ELF artifact. (#12636)

* Debugging: preserve original Wasm bytecode inside of compiled ELF artifact.

This PR adds logic to embed the or

Debugging: preserve original Wasm bytecode inside of compiled ELF artifact. (#12636)

* Debugging: preserve original Wasm bytecode inside of compiled ELF artifact.

This PR adds logic to embed the original core Wasm module(s) from a
compilation into a new ELF section, alongside other metadata
sections. When a component is compiled, the core Wasms inside are
preserved, accessible by their `StaticModuleIndex`es.

The need for this support arises from the guest-debugger
ecosystem. Consider either a debug
component (bytecodealliance/rfcs#45) or a bespoke debugger in native
code using Wasmtime's APIs. In either case, the existing APIs to
introspect execution state provide `Module` references for each
instance from each stack frame, and PC offsets into these `Module`s
are the way in which breakpoints are configured. The debugger will
somehow need to associate these `Module`s with the original Wasm
bytecode, including e.g. any custom sections containing the
producer-specific ways of encoding debug metadata, to do something
useful. In particular also note that the GDB-stub protocol as extended
for Wasm requires read access directly to the Wasm bytecode (it shows
up as part of a "memory map" that is viewed by the standard
read-remote-memory command); we can't delegate this requirement to the
remote end of the stub connection, but have to handle it in the stub
server that runs inside Wasmtime (as a component or bespoke).

We have two main choices: carry the original bytecode all the way
through the Wasmtime compilation pipeline and present it via
`Module::bytecode()`, ready to use; or say that this task is
out-of-scope and that the debugger top-half can find it on disk
somehow.

Unfortunately the latter ("out of scope, find the file") is somewhat
at odds with the desired developer experience:

- It means that we need some way of mapping a compiled Wasm artifact
back to a source Wasm; absent "here's the full bytecode", that means
"here's the path to the full bytecode", but that path is an
identifier that may not be universally accessible (consider
e.g. capabilities/preopens present for a debugger component) or
portable (consider e.g. moving the artifact to a different machine).

- Or we don't even provide that metadata, and require the user to
explicitly specify the same module filename twice -- once to
actually run it, and once as an argument to the debugger.

- It means that we should account for stale artifacts and mark the
mismatch somehow; e.g. if the user starts debugging with Wasmtime,
either from a `.cwasm` on disk or with one produced in-memory just
for this run, and then subsequently rebuilds their source `.wasm`,
we no longer have a reference for it. (The same problem exists one
level up if source code is edited, but source to a Wasm producer
toolchain is definitely out-of-scope for Wasmtime.)

- It means that special logic is required in the case of components to
map a module back to a specific component section (we would
essentially have to expose the static module IDs, then require the
debugger top-half to re-implement our exact flattening algorithm to
find that core module).

The permissions issue alone was enough to convince me that we should
do something better than providing a filename (why should we have to
authorize the adapter to read the user's filesystem?) but all of the
other benefits -- ensuring an exact match and ensuring perfect
availability -- are a nice bonus. The main downside is making the
`.cwasm` larger (possibly substantially so), but this overhead is only
present when enabling guest-debugging, the data has to be present
anyway, and this is likely not a dealbreaker.

* miri ignore tests with compilation

* Review feedback.

show more ...

RR #2: Sha256 checksum for components (#12576)

* Add sha256 checksum for component for record/replay consistency

* Move sha2 crate as workspace dependency

* Run checksum digest only on recording c

RR #2: Sha256 checksum for components (#12576)

* Add sha256 checksum for component for record/replay consistency

* Move sha2 crate as workspace dependency

* Run checksum digest only on recording configs

* Fix CI error and restructure from_binary

show more ...

Refactor `String` usage in `wasmtime_environ::Module` (#12565)

* Refactor `String` usage in `wasmtime_environ::Module`

Do not hold regular `String`s; instead use our own OOM-handling
`wasmtime_core

Refactor `String` usage in `wasmtime_environ::Module` (#12565)

* Refactor `String` usage in `wasmtime_environ::Module`

Do not hold regular `String`s; instead use our own OOM-handling
`wasmtime_core::alloc::String` or, even better, an interned `Atom` from the
`StringPool`.

Also avoid `IndexMap`, as it doesn't handle OOMs.

* Use OOM-handling `IndexMap` in `wasmtime_environ::Module`

* review feedback

show more ...

Refactor the `TypeRegistry` to partially handle OOM (#12500)

* Refactor the `TypeRegistry` to partially handle OOM

This refactors the type registry to be more "columnar" when registering the
types

Refactor the `TypeRegistry` to partially handle OOM (#12500)

* Refactor the `TypeRegistry` to partially handle OOM

This refactors the type registry to be more "columnar" when registering the
types within a rec group, so that we

* Add all the types themselves
* Add the rec group metadata for all types in the rec group
* Add the supertypes metadata for all types in the rec group
* Etc...

Instead of adding one type, its rec group metadata, its supertypes metadata,
etc... and then moving on to the next type.

This makes it easier to pre-reserve space and roll back changes on OOM errors.

This is part of https://github.com/bytecodealliance/wasmtime/issues/12069 and
the OOM handling effort, but doesn't fully get the `TypeRegistry` to a place
where it handles all OOMs yet. There are a couple places that need further
work (usage of hash sets and `Cow::into_owned`) which I have marked with `TODO`
comments. In the meantime, I found this to be a nice refactoring of the existing
functionality, so I think it can land as-is.

* review feedback

show more ...

Fix `Module::image_range` to include non-text part of module. (#12302)

* Fix `Module::image_range` to include non-text part of module.

This method is informational for embedders to be able to, for

Fix `Module::image_range` to include non-text part of module. (#12302)

* Fix `Module::image_range` to include non-text part of module.

This method is informational for embedders to be able to, for example,
ensure an image in memory is `mlock`'d (not swapped out). In the
refactors around the StoreCode/EngineCode split, I mistakenly redefined
this to only the text section. This is not a Wasm execution correctness
issue but may lead to performance issues if an embedder relies on this
behavior. This PR fixes the definition.

* Components as well as core modules.

* Ignore new tests in miri.

show more ...

Migrate the `wasmtime` crate to `wasmtime_environ::error::*` (#12231)

* Migrate the `wasmtime` crate to `wasmtime_environ::error::*`

Instead of `anyhow::Error`.

This commit re-exports the `wasmtim

Migrate the `wasmtime` crate to `wasmtime_environ::error::*` (#12231)

* Migrate the `wasmtime` crate to `wasmtime_environ::error::*`

Instead of `anyhow::Error`.

This commit re-exports the `wasmtime_environ::error` as the `wasmtime::error`
module, updates the prelude to include these new error-handling types, redirects
our top-level `wasmtime::{Error, Result}` re-exports to re-export
`wasmtime::error::{Error, Result}`, and updates various use sites that were
directly using `anyhow` to use the new `wasmtime` versions.

This process also required updating the component macro and wit-bindgen macro to
use the new error types instead of `anyhow`.

Part of https://github.com/bytecodealliance/wasmtime/issues/12069

* Replace wasmtime::error::Thing with wasmtime::Thing where it makes sense

* cargo fmt

* Move `crate::error::Thing` to `crate::Thing` where it makes sense

show more ...

Debug: implement breakpoints and single-stepping. (#12133)

* Debug: implement breakpoints and single-stepping.

This is a PR that puts together a bunch of earlier pieces (patchable
calls in #12061 a

Debug: implement breakpoints and single-stepping. (#12133)

* Debug: implement breakpoints and single-stepping.

This is a PR that puts together a bunch of earlier pieces (patchable
calls in #12061 and #12101, private copies of code in #12051, and all
the prior debug event and instrumentation infrastructure) to implement
breakpoints in the guest debugger.

These are implemented in the way we have planned in #11964: each
sequence point (location prior to a Wasm opcode) is now a patchable call
instruction, patched out (replaced with NOPs) by default. When patched
in, the breakpoint callsite calls a trampoline with the `patchable` ABI
which then invokes the `breakpoint` hostcall. That hostcall emits the
debug event and nothing else.

A few of the interesting bits in this PR include:
- Implementations of "unpublish" (switch permissions back to read/write
from read/execute) for mmap'd code memory on all our platforms.
- Infrastructure in the frame-tables (debug info) metadata producer and
parser to record "breakpoint patches".
- A tweak to the NOP metadata packaged with the `MachBuffer` to allow
multiple NOP sizes. This lets us use one 5-byte NOP on x86-64, for
example (did you know x86-64 had these?!) rather than five 1-byte
NOPs.

This PR also implements single-stepping with a global-per-`Store` flag,
because at this point why not; it's a small additional bit of logic to
do *all* patches in all modules registered in the `Store` when that flag
is enabled.

A few realizations for future work:
- The need for an introspection API available to a debugger to see the
modules within a component is starting to become clear; either that,
or the "module and PC" location identifier for a breakpoint switches
to a "module or component" sum type. Right now, the tests for this
feature use only core modules. Extending to components should not
actually be hard at all, we just need to build the API for it.
- The interaction between inlining and `patchable_call` is interesting:
what happens if we inline a `patchable_call` at a `try_call` callsite?
Right now, we do *not* update the `patchable_call` to a `try_call`,
because there is no `patchable_try_call`; this is fine in the Wasmtime
embedding in practice because we never (today!) throw exceptions from
a breakpoint handler. This does suggest to me that maybe we should
make patchability a property of any callsite, and allow try-calls to
be patchable too (with the same restriction about no return values as
the only restriction); but happy to discuss that one further.

* Add missing debug.wat disas test.

* Review feedback.

* Fix comment on `CodeMemory::text_mut`.

* Review feedback.

* Review feedback: abort process on failure to re-apply executable permissions.

* Implement icache flush for aarch64.

This appears to be necessary as we otherwise see a failure in CI on
macOS/aarch64 that is consistent with patched-in breakpoint calls still
being incorrectly cached after we remove them and republish the code.

There is a longstanding issue in #3310 tracking proper icache coherence
handling on aarch64. We implemented this for Linux with the `membarrier`
syscall but never did so for macOS. Maybe this is the first point at
which it matters, because code was always loaded at new addresses (hence
did not have coherence issues because nothing would have been cached)
previously.

prtest:full

* Review feedback: use `next_multiple_of`.

show more ...

Debug: create private code memories per store when debugging is enabled. (#12051)

* Debug: create private code memories per store when debugging is enabled.

This will allow patching code to implem

Debug: create private code memories per store when debugging is enabled. (#12051)

* Debug: create private code memories per store when debugging is enabled.

This will allow patching code to implement e.g. breakpoints. (That is,
for now the copies are redundant, but soon they will not be.)

This change follows the discussion [here] and offline to define a few
types that better encapsulate the distinction we want to enforce.
Basically, there is almost never a bare `CodeMemory`; they are always
wrapped in an `EngineCode` or `StoreCode`, the latter being a per-store
instance of the former. Accessors are moved to the relevant place so
that, for example, one cannot get a pointer to a Wasm function's body
without being in the context of a `Store` where the containing module
has been registered. The registry then returns a `ModuleWithCode` that
boxes up a `Module` reference and `StoreCode` together for cases where
we need both the metadata from the module and the raw code to derive
something.

The only case where we return raw code pointers to the `EngineCode`
directly have to do with Wasm-to-array trampolines: in some cases, e.g.
`InstancePre` pre-creating data structures with references to host
functions, it breaks our expected performance characteristics to make
the function pointers store-specific. This is fine as long as the
Wasm-to-array trampolines never bake in direct calls to Wasm functions;
the strong invariant is that Wasm functions never execute from
`EngineCode` directly. Some parts of the component runtime would also
have to be substantially refactored if we wanted to do away with this
exception.

The per-`Store` module registry is substantially refactored in this PR.
I got rid of the modules-without-code distinction (the case where a
module only has trampolines and no defined functions still works fine),
and organized the BTreeMaps to key on start address rather than end
address, which I find a little more intuitive (one then queries with the
dual to the range -- 0-up-to-PC and last entry found).

[here]: https://github.com/bytecodealliance/wasmtime/pull/12051#pullrequestreview-3493711812

* Review feedback: do not assume a reasonable code alignment; error when it cannot be known

* Review feedback: fail properly in profiler when we are cloning code

* Fix guest-profiler C API.

* Review feedback: make private-code representation impossible in non-debugging-support builds.

* Add TODO comment referencing issue for cloning only .text.

* clang-format

* Review feedback: add back Component::image_range.

* Review feedback: error on registering profiling metadata when debug is enabled.

* rustfmt

* Remove early bail on profiling-data registration when debugging is enabled: this always happens so we cannot error out.

show more ...

Add missing `gc` feature gates (#12074)

* add more `gc` feature gates

* test debug gc feature combination

Wasmtime: implement debug instrumentation and basic host API to examine runtime state. (#11769)

* Wasmtime: implement debug instrumentation and basic host API to examine runtime state.

This PR impl

Wasmtime: implement debug instrumentation and basic host API to examine runtime state. (#11769)

* Wasmtime: implement debug instrumentation and basic host API to examine runtime state.

This PR implements ideas from the [recent RFC] to serve as the basis
for Wasm (guest) debugging: it adds a stackslot to each function
translated from Wasm, stores to replicate Wasm VM state in the
stackslot as the program runs, and metadata to describe the format of
that state and allow reading it out at runtime.

As an initial user of this state, this PR adds a basic "stack view"
API that, from host code that has been called from Wasm, can examine
Wasm frames currently on the stack and read out all of their locals
and stack slots.

Note in particular that this PR does not include breakpoints,
watchpoints, stepped execution, or any sort of user interface for any
of this; it is only a foundation.

This PR still has a few unsatisfying bits that I intend to address:

- The "stack view" performs some O(n) work when the view is initially
taken, computing some internal data per frame. This is forced by the
current design of `Backtrace`, which takes a closure and walks that
closure over stack frames eagerly (rather than work as an
iterator). It's got some impressive iterator-chain stuff going on
internally, so refactoring it to the latter approach might not
be *too* bad, but I haven't tackled it yet.

A O(1) stack view, that is, one that does work only for frames as
the host API is used to walk up the stack, is desirable because some
use-cases may want to quickly examine e.g. only the deepest
frame (say, running with a breakpoint condition that needs to read a
particular local's value after each step).

- It includes a new `Config::compiler_force_inlining()` option that is
used only for testing that we get the correct frames after
inlining. I couldn't get the existing flags to work on a Wasmtime
config level and suspect there may be an existing bug there; I will
try to split out a fix for it.

This PR renames the existing `debug` option to `native_debug`, to
distinguish it from the new approach.

[recent RFC]: https://github.com/bytecodealliance/rfcs/pull/44

* Update to new APIs on Cranelift side.

* Test update.

* Adjust objdump printing of InstPos on frame progpoints; and adjust progpoint collapsing.

* Convert to iterator form.

* Fix path in native-debug tests (debug -> native_debug rename).

* Enforce that `debug_instrumentation` can only be enabled when feature is enabled.

* Add missing assert.

* Use builtin knob for forcing intra-module inlining instead.

* Review feedback:

- Make StackView own the current frame rather than handing it out. This
prevents the current frame (`FrameView`) from walking away and hiding
somewhere it shouldn't, to be used unsoundly later.
- Assert no-GC during stack walk.

* Merge debug-instrumentation hooks on FuncEnvironment into before/after hooks.

* Review feedback: avoid downcasting funcs twice.

* Add debug feature to `wasmtime` crate's defaults.

* Review feedback: u32s for local and stack indices in debug host API.

* Use *const u8 as stack pointers and `with_exposed_provenance` in debug API.

* Remove some `srcloc` plumbing in Wasm translator.

* Rename native-debug back to debug, and make the new thing "guest debugging".

* rustfmt in debugging test.

* fix disas test after guest-debug CLI option rename.

* Review feedback: no separate debug-instrumentation hooks on FuncEnvironment.

* Review feedback: update doc comment on `Config::guest_debug`.

* Review feedback: rename `generate_debuginfo` to `debug_native` in tunables.

* Review feedback: miscellaneous comments.

* Review comment: fix wording in safety conditions.

* revert wasi-common submodule update

* Properly root values in debug frame slots.

Fixes #11841.

* Fix non-`debug`-feature build.

* Review feedback: naming.

* Ignore tests that compile modules in miri.

show more ...

Lookup functions in the text section by `FuncKey` at runtime (#11630)

This commit refactors our metadata, treating compiled functions homogeneously
and removing the need to add new tables to places

Lookup functions in the text section by `FuncKey` at runtime (#11630)

This commit refactors our metadata, treating compiled functions homogeneously
and removing the need to add new tables to places like `CompiledModuleInfo`
whenever we add a new kind of function. This also simplifies the process of
constructing the metadata for a final, linked compilation artifact. Finally, it
paves the way to doing gc-sections during our linking process (which would give
us smaller code sizes by removing functions that have been inlined into every
caller, for example) as we now allow holes in certain types of function index
spaces that were previously always densely populated.

We have two kinds of index spaces:

1. Mostly-dense index spaces, which take O(max_index) space and provide O(1)
lookups.

2. Sparse index spaces, which take O(num_members) space and provide
O(log n) lookups.

Most of our function index spaces are currently dense, but we can tweak that in
the future if necessary.

Furthermore, code size of `.cwasm` binaries has shrunk very slightly with this
refactoring. Consider `spidermonkey.wasm`'s compiled `.cwasm`:

* Size before: 218756 `.wasmtime.info` section bytes, 20052632 total bytes
* Size after: 213761 `.wasmtime.info` section bytes, 20047640 total bytes

That is a 2.28% reduction on the size of the `.wasmtime.info` section, or a
0.025% reduction total.

However, we previously did a single metadata lookup to get the location of both
a Wasm function itself and its array-to-Wasm trampoline at the same time, and in
the new version of the code two lookups are performed. This is slightly slower,
as shown in our call-indirect micro-benchmark that combines lazy table
initialization (which delays looking up the function element's location until
runtime) with indirect-calling each table element exactly once (which defeats
the amortization of that lookup). So this micro-benchmark is both synthetic and
the worst-case scenario for this commit's change: we are measuring, as much as
we can, *only* the force-initialization-of-a-lazy-funcref-table-slot path.

Ultimately, I believe that the simplification is worth the regression in this
micro-benchmark.

<details>

<summary>call-indirect micro-benchmarks results</summary>

```
call-indirect/same-callee/table-init-lazy/65536-calls
time: [152.77 µs 154.92 µs 157.39 µs]
thrpt: [416.40 Melem/s 423.04 Melem/s 428.99 Melem/s]
change:
time: [−13.749% −10.205% −6.2864%] (p = 0.00 < 0.05)
thrpt: [+6.7081% +11.365% +15.941%]
Performance has improved.
Found 13 outliers among 100 measurements (13.00%)
8 (8.00%) high mild
5 (5.00%) high severe
call-indirect/different-callees/table-init-lazy/65536-calls
time: [4.3564 ms 4.4641 ms 4.5843 ms]
thrpt: [14.296 Melem/s 14.681 Melem/s 15.044 Melem/s]
change:
time: [+38.134% +44.404% +50.927%] (p = 0.00 < 0.05)
thrpt: [−33.743% −30.750% −27.606%]
Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
2 (2.00%) high mild
3 (3.00%) high severe
call-indirect/same-callee/table-init-strict/65536-calls
time: [144.91 µs 148.41 µs 152.02 µs]
thrpt: [431.10 Melem/s 441.58 Melem/s 452.24 Melem/s]
change:
time: [−13.665% −10.470% −7.2626%] (p = 0.00 < 0.05)
thrpt: [+7.8313% +11.694% +15.828%]
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) high mild
3 (3.00%) high severe
call-indirect/different-callees/table-init-strict/65536-calls
time: [195.18 µs 200.67 µs 206.49 µs]
thrpt: [317.38 Melem/s 326.59 Melem/s 335.77 Melem/s]
change:
time: [−15.936% −11.568% −7.0835%] (p = 0.00 < 0.05)
thrpt: [+7.6235% +13.081% +18.957%]
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
5 (5.00%) high mild
```

</details>

show more ...

WebAssembly exception-handling support. (#11326)

* WebAssembly exception-handling support.

This PR introduces support for the [Wasm exception-handling proposal],
which introduces a conventional try

WebAssembly exception-handling support. (#11326)

* WebAssembly exception-handling support.

This PR introduces support for the [Wasm exception-handling proposal],
which introduces a conventional try/catch mechanism to WebAssembly. The
PR supports modules that use `try_table` to register handlers for a
lexical scope; and provides `throw` and `throw_ref` that allocate (in
the first case) and throw exception objects.

This PR builds on top of the work in #10510 for Cranelift-level
exception support, #10919 for an unwinder, and #11230 for exception
objects built on top of GC, in addition a bunch of smaller fix and
enabling PRs around those.

[Wasm exception-handling proposal]: https://github.com/WebAssembly/exception-handling/

prtest:full

* Permit UnwindToWasm to have unused fields in Pulley builds (for now).

* Resolve miri-caught reborrowing issue.

* Ignore exceptions tests in miri for now (Pulley not supported).

* Use wasmtime_test on exceptions tests.

* Get tests passing on pulley platforms

* Add a check to `supports_host` for the generated test and assert
failure also when that is false.
* Remove `pulley_unsupported` test as it falls out of `#[wasmtime_test]`
* Remove `exceptions_store` helper as it falls out of `#[wasmtime_test]`
* Remove miri annotations as they fall out of `#[wasmtime_test]`

* Remove dead import

* Skip some unsupported tests entirely in `#[wasmtime_test]`

If the selected compiler doesn't support the host at all then there's no
need to run it. Actually running it could misinterpret `CraneliftNative`
as "run with pulley" otherwise, so avoid such false negatives.

* Cranelift: dynamic contexts: account for outgoing-args area.

---------

Co-authored-by: Alex Crichton <[email protected]>

show more ...

Add support for the Linux PAGEMAP_SCAN ioctl (#11372)

* WIP: use the pagemap_scan ioctl to selectively reset an instance's dirty pages

* Less hacky, and supporting tables, too

* Bugfixes

* WIP: m

Add support for the Linux PAGEMAP_SCAN ioctl (#11372)

* WIP: use the pagemap_scan ioctl to selectively reset an instance's dirty pages

* Less hacky, and supporting tables, too

* Bugfixes

* WIP: memcpy instead of pread

* Refactor support for pagemap

* Don't hold a raw pointer to the original data, plumb through an `Arc`
to have a safe reference instead.
* Update pagemap bindings to latest version of abstraction written.
* Include pagemap-specific tests.
* Use a `PageMap` structure created once-per-pool instead of a
static-with-a-file.
* Refactor to use the "pagemap path" unconditionally which blends in the
keep_resident bits.

* Improve safety documentation

prtest:full

* Fix some lints

* Skip ioctl tests when it's not supported

* Fix a memory leak by moving impls around

* Fix no vm build

* Review comments

* Add more pagemap-specific documentation

* Add more docs, refactor implementation slightly

* Improve `category_*` docs

Basically forward to the Linux kernel source itself.

* Fix compile

* Make pagemap integration resilient across forks

* Fix non-pooling-allocator-build

* Fix portability issues of new test

* Actually use config on macos

---------

Co-authored-by: Till Schneidereit <[email protected]>

show more ...

Raise `unsafe_op_in_unsafe_fn` further in Wasmtime (#11322)

* Raise `unsafe_op_in_unsafe_fn` further in Wasmtime

Now it's at `wasmtime::runtime`, not just `wasmtime::runtime::vm`.

* Review comments

Remove Stored from wasmtime::Instance (#10909)

* Remove the "export cache" on instances

This is now a relic of the past now that conversion from the internal
to external representation of Wasmtime

Remove Stored from wasmtime::Instance (#10909)

* Remove the "export cache" on instances

This is now a relic of the past now that conversion from the internal
to external representation of Wasmtime items is free. This is
effectively dead code that is no longer needed.

* Remove `Stored` from `wasmtime::Instance`

Powered by all previous commits this is a near-trivial change where an
`Instance` is now more-or-less "just" an `InstanceId`. Additionally the
`host_state: Box<dyn Any>` is no longer needed within `vm::Instance` so
that was removed as well.

show more ...

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtim

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtime to the 2024 Edition to keep
up-to-date with Rust idioms and access many of the edition features
exclusive to the 2024 edition.

prtest:full

* Reformat with the 2024 edition

show more ...

Store stack maps in an ELF section (#10404)

* Store stack maps in an ELF section

This commit moves the storage of stack maps from being embedded within
serde-encoded information to instead being st

Store stack maps in an ELF section (#10404)

* Store stack maps in an ELF section

This commit moves the storage of stack maps from being embedded within
serde-encoded information to instead being stored in a separate ELF
section in the final executable. The motivation for this is to make this
more easily debuggable with a `wasmtime objdump` command in the future
but this additionally should have the nice side effect of making
non-stack-maps modules have smaller encoded information (no need to
encode an empty list) and additionally make stack-maps-using-modules
faster to decode (no serde decoding, it's already "decoded").

This implements a scheme similar to the address map section where
there's a "builder" for the section and then a separate half to decode
the section. The same basic encoding, a bit map, is used. This is likely
going to make accessing stack maps slightly slower, but if that's an
issue we can tweak the representation and align things and/or use
`usize` or such.

* Update crates/environ/src/compile/stack_maps.rs

Co-authored-by: Andrew Brown <[email protected]>

* Review comments

* More review comments

* Fix MIRI test by enabling `unaligned` object feature

---------

Co-authored-by: Andrew Brown <[email protected]>

show more ...

Add Engine/Component::deserialize_raw (#10321)

* Add Engine/Component::deserialize_raw

For targets without virtual memory support, the only mechanism for
load code previously was to copy bytes from

Add Engine/Component::deserialize_raw (#10321)

* Add Engine/Component::deserialize_raw

For targets without virtual memory support, the only mechanism for
load code previously was to copy bytes from a provided slice into
a separate, owned, allocated buffer containing a copy of the serialized
module contents. This is expensive for constrained targets and
prohibits embedded runtimes like doing things like running code
directly out of memory-mapped devices such as NOR flash.

The tradeoff for directly using the externally owned memory is that
the caller must ensure that the code memory will not be written
to for the lifetime of the CodeMemory.

Loading code from externally owned memory is supported for all
configurations but is expected to fail to deserialize on platforms
that suport virtual memory and are attempting to load modules
or components that require that the memory be made executable
(native code rather than pulley).

https://github.com/bytecodealliance/wasmtime/issues/10245

* Fix tests for s390x

Previously, there was a bad assumption that pulley32/64 were the
only targets but there's also the "be" variants. Use the
`pulley_host` helper to better get the right pulley target
for the host.

show more ...

Wasm GC: Fix an incorrect assertion and canonicalize types for runtime usage in ExternType::from_wasmtime (#10223)

* Fix assertion in `PartialEq` for `RegisteredType` again

It is possible for two `

Wasm GC: Fix an incorrect assertion and canonicalize types for runtime usage in ExternType::from_wasmtime (#10223)

* Fix assertion in `PartialEq` for `RegisteredType` again

It is possible for two `WasmSubType`s to be equal to each other, as far as
`derive(PartialEq)` is concerned, but still different from each other if they
are in different rec groups or even if they are at different indices within the
same rec group. The assertion mistakenly did not permit either of these,
however.

Fixes #9714

* Canonicalize all types for runtime usage when creating `wasmtime::{Module,Component}`s

Rather than canonicalizing them on demand in functions like
`{Func,Global,Table}Type::from_wasmtime` and other places. Instead, we do it in
one place, up front, so that it is very unlikely we miss anything. Doing this
involves changing some things from `ModuleInternedTypeIndex`es to
`EngineOrModuleTypeIndex`es in `wasmtime_environ`, which means that a bunch of
uses of those things need to unwrap the appropriate kind of type index at usage
sites (e.g. compilation uses will unwrap `ModuleInternedTypeIndex`es, runtime
uses usage will unwrap `VMSharedTypeIndex`es). And it additionally required
implementing the `TypeTrace` trait for a handful of things to unlock the
provided `canonicalize_for_runtime_usage` trait method for those things.

All this machinery is required to avoid an assertion failure in the regression
test introduced in the previous commit, which was triggered because we were
failing to canonicalize type indices inside `ExternType`s for runtime usage on
some code paths. We shouldn't have to play that kind of whack-a-mole in the
future, thanks to this new approach.

* Fix a warning in no-default-features builds

* Fix another warning in weird cfg builds

show more ...

Enable warnings if `cranelift` is disabled (#10157)

Continuation of work in #10131.

Enable warnings if `gc` is disabled (#10149)

* Enable warnings if `gc` is disabled

Continuation of work in #10131. This additionally handles turning off
`gc-null` and `gc-drc` and the various combi

Enable warnings if `gc` is disabled (#10149)

* Enable warnings if `gc` is disabled

Continuation of work in #10131. This additionally handles turning off
`gc-null` and `gc-drc` and the various combinations within.

* Fix some more warnings

* Fix a feature combo build

show more ...