Update object to 0.39.0 (#12866)

* Update object to 0.39.0

* Add vets for `object`

---------

Co-authored-by: Alex Crichton <[email protected]>

fix: correct various typos (#12807)

Signed-off-by: Ho Kim <[email protected]>

Debugging: preserve original Wasm bytecode inside of compiled ELF artifact. (#12636)

* Debugging: preserve original Wasm bytecode inside of compiled ELF artifact.

This PR adds logic to embed the or

Debugging: preserve original Wasm bytecode inside of compiled ELF artifact. (#12636)

* Debugging: preserve original Wasm bytecode inside of compiled ELF artifact.

This PR adds logic to embed the original core Wasm module(s) from a
compilation into a new ELF section, alongside other metadata
sections. When a component is compiled, the core Wasms inside are
preserved, accessible by their `StaticModuleIndex`es.

The need for this support arises from the guest-debugger
ecosystem. Consider either a debug
component (bytecodealliance/rfcs#45) or a bespoke debugger in native
code using Wasmtime's APIs. In either case, the existing APIs to
introspect execution state provide `Module` references for each
instance from each stack frame, and PC offsets into these `Module`s
are the way in which breakpoints are configured. The debugger will
somehow need to associate these `Module`s with the original Wasm
bytecode, including e.g. any custom sections containing the
producer-specific ways of encoding debug metadata, to do something
useful. In particular also note that the GDB-stub protocol as extended
for Wasm requires read access directly to the Wasm bytecode (it shows
up as part of a "memory map" that is viewed by the standard
read-remote-memory command); we can't delegate this requirement to the
remote end of the stub connection, but have to handle it in the stub
server that runs inside Wasmtime (as a component or bespoke).

We have two main choices: carry the original bytecode all the way
through the Wasmtime compilation pipeline and present it via
`Module::bytecode()`, ready to use; or say that this task is
out-of-scope and that the debugger top-half can find it on disk
somehow.

Unfortunately the latter ("out of scope, find the file") is somewhat
at odds with the desired developer experience:

- It means that we need some way of mapping a compiled Wasm artifact
back to a source Wasm; absent "here's the full bytecode", that means
"here's the path to the full bytecode", but that path is an
identifier that may not be universally accessible (consider
e.g. capabilities/preopens present for a debugger component) or
portable (consider e.g. moving the artifact to a different machine).

- Or we don't even provide that metadata, and require the user to
explicitly specify the same module filename twice -- once to
actually run it, and once as an argument to the debugger.

- It means that we should account for stale artifacts and mark the
mismatch somehow; e.g. if the user starts debugging with Wasmtime,
either from a `.cwasm` on disk or with one produced in-memory just
for this run, and then subsequently rebuilds their source `.wasm`,
we no longer have a reference for it. (The same problem exists one
level up if source code is edited, but source to a Wasm producer
toolchain is definitely out-of-scope for Wasmtime.)

- It means that special logic is required in the case of components to
map a module back to a specific component section (we would
essentially have to expose the static module IDs, then require the
debugger top-half to re-implement our exact flattening algorithm to
find that core module).

The permissions issue alone was enough to convince me that we should
do something better than providing a filename (why should we have to
authorize the adapter to read the user's filesystem?) but all of the
other benefits -- ensuring an exact match and ensuring perfect
availability -- are a nice bonus. The main downside is making the
`.cwasm` larger (possibly substantially so), but this overhead is only
present when enabling guest-debugging, the data has to be present
anyway, and this is likely not a dealbreaker.

* miri ignore tests with compilation

* Review feedback.

show more ...

Use low-level ELF-parsing APIs in `CodeMemory::new` (#12555)

* Use low-level ELF-parsing APIs in `CodeMemory::new`

These avoid allocations and should generally be faster because they avoid
unnecess

Use low-level ELF-parsing APIs in `CodeMemory::new` (#12555)

* Use low-level ELF-parsing APIs in `CodeMemory::new`

These avoid allocations and should generally be faster because they avoid
unnecessary work.

* review feedback

* fix compilation on certain cfgs

* silence warning on build cfg

* .pdata is only used as the unwind section on windows, which is covered by a different match arm

* cargo fmt

show more ...

Forbid rustdoc warnings in CI (#12420)

* Forbid rustdoc warnings in CI

This commit corrects our handling of rustdoc flags in CI to ensure that
warnings indeed fire. Additionally this changes our fl

Forbid rustdoc warnings in CI (#12420)

* Forbid rustdoc warnings in CI

This commit corrects our handling of rustdoc flags in CI to ensure that
warnings indeed fire. Additionally this changes our flags to pass
`-Dwarnings` to ensure that we have warning-free doc builds when all
features are enabled at least.

There were quite a lot of preexisting issues to fix, so this
additionally goes through and fixes all the warnings that cropped up.

* Update nightly toolchain again

prtest:full

* Update another nightly

* Fix a warning in generated code

show more ...

Do not re-export `anyhow!` in the `wasmtime::prelude` (#12298)

We are trying to move to `format_err!` instead.

Migrate the `wasmtime` crate to `wasmtime_environ::error::*` (#12231)

* Migrate the `wasmtime` crate to `wasmtime_environ::error::*`

Instead of `anyhow::Error`.

This commit re-exports the `wasmtim

Migrate the `wasmtime` crate to `wasmtime_environ::error::*` (#12231)

* Migrate the `wasmtime` crate to `wasmtime_environ::error::*`

Instead of `anyhow::Error`.

This commit re-exports the `wasmtime_environ::error` as the `wasmtime::error`
module, updates the prelude to include these new error-handling types, redirects
our top-level `wasmtime::{Error, Result}` re-exports to re-export
`wasmtime::error::{Error, Result}`, and updates various use sites that were
directly using `anyhow` to use the new `wasmtime` versions.

This process also required updating the component macro and wit-bindgen macro to
use the new error types instead of `anyhow`.

Part of https://github.com/bytecodealliance/wasmtime/issues/12069

* Replace wasmtime::error::Thing with wasmtime::Thing where it makes sense

* cargo fmt

* Move `crate::error::Thing` to `crate::Thing` where it makes sense

show more ...

Debug: implement breakpoints and single-stepping. (#12133)

* Debug: implement breakpoints and single-stepping.

This is a PR that puts together a bunch of earlier pieces (patchable
calls in #12061 a

Debug: implement breakpoints and single-stepping. (#12133)

* Debug: implement breakpoints and single-stepping.

This is a PR that puts together a bunch of earlier pieces (patchable
calls in #12061 and #12101, private copies of code in #12051, and all
the prior debug event and instrumentation infrastructure) to implement
breakpoints in the guest debugger.

These are implemented in the way we have planned in #11964: each
sequence point (location prior to a Wasm opcode) is now a patchable call
instruction, patched out (replaced with NOPs) by default. When patched
in, the breakpoint callsite calls a trampoline with the `patchable` ABI
which then invokes the `breakpoint` hostcall. That hostcall emits the
debug event and nothing else.

A few of the interesting bits in this PR include:
- Implementations of "unpublish" (switch permissions back to read/write
from read/execute) for mmap'd code memory on all our platforms.
- Infrastructure in the frame-tables (debug info) metadata producer and
parser to record "breakpoint patches".
- A tweak to the NOP metadata packaged with the `MachBuffer` to allow
multiple NOP sizes. This lets us use one 5-byte NOP on x86-64, for
example (did you know x86-64 had these?!) rather than five 1-byte
NOPs.

This PR also implements single-stepping with a global-per-`Store` flag,
because at this point why not; it's a small additional bit of logic to
do *all* patches in all modules registered in the `Store` when that flag
is enabled.

A few realizations for future work:
- The need for an introspection API available to a debugger to see the
modules within a component is starting to become clear; either that,
or the "module and PC" location identifier for a breakpoint switches
to a "module or component" sum type. Right now, the tests for this
feature use only core modules. Extending to components should not
actually be hard at all, we just need to build the API for it.
- The interaction between inlining and `patchable_call` is interesting:
what happens if we inline a `patchable_call` at a `try_call` callsite?
Right now, we do *not* update the `patchable_call` to a `try_call`,
because there is no `patchable_try_call`; this is fine in the Wasmtime
embedding in practice because we never (today!) throw exceptions from
a breakpoint handler. This does suggest to me that maybe we should
make patchability a property of any callsite, and allow try-calls to
be patchable too (with the same restriction about no return values as
the only restriction); but happy to discuss that one further.

* Add missing debug.wat disas test.

* Review feedback.

* Fix comment on `CodeMemory::text_mut`.

* Review feedback.

* Review feedback: abort process on failure to re-apply executable permissions.

* Implement icache flush for aarch64.

This appears to be necessary as we otherwise see a failure in CI on
macOS/aarch64 that is consistent with patched-in breakpoint calls still
being incorrectly cached after we remove them and republish the code.

There is a longstanding issue in #3310 tracking proper icache coherence
handling on aarch64. We implemented this for Linux with the `membarrier`
syscall but never did so for macOS. Maybe this is the first point at
which it matters, because code was always loaded at new addresses (hence
did not have coherence issues because nothing would have been cached)
previously.

prtest:full

* Review feedback: use `next_multiple_of`.

show more ...

Debug: create private code memories per store when debugging is enabled. (#12051)

* Debug: create private code memories per store when debugging is enabled.

This will allow patching code to implem

Debug: create private code memories per store when debugging is enabled. (#12051)

* Debug: create private code memories per store when debugging is enabled.

This will allow patching code to implement e.g. breakpoints. (That is,
for now the copies are redundant, but soon they will not be.)

This change follows the discussion [here] and offline to define a few
types that better encapsulate the distinction we want to enforce.
Basically, there is almost never a bare `CodeMemory`; they are always
wrapped in an `EngineCode` or `StoreCode`, the latter being a per-store
instance of the former. Accessors are moved to the relevant place so
that, for example, one cannot get a pointer to a Wasm function's body
without being in the context of a `Store` where the containing module
has been registered. The registry then returns a `ModuleWithCode` that
boxes up a `Module` reference and `StoreCode` together for cases where
we need both the metadata from the module and the raw code to derive
something.

The only case where we return raw code pointers to the `EngineCode`
directly have to do with Wasm-to-array trampolines: in some cases, e.g.
`InstancePre` pre-creating data structures with references to host
functions, it breaks our expected performance characteristics to make
the function pointers store-specific. This is fine as long as the
Wasm-to-array trampolines never bake in direct calls to Wasm functions;
the strong invariant is that Wasm functions never execute from
`EngineCode` directly. Some parts of the component runtime would also
have to be substantially refactored if we wanted to do away with this
exception.

The per-`Store` module registry is substantially refactored in this PR.
I got rid of the modules-without-code distinction (the case where a
module only has trampolines and no defined functions still works fine),
and organized the BTreeMaps to key on start address rather than end
address, which I find a little more intuitive (one then queries with the
dual to the range -- 0-up-to-PC and last entry found).

[here]: https://github.com/bytecodealliance/wasmtime/pull/12051#pullrequestreview-3493711812

* Review feedback: do not assume a reasonable code alignment; error when it cannot be known

* Review feedback: fail properly in profiler when we are cloning code

* Fix guest-profiler C API.

* Review feedback: make private-code representation impossible in non-debugging-support builds.

* Add TODO comment referencing issue for cloning only .text.

* clang-format

* Review feedback: add back Component::image_range.

* Review feedback: error on registering profiling metadata when debug is enabled.

* rustfmt

* Remove early bail on profiling-data registration when debugging is enabled: this always happens so we cannot error out.

show more ...

Wasmtime: implement debug instrumentation and basic host API to examine runtime state. (#11769)

* Wasmtime: implement debug instrumentation and basic host API to examine runtime state.

This PR impl

Wasmtime: implement debug instrumentation and basic host API to examine runtime state. (#11769)

* Wasmtime: implement debug instrumentation and basic host API to examine runtime state.

This PR implements ideas from the [recent RFC] to serve as the basis
for Wasm (guest) debugging: it adds a stackslot to each function
translated from Wasm, stores to replicate Wasm VM state in the
stackslot as the program runs, and metadata to describe the format of
that state and allow reading it out at runtime.

As an initial user of this state, this PR adds a basic "stack view"
API that, from host code that has been called from Wasm, can examine
Wasm frames currently on the stack and read out all of their locals
and stack slots.

Note in particular that this PR does not include breakpoints,
watchpoints, stepped execution, or any sort of user interface for any
of this; it is only a foundation.

This PR still has a few unsatisfying bits that I intend to address:

- The "stack view" performs some O(n) work when the view is initially
taken, computing some internal data per frame. This is forced by the
current design of `Backtrace`, which takes a closure and walks that
closure over stack frames eagerly (rather than work as an
iterator). It's got some impressive iterator-chain stuff going on
internally, so refactoring it to the latter approach might not
be *too* bad, but I haven't tackled it yet.

A O(1) stack view, that is, one that does work only for frames as
the host API is used to walk up the stack, is desirable because some
use-cases may want to quickly examine e.g. only the deepest
frame (say, running with a breakpoint condition that needs to read a
particular local's value after each step).

- It includes a new `Config::compiler_force_inlining()` option that is
used only for testing that we get the correct frames after
inlining. I couldn't get the existing flags to work on a Wasmtime
config level and suspect there may be an existing bug there; I will
try to split out a fix for it.

This PR renames the existing `debug` option to `native_debug`, to
distinguish it from the new approach.

[recent RFC]: https://github.com/bytecodealliance/rfcs/pull/44

* Update to new APIs on Cranelift side.

* Test update.

* Adjust objdump printing of InstPos on frame progpoints; and adjust progpoint collapsing.

* Convert to iterator form.

* Fix path in native-debug tests (debug -> native_debug rename).

* Enforce that `debug_instrumentation` can only be enabled when feature is enabled.

* Add missing assert.

* Use builtin knob for forcing intra-module inlining instead.

* Review feedback:

- Make StackView own the current frame rather than handing it out. This
prevents the current frame (`FrameView`) from walking away and hiding
somewhere it shouldn't, to be used unsoundly later.
- Assert no-GC during stack walk.

* Merge debug-instrumentation hooks on FuncEnvironment into before/after hooks.

* Review feedback: avoid downcasting funcs twice.

* Add debug feature to `wasmtime` crate's defaults.

* Review feedback: u32s for local and stack indices in debug host API.

* Use *const u8 as stack pointers and `with_exposed_provenance` in debug API.

* Remove some `srcloc` plumbing in Wasm translator.

* Rename native-debug back to debug, and make the new thing "guest debugging".

* rustfmt in debugging test.

* fix disas test after guest-debug CLI option rename.

* Review feedback: no separate debug-instrumentation hooks on FuncEnvironment.

* Review feedback: update doc comment on `Config::guest_debug`.

* Review feedback: rename `generate_debuginfo` to `debug_native` in tunables.

* Review feedback: miscellaneous comments.

* Review comment: fix wording in safety conditions.

* revert wasi-common submodule update

* Properly root values in debug frame slots.

Fixes #11841.

* Fix non-`debug`-feature build.

* Review feedback: naming.

* Ignore tests that compile modules in miri.

show more ...

Get the min-platform example working again (#11516)

A few various changes have happened in the meantime which means that
this wasn't actually testing anything on CI. Notably these changes were
made:

Get the min-platform example working again (#11516)

A few various changes have happened in the meantime which means that
this wasn't actually testing anything on CI. Notably these changes were
made:

* Sink icache maintenance into `vm::sys` modules. This is fallout of
#11152 where no_std support was added for unix/windows targets. This
commit moves the error-on-lack-of-std to the `unix` and `windows`
modules, relegating the custom module to figure its own pieces out as
necessary. This also feels like a more accurate reflection of how
responsibilities should be sliced up.

* The example runner now uses `bail!` to return an error in case
something bad happens instead of continuing as usual and pretending
nothing bad happened.

show more ...

WebAssembly exception-handling support. (#11326)

* WebAssembly exception-handling support.

This PR introduces support for the [Wasm exception-handling proposal],
which introduces a conventional try

WebAssembly exception-handling support. (#11326)

* WebAssembly exception-handling support.

This PR introduces support for the [Wasm exception-handling proposal],
which introduces a conventional try/catch mechanism to WebAssembly. The
PR supports modules that use `try_table` to register handlers for a
lexical scope; and provides `throw` and `throw_ref` that allocate (in
the first case) and throw exception objects.

This PR builds on top of the work in #10510 for Cranelift-level
exception support, #10919 for an unwinder, and #11230 for exception
objects built on top of GC, in addition a bunch of smaller fix and
enabling PRs around those.

[Wasm exception-handling proposal]: https://github.com/WebAssembly/exception-handling/

prtest:full

* Permit UnwindToWasm to have unused fields in Pulley builds (for now).

* Resolve miri-caught reborrowing issue.

* Ignore exceptions tests in miri for now (Pulley not supported).

* Use wasmtime_test on exceptions tests.

* Get tests passing on pulley platforms

* Add a check to `supports_host` for the generated test and assert
failure also when that is false.
* Remove `pulley_unsupported` test as it falls out of `#[wasmtime_test]`
* Remove `exceptions_store` helper as it falls out of `#[wasmtime_test]`
* Remove miri annotations as they fall out of `#[wasmtime_test]`

* Remove dead import

* Skip some unsupported tests entirely in `#[wasmtime_test]`

If the selected compiler doesn't support the host at all then there's no
need to run it. Actually running it could misinterpret `CraneliftNative`
as "run with pulley" otherwise, so avoid such false negatives.

* Cranelift: dynamic contexts: account for outgoing-args area.

---------

Co-authored-by: Alex Crichton <[email protected]>

show more ...

Raise `unsafe_op_in_unsafe_fn` further in Wasmtime (#11322)

* Raise `unsafe_op_in_unsafe_fn` further in Wasmtime

Now it's at `wasmtime::runtime`, not just `wasmtime::runtime::vm`.

* Review comments

fix(no_std)!: respect `std` feature when target is windows/unix (#11152)

* fix(no_std)!: respect `std` feature when target is windows/unix

BREAKING CHANGE: the fix disables standard features for de

fix(no_std)!: respect `std` feature when target is windows/unix (#11152)

* fix(no_std)!: respect `std` feature when target is windows/unix

BREAKING CHANGE: the fix disables standard features for dependents that use wasmtime with `default-features = false`.

* fix cargo check workflow

* Don't require `std` feature for invalid faults

Fill out the `compile_error!` to avoid failing a build.

* addressing review comments

* add `std` to `stack-switching` feature

* remove `rustix/mm` from `std` feature dependencies

* add `std` feature flag to `jit-icache-coherence`
- The `std` feature gates the use of standard library for icache coherence in Windows; otherwise, defaults to the `libc` implementation.
- The `std` feature of Wasmtime now depends on the `wasmtime-jit-icache-coherence/std`

* Only include jit-icache-coherence on `std` builds

prtest:full

* Fix some idiom issues

* More idiom issues

* Require `std` for loading native code

---------

Co-authored-by: Alex Crichton <[email protected]>

show more ...

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtim

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtime to the 2024 Edition to keep
up-to-date with Rust idioms and access many of the edition features
exclusive to the 2024 edition.

prtest:full

* Reformat with the 2024 edition

show more ...

Switch to using Wasmtime-style builtins for ceil, floor, etc. (#10657)

* Switch to using Wasmtime-style builtins for ceil, floor, etc.

With this patch, all emitted calls are Wasmtime-style builtins

Switch to using Wasmtime-style builtins for ceil, floor, etc. (#10657)

* Switch to using Wasmtime-style builtins for ceil, floor, etc.

With this patch, all emitted calls are Wasmtime-style builtins, rather
than Cranelift-style libcalls. This ensures that all calls from
Cranelift-generated code into Wasmtime host code use the same mechanism,
and eliminates the relocation handling code for the libcall mechanism.

* Update tests.

* Avoid using x86_pshufb on non-x86 platforms.

* Revert unneeded f32/f64 changes in Pulley.

* Define i8x16 as an unconstructible type if sse isn't available.

* Delete the setters too.

* Fix f32/f64 setters.

* Test with prtest:full.

prtest:full

* Support fma.

* Return true for `has_native_fma` on pulley.

This works because pulley already has code implementing fma. This avoids
needing to marshal f32x4 values into builtin function calls on pulley.

* Update tests.

show more ...

Store stack maps in an ELF section (#10404)

* Store stack maps in an ELF section

This commit moves the storage of stack maps from being embedded within
serde-encoded information to instead being st

Store stack maps in an ELF section (#10404)

* Store stack maps in an ELF section

This commit moves the storage of stack maps from being embedded within
serde-encoded information to instead being stored in a separate ELF
section in the final executable. The motivation for this is to make this
more easily debuggable with a `wasmtime objdump` command in the future
but this additionally should have the nice side effect of making
non-stack-maps modules have smaller encoded information (no need to
encode an empty list) and additionally make stack-maps-using-modules
faster to decode (no serde decoding, it's already "decoded").

This implements a scheme similar to the address map section where
there's a "builder" for the section and then a separate half to decode
the section. The same basic encoding, a bit map, is used. This is likely
going to make accessing stack maps slightly slower, but if that's an
issue we can tweak the representation and align things and/or use
`usize` or such.

* Update crates/environ/src/compile/stack_maps.rs

Co-authored-by: Andrew Brown <[email protected]>

* Review comments

* More review comments

* Fix MIRI test by enabling `unaligned` object feature

---------

Co-authored-by: Andrew Brown <[email protected]>

show more ...

Don't pad .text to page size for pulley (#10285)

* Don't pad .text to page size for pulley

When targeting pulley we aren't directly emitting executable
code in the .text section and we don't have a

Don't pad .text to page size for pulley (#10285)

* Don't pad .text to page size for pulley

When targeting pulley we aren't directly emitting executable
code in the .text section and we don't have a good idea of the
true target page size so we end up with ELF files that can
have a significant amount of extra padding around the .text
section with no benefit to the consumer.

The change here aligns with the already present section flag
SH_WASMTIME_NOT_EXECUTED. Padding elimination for the .rodata.wasm
section is already handled by the presence/absence of the
memory-init-on-cow configuration.

For a basic wasip1 hello-world rust program with the combination
of this patch and `-O memory-init-cow=n` I saw a reduction in
the compiled wasm ELF from 421KB to 189KB with the sections no
longer showing as being padded out significantly:

```
$ objdump --section-headers target/wasm32-wasip1/release/hello-wasm-world-0x00.cwasm

target/wasm32-wasip1/release/hello-wasm-world-0x00.cwasm: file format elf64-littleriscv

Sections:
Idx Name Size VMA Type
0 00000000 0000000000000000
1 .wasmtime.engine 00000353 0000000000000000 DATA
2 .wasmtime.bti 00000001 0000000000000000 DATA
3 .text 00011bdc 0000000000000000
4 .wasmtime.addrmap 00011c5c 0000000000000000 DATA
5 .wasmtime.traps 00000ae0 0000000000000000 DATA
6 .rodata.wasm 000019e8 0000000000000000 DATA
7 .name.wasm 000027a6 0000000000000000 DATA
8 .wasmtime.info 000010f9 0000000000000000 DATA
9 .symtab 00001788 0000000000000000
10 .strtab 000040f0 0000000000000000
11 .shstrtab 00000089 0000000000000000
```

* code_memory: don't unpublish non-published memory

For the case that we construct a CodeMemory with a custom_code_memory
implementation but we don't end up publishing memory, do not try
to unpublish the memory.

This could happen in a few cases:
- The .text didn't require being made executable
- There's an error or other circumstances cause the CodeMemory
to be dropped prior to publish being called.

Within the context of the .text size optimizations for pulley,
this caused CI failures for a default pulley target as the
errant unpublish was now being called on non-aligned memory.
Prior to the .text optimizations there was still an mprotect
being called to change bits on the .text region which just happened
to be benign.

* In addition to published, check needs_executable before custom unpublishing

Changing the placement of published or introducing new state could
be other ways of addressing the problem of doing a custom
unpublish but those approaches would potentially create other issues
with avoiding duplicate relocations.

Given where this code would be used and the ability for the
`custom_code_memory` to provide their own protections, this
seems very much so Good Enough and gets the tests passing
again while providing more reasonable behavior for user code.

show more ...

Add Engine/Component::deserialize_raw (#10321)

* Add Engine/Component::deserialize_raw

For targets without virtual memory support, the only mechanism for
load code previously was to copy bytes from

Add Engine/Component::deserialize_raw (#10321)

* Add Engine/Component::deserialize_raw

For targets without virtual memory support, the only mechanism for
load code previously was to copy bytes from a provided slice into
a separate, owned, allocated buffer containing a copy of the serialized
module contents. This is expensive for constrained targets and
prohibits embedded runtimes like doing things like running code
directly out of memory-mapped devices such as NOR flash.

The tradeoff for directly using the externally owned memory is that
the caller must ensure that the code memory will not be written
to for the lifetime of the CodeMemory.

Loading code from externally owned memory is supported for all
configurations but is expected to fail to deserialize on platforms
that suport virtual memory and are attempting to load modules
or components that require that the memory be made executable
(native code rather than pulley).

https://github.com/bytecodealliance/wasmtime/issues/10245

* Fix tests for s390x

Previously, there was a bad assumption that pulley32/64 were the
only targets but there's also the "be" variants. Use the
`pulley_host` helper to better get the right pulley target
for the host.

show more ...

Improve support for completely unknown architectures (#10107)

* Improve support for completely unknown architectures

This commit is a step in the direction of trying to make Wasmtime more
portable

Improve support for completely unknown architectures (#10107)

* Improve support for completely unknown architectures

This commit is a step in the direction of trying to make Wasmtime more
portable by default. The goal here is to enable Wasmtime to compile for
architectures that it has no prior knowledge of. There's a few
miscellaneous locations through Wasmtime where we need
architecture-specific intrinsics and such but that's all in service of
Cranelift itself. Without Cranelift support none of them are necessary.

This commit plumbs a custom `#[cfg]` from Wasmtime's `build.rs` script
into the crate about whether there's a supported Cranelift backend. If
this isn't available some architecture-specific intrinsics are turned
off and not included. An example is that `vm::arch` entirely disappears
which is only in service of `UnwindHost`, which also disappears.
Furthermore the `helpers.c` file also entirely disappears as it's not
necessary on unknown architectures.

To help keep this working I've added CI to build Wasmtime for
`powerpc64le-unknown-linux-gnu`. Wasmtime currently has no support for
this architecture, although if it grows such support in the future
this'll need to be changed to some other unsupported architecture.

* Review feedback

* Fix powerpc build

* Refactor windows trap handling to look like Unix

Shuffle some files around to be more amenable to #[cfg]

* Move signal-handling tests to wasmtime crate

That way it's got easy access to the #[cfg]'s from the build script

* Disable signals support without a host compiler

Even if custom signals are enabled, don't compile it in.

prtest:full

* Fix windows unused imports

* Fix unused imports on Windows

* Remove untested stubs for arch intrinsics

These aren't needed any more to compile Pulley

* Defer tunables validation to loading modules

Instead of validating at `Engine` config time instead validate at
`Module` config time to enable cross-compilation.

* Skip `Tunables` auto-configuration if cross-compiling

This commit

* Tweak some Tunables based on Pulley

Ensures that specific `--target pulleyNN` works most of the time.

* Update host_segfault.rs to handle new 32-bit defaults

No signal handlers are used at all with Pulley so when the async stack
overflows there's no message printed any more.

* Disable Tunables::signals_based_traps on miri

show more ...

Replace `signals-based-traps` with auto-detection (#9941)

* Replace `signals-based-traps` with auto-detection

This commit refactors the platform support of the `wasmtime` crate
itself to remove the

Replace `signals-based-traps` with auto-detection (#9941)

* Replace `signals-based-traps` with auto-detection

This commit refactors the platform support of the `wasmtime` crate
itself to remove the previously added `signals-based-traps` feature in
favor of auto-detecting whether it's there or not. The `build.rs`
script for the `wasmtime` crate will now detect the target platform and
auto-enable this feature as necessary.

The `signals-based-traps` cargo feature is removed and split into two
custom `#[cfg]` directives that the build script sets:

* `has_virtual_memory` - this is used to gate mmap implementations for
example. This is enabled on `unix || windows` and will be off for
`no_std` targets for example. This is split out of
"signals-based-traps" to better handle platforms like iOS which have
virtual memory but don't execute native code (removing the need for
native signals).

* `has_native_signals` - gates signal handlers on Unix for example. This
is disabled on MIRI but otherwise enabled for `unix || windows`. This
is intended to in the future get disabled for iOS by default for
example since it's not necessary when using Pulley. This is
additionally off-by-default for `no_std` platforms.

Two new crate features were added for `no_std` or "custom" platforms to
opt-in to the `wasmtime-platform.h` C APIs for implementing virtual
memory and signals. These are used in the `min-platform` embedding example.

This commit additionally updates some various documentation here and
there to be more up-to-date.

* Update CI configuration

* Fix compile warnings

* Fix test on miri

* Fix more tests on miri

* Fix some warnings

* Another round of miri/CI attempts/fixes

prtest:full

show more ...

Wasmtime: support a notion of "custom code publisher". (#9778)

* Wasmtime: support a notion of "custom code publisher".

In some `no_std` environments, virtual memory usage is *generally*
prohibited

Wasmtime: support a notion of "custom code publisher". (#9778)

* Wasmtime: support a notion of "custom code publisher".

In some `no_std` environments, virtual memory usage is *generally*
prohibited for performance-predictability reasons, but the MMU
hardware is still in use for permissions (e.g., `W^X`
write-xor-execute). Occasional changes to page mapping permissions are
thus necessary when new modules are loaded dynamically, and are
acceptable in that context. Wasmtime needs a way to support
"publishing" code (making it executable) in such environments.

Rather than try to segment the `signals-based-traps` divide further,
and piece out the code-publishing parts from the heap parts, and
backdoor a path to `mprotect` in an otherwise `no_std` build, in this
PR I have opted to add a trait an impl of which the embedder can
provide to the `Config` to implement custom actions for "code
publish". This otherwise operates properly in a
no-`signals-based-traps` environment, e.g., the module backing memory
itself is regularly allocated rather than mmap'd (but is now aligned
to the degree requested by the trait impl).

* Review feedback.

* Plumb through custom alignment for runtime code generation

* Add a test for custom code memory.

show more ...

Rely on `core::error::Error` (#9702)

* Rely on `core::error::Error`

With Wasmtime's new MSRV at 1.81 this means that `core::error::Error` is
available which means that in `no_std` mode the `Error`

Rely on `core::error::Error` (#9702)

* Rely on `core::error::Error`

With Wasmtime's new MSRV at 1.81 this means that `core::error::Error` is
available which means that in `no_std` mode the `Error` trait can be
used. This has been integrated into `anyhow::Error` already upstream and
means that we can remove our own local hacks such as the `Err2Anyhow` trait.

This commit removes the `Err2Anyhow` trait and all usage, going back to
idiomatic Rust error propagation and conversion even in the `no_std`
world. This should make code more portable by default and remove some
weird idioms we had for supporting this.

prtest:full

* Add some trusted vets

* Audit object crate update

* Disable backtraces on CI

show more ...

Start plumbing the ability to run Pulley in Wasmtime (#9646)

* Start plumbing the ability to run Pulley in Wasmtime

This commit starts setting up some infrastructure to run Pulley modules
in Wasmti

Start plumbing the ability to run Pulley in Wasmtime (#9646)

* Start plumbing the ability to run Pulley in Wasmtime

This commit starts setting up some infrastructure to run Pulley modules
in Wasmtime, for example on the CLI. This doesn't actually work just yet
due to other missing pieces of integration but this is enough to get to
a fault and/or other compiler errors at this time. The changes here are:

* An `Engine` allows mismatching targets in the case of Pulley, but the
pointer widths must match.
* All subcommands of Wasmtime now support `--target`, e.g. `wasmtime
run`, as previously only the host was allowed.

* Fix compile of bench-api

* Fix miri tests

* Switch from `NativeEndian` to `Endianness`

This switches parsing ELF from always using the native endianness to
instead using a runtime-defined value of endianness. This enables
big-endian platforms such as s390x to load objects produced for pulley,
a little-endian platform.

show more ...

Add a `signals-based-traps` Cargo compile-time feature (#9614)

* Gate signal handlers behind a new Cargo feature

This commit adds a new on-by-default Cargo feature to the `wasmtime`
crate named `si

Add a `signals-based-traps` Cargo compile-time feature (#9614)

* Gate signal handlers behind a new Cargo feature

This commit adds a new on-by-default Cargo feature to the `wasmtime`
crate named `signals-based-traps`. This is modeled after the
`Config::signals_based_traps` configuration at runtime and can be used
to statically disable the use of signal handlers in Wasmtime. This
notably reduces the number of platform dependencies that Wasmtime has
and provides a mode of avoiding relying on signals altogether.

This introduces a new `MallocMemory` which is a linear memory backed by
the system allocator. This new type of memory is enabled when virtual
memory guards are disabled and signals-based-traps are disabled. This
means that this new type of memory will be candidate for fuzzing for
example.

prtest:full

* Fix rebase conflict

* Refactor `MmapVec` documentation and representation

* Remove no-longer-needed `Arc`
* Document it may be backed by `Vec<u8>`

show more ...