Migrate fuzzing to `wasmtime::error` (#12263)

* Migrate fuzzing to `wasmtime::error`

* fix

cranelift: stack-switching support (#11003)

* cranelift: stack-switching support

This initial commit represents the "pr2" base commit with
minimal merge conflicts resolved. Due to OOB conflicts, t

cranelift: stack-switching support (#11003)

* cranelift: stack-switching support

This initial commit represents the "pr2" base commit with
minimal merge conflicts resolved. Due to OOB conflicts, this
commit is not functional as-is, but using it as a base in
order to allow for easier reviewing of the delta from
this commit to what will be used for the PR against upstream.

Co-authored-by: Daniel Hillerström <[email protected]>
Co-authored-by: Paul Osborne <[email protected]>

* cranelift: stack-switching updates pass 1

This first set of changes updates the base pr in order to
compiled and pass basic checks (compile, clippy, fmt) with
the biggest part of the change being to eliminate injection
of tracing/assertions in JIT'ed code.

* cranelift: stack-switching: restore original visibility for a few func_environ members

* cranelift: stack-switching conditional compilation

At this point, the only bit we really branch on is what we
do in order to avoid problems tying into wasmtime_environ.
This is basd on the approach and macro used by the gc code for
converting presence/absence of the cranelift feature flag to
cranelift compile time. This is a bit of a half-measure for now
as we still compile most stack-switching code in cranelift, but
this does enough to avoid causing problems with missing definitions
in wasmtime_environ.

* cranelift: avoid "as" casts in stack-switching

Replace either with infallible From or fallible, panicing
TryFrom alternatives where required.

* cranelift: cleanup stack-switching control_effect signatures

After removing emission of runtime trace logging and assertions,
there were several unused parameters. Remove those from the
ControlEffect signatures completely.

* cranelift: rename stack-switching VMArray to VMHostArray

This matches a change to the mirrored runtime type in
the upstream changes.

* stack-switching: fix typo

Co-authored-by: Daniel Hillerström <[email protected]>

* stack-switching: used Index impl for get_stack_slot_data

* stack-switching: use smallvec over vec in several cases

* stack-switching: avoid resumetable naming confusion

* stack-switching: cleanup unused params from unchecked_get_continuation

The extra parameters here used to be used for emitting runtime
assertions, but with those gone we just had unused params
and lifetimes, clean those out.

* stack_switching: simplify store_data_entries assertion

* stack-switching: simplify translate_table_{grow,fill} control flow

* stack-switching: remove translate_resume_throw stub

There's already a stub elsewhere and this is not called, when
exceptions are added and it is time to revisit, this method
can be restored.

* stack-switching: compute control_context_size based on target triple

* stack-switching: VMHostArrayRef updates

Rename VMHostArray -> VMHostArrayRef
Change impl to compute address with offset upfront rather than
on each load.

* stack-switching: move cranelift code to live under func_environ

This matches the directory structure for gc and aids in visibility
for a few members required by stack-switching code in cranelift.

* stack-switching: formatting fix

* stack-switching: reduce visibility on a few additional items

* stack-switching: simplify contobj fatptr con/de-struction

* stack-switching: add disas tests to cover new instructions

* stack-switching: fix layout of VMContObj

In the course of the various runtime updates, the layout of the
runtime VMContObj got switched around. This resulted in failures
when doing certain table operations on continuations.

This change fixes that layout problem and adds some tests with
offsets to avoid the problem. Due to the way that we interact
with the VMContObj in cranelift, we don't use these offsets outside
of the tests.

* Fix formatting of merge conflict resolution

* cranelift: remove ir::function::get_stack_slot_data

This method isn't required as sized_stack_slots is already pub.

* stack-switching: reduce visibility of a couple func_environ methods

* stack-switching: define VMContObj as two words

This change migrates VMContObj and its usages in cranelift and runtime
to work with the VMContObj fat pointer as two words in order to better
target different architectures (still gated to x86_64 for now).

To support this, a size type was plumbed into the builtins function
signature types (as is done for component types) that maps to
usize.

* fixup! stack-switching: define VMContObj as two words

* stack-switching: add stub Val::ContRef

This type is not fully complete until continuation/gc integration
is revisited (#10248) but without these changes, test cases are
now failing on panics as we need some representation of
continuation references in the runtime Val enumeration.

Runtime errors with TODO notes are added for the stubbed
code paths to revisit later.

* fixup! stack-switching: add stub Val::ContRef

* fixup! stack-switching: add stub Val::ContRef

* fixup! stack-switching: define VMContObj as two words

prtest:full

* stack-switching: don't conflate host and target pointer sizes

Disas tests were failing on i686 targeting x86_64 as the size of the
host pointer was leaking into what we were using to do codegen
in a few paths. This patch is a bit of a hack as it seems like
using a generic <T> for T: *mut u8 (as an example) is a bit
questionable. To keep things small, I do a hacky typecheck to map
pointers to the target pointer size here.

* stack-switching: VMHostArray entry sizes based off env PtrSize

Revisiting the previous commit with an approach that should be
less brittle.

---------

Co-authored-by: Frank Emrich <[email protected]>
Co-authored-by: Daniel Hillerström <[email protected]>

show more ...

Implement exception objects. (#11230)

* WIP: Working exception objects

* Clean build with gc disabled (`cargo check -p wasmtime --no-default-features --features runtime`).

* Review feedback.

* St

Implement exception objects. (#11230)

* WIP: Working exception objects

* Clean build with gc disabled (`cargo check -p wasmtime --no-default-features --features runtime`).

* Review feedback.

* Stub out C-API support.

* Fix Clippy complaints.

* Fix dead-code warning in c-api build.

* Actually fix 27->26 reserved bit rename and test.

* Fix exnref doc-test.

* fix fuzzing build

* fix feature-flagging on Instance::id

* Bless disas test diff due to reserved-bits change.

* Review feedback.

show more ...

Update the `log` dependency (#11197)

* Update the `log` dependency

This enables getting warnings about formatting strings in the `log`
crate directives which are then additionally fixed here as wel

Update the `log` dependency (#11197)

* Update the `log` dependency

This enables getting warnings about formatting strings in the `log`
crate directives which are then additionally fixed here as well.

* Update dependency directive in `Cargo.toml`

show more ...

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtim

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtime to the 2024 Edition to keep
up-to-date with Rust idioms and access many of the edition features
exclusive to the 2024 edition.

prtest:full

* Reformat with the 2024 edition

show more ...

Enable the GC proposal during general fuzzing (#10332)

* Enable the GC proposal during general fuzzing

This allows us to fuzz Wasm GC in our fuzz targets that use the common
config-generation infra

Enable the GC proposal during general fuzzing (#10332)

* Enable the GC proposal during general fuzzing

This allows us to fuzz Wasm GC in our fuzz targets that use the common
config-generation infrastructure, such as the differential fuzz target.

Fixes #10328

* Make handling of non-deterministic errors more robust in differential fuzzer

* remove logging from functions that can be called from signal handlers

show more ...

Improve some logging in the differential fuzzer (#9982)

Clearly show "lhs" and "rhs" more often and then also swap the order of
the arguments in `assert_error_match` to match the "lhs" and "rhs"
ter

Improve some logging in the differential fuzzer (#9982)

Clearly show "lhs" and "rhs" more often and then also swap the order of
the arguments in `assert_error_match` to match the "lhs" and "rhs"
terminology of the original execution.

show more ...

Add general fuzzing support for custom page sizes (#9462)

* Add general fuzzing support for custom page sizes

* Add custom-page-sizes as an expected feature for module generation

Implement the wide-arithmetic proposal (#9403)

This commit implements the [wide-arithmetic] proposal in Wasmtime. This
is a pretty easy proposal to implement due to Cranelift already having
support

Implement the wide-arithmetic proposal (#9403)

This commit implements the [wide-arithmetic] proposal in Wasmtime. This
is a pretty easy proposal to implement due to Cranelift already having
support for all the various instructions. The features implemented here
are:

* Cranelift support for the four new instructions.
* A new `Config::wasm_wide_arithmetic` option.
* A new `-Wwide-arithmetic` CLI flag.
* A new `wasmtime_config_wasm_wide_arithmetic_set` C API function.
* Support for fuzzing this proposal
* Generation is implemented in `wasm-smith` .
* While it's off-by-default in `wasm-smith` it's enabled-by-default here.
* Differential execution is only possible against Wasmtime right now.
* Single-instruction module support was added for these new instructions.
* Tests for some simple cases plus randomly-generated tests.
* Example disassemblies for new instructions on Cranelift architectures.

[wide-arithmetic]: https://github.com/WebAssembly/wide-arithmetic

show more ...

Update v8 used during fuzzing (#9346)

Been awhile since the last update. I'd hoped that this fixed the local
segfault I've been seeing but alas it did not. Nevertheless seems like a
reasonable idea

Update v8 used during fuzzing (#9346)

Been awhile since the last update. I'd hoped that this fixed the local
segfault I've been seeing but alas it did not. Nevertheless seems like a
reasonable idea to update.

show more ...

Enforce `uninlined_format_args` for the workspace (#9065)

* Enforce `uninlined_format_args` for the workspace

* fix: failing `Monolith Checks` job

* fix: formatting

V8 differential fuzzing: add new table out-of-bounds error message to expectations list. (#8564)

Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=68735.

That fuzzbug bisected to the call

V8 differential fuzzing: add new table out-of-bounds error message to expectations list. (#8564)

Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=68735.

That fuzzbug bisected to the call-indirect caching changes, but this
turned out to be a red herring (the options added in that PR mean that
the fuzzbug config deserializes differently prior to the commit). In
any case, it's an easy fix -- it appears that V8 added a new error
message, so we need to add it to the allowlist of messages that we
expect for a table out-of-bounds condition.

show more ...

Add `GcRuntime` and `GcCompiler` traits; `i31ref` support (#8196)

\### The `GcRuntime` and `GcCompiler` Traits

This commit factors out the details of the garbage collector away from the rest
of the

Add `GcRuntime` and `GcCompiler` traits; `i31ref` support (#8196)

\### The `GcRuntime` and `GcCompiler` Traits

This commit factors out the details of the garbage collector away from the rest
of the runtime and the compiler. It does this by introducing two new traits,
very similar to a subset of [those proposed in the Wasm GC RFC], although not
all equivalent functionality has been added yet because Wasmtime doesn't
support, for example, GC structs yet:

[those proposed in the Wasm GC RFC]: https://github.com/bytecodealliance/rfcs/blob/main/accepted/wasm-gc.md#defining-the-pluggable-gc-interface

1. The `GcRuntime` trait: This trait defines how to create new GC heaps, run
collections within them, and execute the various GC barriers the collector
requires.

Rather than monomorphize all of Wasmtime on this trait, we use it
as a dynamic trait object. This does imply some virtual call overhead and
missing some inlining (and resulting post-inlining) optimization
opportunities. However, it is *much* less disruptive to the existing embedder
API, results in a cleaner embedder API anyways, and we don't believe that VM
runtime/embedder code is on the hot path for working with the GC at this time
anyways (that would be the actual Wasm code, which has inlined GC barriers
and direct calls and all of that). In the future, once we have optimized
enough of the GC that such code is ever hot, we have options we can
investigate at that time to avoid these dynamic virtual calls, like only
enabling one single collector at build time and then creating a static type
alias like `type TheOneGcImpl = ...;` based on the compile time
configuration, and using this type alias in the runtime rather than a dynamic
trait object.

The `GcRuntime` trait additionally defines a method to reset a GC heap, for
use by the pooling allocator. This allows reuse of GC heaps across different
stores. This integration is very rudimentary at the moment, and is missing
all kinds of configuration knobs that we should have before deploying Wasm GC
in production. This commit is large enough as it is already! Ideally, in the
future, I'd like to make it so that GC heaps receive their memory region,
rather than allocate/reserve it themselves, and let each slot in the pooling
allocator's memory pool be *either* a linear memory or a GC heap. This would
unask various capacity planning questions such as "what percent of memory
capacity should we dedicate to linear memories vs GC heaps?". It also seems
like basically all the same configuration knobs we have for linear memories
apply equally to GC heaps (see also the "Indexed Heaps" section below).

2. The `GcCompiler` trait: This trait defines how to emit CLIF that implements
GC barriers for various operations on GC-managed references. The Rust code
calls into this trait dynamically via a trait object, but since it is
customizing the CLIF that is generated for Wasm code, the Wasm code itself is
not making dynamic, indirect calls for GC barriers. The `GcCompiler`
implementation can inline the parts of GC barrier that it believes should be
inline, and leave out-of-line calls to rare slow paths.

All that said, there is still only a single implementation of each of these
traits: the existing deferred reference-counting (DRC) collector. So there is a
bunch of code motion in this commit as the DRC collector was further isolated
from the rest of the runtime and moved to its own submodule. That said, this was
not *purely* code motion (see "Indexed Heaps" below) so it is worth not simply
skipping over the DRC collector's code in review.

\### Indexed Heaps

This commit does bake in a couple assumptions that must be shared across all
collector implementations, such as a shared `VMGcHeader` that all objects
allocated within a GC heap must begin with, but the most notable and
far-reaching of these assumptions is that all collectors will use "indexed
heaps".

What we are calling indexed heaps are basically the three following invariants:

1. All GC heaps will be a single contiguous region of memory, and all GC objects
will be allocated within this region of memory. The collector may ask the
system allocator for additional memory, e.g. to maintain its free lists, but
GC objects themselves will never be allocated via `malloc`.

2. A pointer to a GC-managed object (i.e. a `VMGcRef`) is a 32-bit offset into
the GC heap's contiguous region of memory. We never hold raw pointers to GC
objects (although, of course, we have to compute them and use them
temporarily when actually accessing objects). This means that deref'ing GC
pointers is equivalent to deref'ing linear memory pointers: we need to add a
base and we also check that the GC pointer/index is within the bounds of the
GC heap. Furthermore, compressing 64-bit pointers into 32 bits is a fairly
common technique among high-performance GC
implementations[^compressed-oops][^v8-ptr-compression] so we are in good
company.

3. Anything stored inside the GC heap is untrusted. Even each GC reference that
is an element of an `(array (ref any))` is untrusted, and bounds checked on
access. This means that, for example, we do not store the raw pointer to an
`externref`'s host object inside the GC heap. Instead an `externref` now
stores an ID that can be used to index into a side table in the store that
holds the actual `Box<dyn Any>` host object, and accessing that side table is
always checked.

[^compressed-oops]: See ["Compressed OOPs" in
OpenJDK.](https://wiki.openjdk.org/display/HotSpot/CompressedOops)

[^v8-ptr-compression]: See [V8's pointer
compression](https://v8.dev/blog/pointer-compression).

The good news with regards to all the bounds checking that this scheme implies
is that we can use all the same virtual memory tricks that linear memories use
to omit explicit bounds checks. Additionally, (2) means that the sizes of GC
objects is that much smaller (and therefore that much more cache friendly)
because they are only holding onto 32-bit, rather than 64-bit, references to
other GC objects. (We can, in the future, support GC heaps up to 16GiB in size
without losing 32-bit GC pointers by taking advantage of `VMGcHeader` alignment
and storing aligned indices rather than byte indices, while still leaving the
bottom bit available for tagging as an `i31ref` discriminant. Should we ever
need to support even larger GC heap capacities, we could go to full 64-bit
references, but we would need explicit bounds checks.)

The biggest benefit of indexed heaps is that, because we are (explicitly or
implicitly) bounds checking GC heap accesses, and because we are not otherwise
trusting any data from inside the GC heap, we greatly reduce how badly things
can go wrong in the face of collector bugs and GC heap corruption. We are
essentially sandboxing the GC heap region, the same way that linear memory is a
sandbox. GC bugs could lead to the guest program accessing the wrong GC object,
or getting garbage data from within the GC heap. But only garbage data from
within the GC heap, never outside it. The worse that could happen would be if we
decided not to zero out GC heaps between reuse across stores (which is a valid
trade off to make, since zeroing a GC heap is a defense-in-depth technique
similar to zeroing a Wasm stack and not semantically visible in the absence of
GC bugs) and then a GC bug would allow the current Wasm guest to read old GC
data from the old Wasm guest that previously used this GC heap. But again, it
could never access host data.

Taken altogether, this allows for collector implementations that are nearly free
from `unsafe` code, and unsafety can otherwise be targeted and limited in scope,
such as interactions with JIT code. Most importantly, we do not have to maintain
critical invariants across the whole system -- invariants which can't be nicely
encapsulated or abstracted -- to preserve memory safety. Such holistic
invariants that refuse encapsulation are otherwise generally a huge safety
problem with GC implementations.

\### `VMGcRef` is *NOT* `Clone` or `Copy` Anymore

`VMGcRef` used to be `Clone` and `Copy`. It is not anymore. The motivation here
was to be sure that I was actually calling GC barriers at all the correct
places. I couldn't be sure before. Now, you can still explicitly copy a raw GC
reference without running GC barriers if you need to and understand why that's
okay (aka you are implementing the collector), but that is something you have to
opt into explicitly by calling `unchecked_copy`. The default now is that you
can't just copy the reference, and instead call an explicit `clone` method (not
*the* `Clone` trait, because we need to pass in the GC heap context to run the
GC barriers) and it is hard to forget to do that accidentally. This resulted in
a pretty big amount of churn, but I am wayyyyyy more confident that the correct
GC barriers are called at the correct times now than I was before.

\### `i31ref`

I started this commit by trying to add `i31ref` support. And it grew into the
whole traits interface because I found that I needed to abstract GC barriers
into helpers anyways to avoid running them for `i31ref`s, so I figured that I
might as well add the whole traits interface. In comparison, `i31ref` support is
much easier and smaller than that other part! But it was also difficult to pull
apart from this commit, sorry about that!

---------------------

Overall, I know this is a very large commit. I am super happy to have some
synchronous meetings to walk through this all, give an overview of the
architecture, answer questions directly, etc... to make review easier!

prtest:full

show more ...

Adjust an error message expectation for v8 (#8154)

Looks like the new error message has a data segment index in it so match
a more general error here.

Configure workspace lints, enable running some Clippy lints on CI (#7561)

* Configure Rust lints at the workspace level

This commit adds necessary configuration knobs to have lints configured
at th

Configure workspace lints, enable running some Clippy lints on CI (#7561)

* Configure Rust lints at the workspace level

This commit adds necessary configuration knobs to have lints configured
at the workspace level in Wasmtime rather than the crate level. This
uses a feature of Cargo first released with 1.74.0 (last week) of the
`[workspace.lints]` table. This should help create a more consistent set
of lints applied across all crates in our workspace in addition to
possibly running select clippy lints on CI as well.

* Move `unused_extern_crates` to the workspace level

This commit configures a `deny` lint level for the
`unused_extern_crates` lint to the workspace level rather than the
previous configuration at the individual crate level.

* Move `trivial_numeric_casts` to workspace level

* Change workspace lint levels to `warn`

CI will ensure that these don't get checked into the codebase and
otherwise provide fewer speed bumps for in-process development.

* Move `unstable_features` lint to workspace level

* Move `unused_import_braces` lint to workspace level

* Start running Clippy on CI

This commit configures our CI to run `cargo clippy --workspace` for all
merged PRs. Historically this hasn't been all the feasible due to the
amount of configuration required to control the number of warnings on
CI, but with Cargo's new `[lint]` table it's possible to have a
one-liner to silence all lints from Clippy by default. This commit by
default sets the `all` lint in Clippy to `allow` to by-default disable
warnings from Clippy. The goal of this PR is to enable selective access
to Clippy lints for Wasmtime on CI.

* Selectively enable `clippy::cast_sign_loss`

This would have fixed #7558 so try to head off future issues with that
by warning against this situation in a few crates. This lint is still
quite noisy though for Cranelift for example so it's not worthwhile at
this time to enable it for the whole workspace.

* Fix CI error

prtest:full

show more ...

Return `anyhow::Error` from host functions instead of `Trap`, redesign `Trap` (#5149)

* Return `anyhow::Error` from host functions instead of `Trap`

This commit refactors how errors are modeled w

Return `anyhow::Error` from host functions instead of `Trap`, redesign `Trap` (#5149)

* Return `anyhow::Error` from host functions instead of `Trap`

This commit refactors how errors are modeled when returned from host
functions and additionally refactors how custom errors work with `Trap`.
At a high level functions in Wasmtime that previously worked with
`Result<T, Trap>` now work with `Result<T>` instead where the error is
`anyhow::Error`. This includes functions such as:

* Host-defined functions in a `Linker<T>`
* `TypedFunc::call`
* Host-related callbacks like call hooks

Errors are now modeled primarily as `anyhow::Error` throughout Wasmtime.
This subsequently removes the need for `Trap` to have the ability to
represent all host-defined errors as it previously did. Consequently the
`From` implementations for any error into a `Trap` have been removed
here and the only embedder-defined way to create a `Trap` is to use
`Trap::new` with a custom string.

After this commit the distinction between a `Trap` and a host error is
the wasm backtrace that it contains. Previously all errors in host
functions would flow through a `Trap` and get a wasm backtrace attached
to them, but now this only happens if a `Trap` itself is created meaning
that arbitrary host-defined errors flowing from a host import to the
other side won't get backtraces attached. Some internals of Wasmtime
itself were updated or preserved to use `Trap::new` to capture a
backtrace where it seemed useful, such as when fuel runs out.

The main motivation for this commit is that it now enables hosts to
thread a concrete error type from a host function all the way through to
where a wasm function was invoked. Previously this could not be done
since the host error was wrapped in a `Trap` that didn't provide the
ability to get at the internals.

A consequence of this commit is that when a host error is returned that
isn't a `Trap` we'll capture a backtrace and then won't have a `Trap` to
attach it to. To avoid losing the contextual information this commit
uses the `Error::context` method to attach the backtrace as contextual
information to ensure that the backtrace is itself not lost.

This is a breaking change for likely all users of Wasmtime, but it's
hoped to be a relatively minor change to workaround. Most use cases can
likely change `-> Result<T, Trap>` to `-> Result<T>` and otherwise
explicit creation of a `Trap` is largely no longer necessary.

* Fix some doc links

* add some tests and make a backtrace type public (#55)

* Trap: avoid a trailing newline in the Display impl

which in turn ends up with three newlines between the end of the
backtrace and the `Caused by` in the anyhow Debug impl

* make BacktraceContext pub, and add tests showing downcasting behavior of anyhow::Error to traps or backtraces

* Remove now-unnecesary `Trap` downcasts in `Linker::module`

* Fix test output expectations

* Remove `Trap::i32_exit`

This commit removes special-handling in the `wasmtime::Trap` type for
the i32 exit code required by WASI. This is now instead modeled as a
specific `I32Exit` error type in the `wasmtime-wasi` crate which is
returned by the `proc_exit` hostcall. Embedders which previously tested
for i32 exits now downcast to the `I32Exit` value.

* Remove the `Trap::new` constructor

This commit removes the ability to create a trap with an arbitrary error
message. The purpose of this commit is to continue the prior trend of
leaning into the `anyhow::Error` type instead of trying to recreate it
with `Trap`. A subsequent simplification to `Trap` after this commit is
that `Trap` will simply be an `enum` of trap codes with no extra
information. This commit is doubly-motivated by the desire to always use
the new `BacktraceContext` type instead of sometimes using that and
sometimes using `Trap`.

Most of the changes here were around updating `Trap::new` calls to
`bail!` calls instead. Tests which assert particular error messages
additionally often needed to use the `:?` formatter instead of the `{}`
formatter because the prior formats the whole `anyhow::Error` and the
latter only formats the top-most error, which now contains the
backtrace.

* Merge `Trap` and `TrapCode`

With prior refactorings there's no more need for `Trap` to be opaque or
otherwise contain a backtrace. This commit parse down `Trap` to simply
an `enum` which was the old `TrapCode`. All various tests and such were
updated to handle this.

The main consequence of this commit is that all errors have a
`BacktraceContext` context attached to them. This unfortunately means
that the backtrace is printed first before the error message or trap
code, but given all the prior simplifications that seems worth it at
this time.

* Rename `BacktraceContext` to `WasmBacktrace`

This feels like a better name given how this has turned out, and
additionally this commit removes having both `WasmBacktrace` and
`BacktraceContext`.

* Soup up documentation for errors and traps

* Fix build of the C API

Co-authored-by: Pat Hickey <[email protected]>

show more ...

Throw out fewer fuzz inputs with differential fuzzer (#4859)

* Throw out fewer fuzz inputs with differential fuzzer

Prior to this commit the differential fuzzer would generate a module and
then

Throw out fewer fuzz inputs with differential fuzzer (#4859)

* Throw out fewer fuzz inputs with differential fuzzer

Prior to this commit the differential fuzzer would generate a module and
then select an engine to execute the module against Wasmtime. This
meant, however, that the candidate list of engines were filtered against
the configuration used to generate the module to ensure that the
selected engine could run the generated module.

This commit inverts this logic and instead selects an engine first,
allowing the engine to then tweak the module configuration to ensure
that the generated module is compatible with the engine selected. This
means that fewer fuzz inputs are discarded because every fuzz input will
result in an engine being executed.

Internally the engine constructors have all been updated to update the
configuration to work instead of filtering the configuration. Some other
fixes were applied for the spec interpreter as well to work around #4852

* Fix tests

show more ...

Various improvements to differential fuzzing (#4845)

* Improve wasmi differential fuzzer

* Support modules with a `start` function
* Implement trap-matching to ensure that wasmi and Wasmtime bot

Various improvements to differential fuzzing (#4845)

* Improve wasmi differential fuzzer

* Support modules with a `start` function
* Implement trap-matching to ensure that wasmi and Wasmtime both report
the same flavor of trap.

* Support differential fuzzing where no engines match

Locally I was attempting to run against just one wasm engine with
`ALLOWED_ENGINES=wasmi` but the fuzzer quickly panicked because the
generated test case didn't match wasmi's configuration. This commit
updates engine-selection in the differential fuzzer to return `None` if
no engine is applicable, throwing out the test case. This won't be hit
at all with oss-fuzz-based runs but for local runs it'll be useful to
have.

* Improve proposal support in differential fuzzer

* De-prioritize unstable wasm proposals such as multi-memory and
memory64 by making them more unlikely with `Unstructured::ratio`.
* Allow fuzzing multi-table (reference types) and multi-memory by
avoiding setting their maximums to 1 in `set_differential_config`.
* Update selection of the pooling strategy to unconditionally support
the selected module config rather than the other way around.

* Improve handling of traps in differential fuzzing

This commit fixes an issue found via local fuzzing where engines were
reporting different results but the underlying reason for this was that
one engine was hitting stack overflow before the other. To fix the
underlying issue I updated the execution to check for stack overflow
and, if hit, it discards the entire fuzz test case from then on.

The rationale behind this is that each engine can have unique limits for
stack overflow. One test case I was looking at for example would stack
overflow at less than 1000 frames with epoch interruption enabled but
would stack overflow at more than 1000 frames with it disabled. This
means that the state after the trap started to diverge and it looked
like the engines produced different results.

While I was at it I also improved the "function call returned a trap"
case to compare traps to make sure the same trap reason popped out.

* Fix fuzzer tests

show more ...

Port v8 fuzzer to the new framework (#4739)

* Port v8 fuzzer to the new framework

This commit aims to improve the support for the new "meta" differential
fuzzer added in #4515 by ensuring that a

Port v8 fuzzer to the new framework (#4739)

* Port v8 fuzzer to the new framework

This commit aims to improve the support for the new "meta" differential
fuzzer added in #4515 by ensuring that all existing differential fuzzing
is migrated to this new fuzzer. This PR includes features such as:

* The V8 differential execution is migrated to the new framework.
* `Config::set_differential_config` no longer force-disables wasm
features, instead allowing them to be enabled as per the fuzz input.
* `DiffInstance::{hash, hash}` was replaced with
`DiffInstance::get_{memory,global}` to allow more fine-grained
assertions.
* Support for `FuncRef` and `ExternRef` have been added to `DiffValue`
and `DiffValueType`. For now though generating an arbitrary
`ExternRef` and `FuncRef` simply generates a null value.
* Arbitrary `DiffValue::{F32,F64}` values are guaranteed to use
canonical NaN representations to fix an issue with v8 where with the
v8 engine we can't communicate non-canonical NaN values through JS.
* `DiffEngine::evaluate` allows "successful failure" for cases where
engines can't support that particular invocation, for example v8 can't
support `v128` arguments or return values.
* Smoke tests were added for each engine to ensure that a simple wasm
module works at PR-time.
* Statistics printed from the main fuzzer now include percentage-rates
for chosen engines as well as percentage rates for styles-of-module.

There's also a few small refactorings here and there but mostly just
things I saw along the way.

* Update the fuzzing README

show more ...