fuzz: Enable winch-aarch64 (#12646)

As of https://github.com/bytecodealliance/wasmtime/pull/11051, Winch
supports all of Core Wasm on aarch64. This commit enables fuzzing
respectively.

Migrate fuzzing to `wasmtime::error` (#12263)

* Migrate fuzzing to `wasmtime::error`

* fix

Cleanup/add some testing in `wasmtime-fuzzing` (#12038)

Share the fuzz-generator-helpers across more tests and add smoke tests
for the dynamic/static component API tests too.

Enable the GC proposal during general fuzzing (#10332)

* Enable the GC proposal during general fuzzing

This allows us to fuzz Wasm GC in our fuzz targets that use the common
config-generation infra

Enable the GC proposal during general fuzzing (#10332)

* Enable the GC proposal during general fuzzing

This allows us to fuzz Wasm GC in our fuzz targets that use the common
config-generation infrastructure, such as the differential fuzz target.

Fixes #10328

* Make handling of non-deterministic errors more robust in differential fuzzer

* remove logging from functions that can be called from signal handlers

show more ...

Enable fuzzing for Pulley & Winch (#9966)

* Enable fuzzing for Pulley & Winch

This commit refactors the `wasmtime-fuzzing` crate to enable fuzzing the
Pulley interpreter and the Winch compiler more

Enable fuzzing for Pulley & Winch (#9966)

* Enable fuzzing for Pulley & Winch

This commit refactors the `wasmtime-fuzzing` crate to enable fuzzing the
Pulley interpreter and the Winch compiler more often. Winch has been
feature-complete for much of wasm for a good amount of time now and
Pulley now supports many wasm proposals as well. Both strategies have
wasm proposals that are still disabled, however.

I've run fuzzers for a bit locally and haven't turned up too too much,
but I'm sure OSS-Fuzz will chastise me and tell me all the places I'm
forgetting to add various configs and tweaks.

* Fix test build

show more ...

Improve some logging in the differential fuzzer (#9982)

Clearly show "lhs" and "rhs" more often and then also swap the order of
the arguments in `assert_error_match` to match the "lhs" and "rhs"
ter

Improve some logging in the differential fuzzer (#9982)

Clearly show "lhs" and "rhs" more often and then also swap the order of
the arguments in `assert_error_match` to match the "lhs" and "rhs"
terminology of the original execution.

show more ...

Change how env vars work in `differential` fuzzer (#9457)

* Change how env vars work in `differential` fuzzer

This commit updates the processing of the `ALLOWED_*` environment
variables to work dif

Change how env vars work in `differential` fuzzer (#9457)

* Change how env vars work in `differential` fuzzer

This commit updates the processing of the `ALLOWED_*` environment
variables to work differently than before. Previously the list of
engines and module-generation-strategies were filtered based on
`ALLOWED_*` environment variables but this meant that the meaning of a
fuzz input changed depending on environment variables. This commit
instead replaces the handling of these environment variables to ensure
that the meaning of the fuzz input doesn't change depending on their
values. Instead fuzz test cases are early-thrown-out if they request an
engine that's disabled or a module-generation-strategy that's disabled.

* Update fuzz/fuzz_targets/differential.rs

Co-authored-by: Trevor Elliott <[email protected]>

---------

Co-authored-by: Trevor Elliott <[email protected]>

show more ...

Implement most `array.*` instructions for the GC proposal (#9326)

* Implement most `array.*` instructions for the GC proposal

This does not implement `array.copy` and `array.init_elem` yet, but imp

Implement most `array.*` instructions for the GC proposal (#9326)

* Implement most `array.*` instructions for the GC proposal

This does not implement `array.copy` and `array.init_elem` yet, but implements
all other `array.*` instructions:

* `array.new`
* `array.new_fixed`
* `array.new_default`
* `array.new_data`
* `array.new_elem`
* `array.fill`
* `array.init_data`
* `array.len`
* `array.get`
* `array.get_s`
* `array.get_u`
* `array.set`

Note that the initial plumbing for `array.{copy,init_elem}` is in place, but the
instructions themselves are not implemented yet.

* Fix no-gc builds

* Fix some clippy warnings

* cargo fmt

* Fix another clippy error

* Fix more clippy errors

* Remove debug logging

* Add array.fill helper

* exit scope even on panic

show more ...

Enforce `uninlined_format_args` for the workspace (#9065)

* Enforce `uninlined_format_args` for the workspace

* fix: failing `Monolith Checks` job

* fix: formatting

Fully enable Winch in the `differential` fuzzer (#7932)

This commit fully enables usage of Winch in the `differential` fuzzer
against all other engines with no special cases. I attempted enabling
wi

Fully enable Winch in the `differential` fuzzer (#7932)

This commit fully enables usage of Winch in the `differential` fuzzer
against all other engines with no special cases. I attempted enabling
winch for the other fuzzers as well but Winch doesn't currently
implement all methods for generating various trampolines required so
it's currently only limited to the `differential` fuzzer.

This adds Winch as an "engine" and additionally ensures that when
configured various wasm proposals are disabled that Winch doesn't
support (similar to how enabling `wasmi` disables proposals that `wasmi`
doesn't support).

This does reduce fuzzing of Winch slightly in that the reference-types
proposal is completely disabled for Winch rather than half-enabled where
Winch doesn't implement `externref` operations yet but does implement
`funcref` operations. This, however, enables integrating it more cleanly
into the rest of the fuzzing infrastructure with fewer special cases.

show more ...

add riscv64 backend for cranelift. (#4271)

Add a RISC-V 64 (`riscv64`, RV64GC) backend.

Co-authored-by: yuyang <[email protected]>
Co-authored-by: Chris Fallin <[email protected]>
Co-authored-by

add riscv64 backend for cranelift. (#4271)

Add a RISC-V 64 (`riscv64`, RV64GC) backend.

Co-authored-by: yuyang <[email protected]>
Co-authored-by: Chris Fallin <[email protected]>
Co-authored-by: Afonso Bordado <[email protected]>

show more ...

Throw out fewer fuzz inputs with differential fuzzer (#4859)

* Throw out fewer fuzz inputs with differential fuzzer

Prior to this commit the differential fuzzer would generate a module and
then

Throw out fewer fuzz inputs with differential fuzzer (#4859)

* Throw out fewer fuzz inputs with differential fuzzer

Prior to this commit the differential fuzzer would generate a module and
then select an engine to execute the module against Wasmtime. This
meant, however, that the candidate list of engines were filtered against
the configuration used to generate the module to ensure that the
selected engine could run the generated module.

This commit inverts this logic and instead selects an engine first,
allowing the engine to then tweak the module configuration to ensure
that the generated module is compatible with the engine selected. This
means that fewer fuzz inputs are discarded because every fuzz input will
result in an engine being executed.

Internally the engine constructors have all been updated to update the
configuration to work instead of filtering the configuration. Some other
fixes were applied for the spec interpreter as well to work around #4852

* Fix tests

show more ...

Various improvements to differential fuzzing (#4845)

* Improve wasmi differential fuzzer

* Support modules with a `start` function
* Implement trap-matching to ensure that wasmi and Wasmtime bot

Various improvements to differential fuzzing (#4845)

* Improve wasmi differential fuzzer

* Support modules with a `start` function
* Implement trap-matching to ensure that wasmi and Wasmtime both report
the same flavor of trap.

* Support differential fuzzing where no engines match

Locally I was attempting to run against just one wasm engine with
`ALLOWED_ENGINES=wasmi` but the fuzzer quickly panicked because the
generated test case didn't match wasmi's configuration. This commit
updates engine-selection in the differential fuzzer to return `None` if
no engine is applicable, throwing out the test case. This won't be hit
at all with oss-fuzz-based runs but for local runs it'll be useful to
have.

* Improve proposal support in differential fuzzer

* De-prioritize unstable wasm proposals such as multi-memory and
memory64 by making them more unlikely with `Unstructured::ratio`.
* Allow fuzzing multi-table (reference types) and multi-memory by
avoiding setting their maximums to 1 in `set_differential_config`.
* Update selection of the pooling strategy to unconditionally support
the selected module config rather than the other way around.

* Improve handling of traps in differential fuzzing

This commit fixes an issue found via local fuzzing where engines were
reporting different results but the underlying reason for this was that
one engine was hitting stack overflow before the other. To fix the
underlying issue I updated the execution to check for stack overflow
and, if hit, it discards the entire fuzz test case from then on.

The rationale behind this is that each engine can have unique limits for
stack overflow. One test case I was looking at for example would stack
overflow at less than 1000 frames with epoch interruption enabled but
would stack overflow at more than 1000 frames with it disabled. This
means that the state after the trap started to diverge and it looked
like the engines produced different results.

While I was at it I also improved the "function call returned a trap"
case to compare traps to make sure the same trap reason popped out.

* Fix fuzzer tests

show more ...

[fuzz] Configure the `differential` target (#4773)

This change is a follow-on from #4515 to add the ability to configure
the `differential` fuzz target by limiting which engines and modules are
us

[fuzz] Configure the `differential` target (#4773)

This change is a follow-on from #4515 to add the ability to configure
the `differential` fuzz target by limiting which engines and modules are
used for fuzzing. This is incredibly useful when troubleshooting, e.g.,
when an engine is more prone to failure, we can target that engine
exclusively. The effect of this configuration is visible in the
statistics now printed out from #4739.

Engines are configured using the `ALLOWED_ENGINES` environment variable.
We can either subtract from the set of allowed engines (e.g.,
`ALLOWED_ENGINES=-v8`) or build up a set of allowed engines (e.g.,
`ALLOWED_ENGINES=wasmi,spec`), but not both at the same time.
`ALLOWED_ENGINES` only configures the left-hand side engine; the
right-hand side is always Wasmtime. When omitted, `ALLOWED_ENGINES`
defaults to [`wasmtime`, `wasmi`, `spec`, `v8`].

The generated WebAssembly modules are configured using
`ALLOWED_MODULES`. This environment variables works the same as above
but the available options are: [`wasm-smith`, `single-inst`].

show more ...

Port v8 fuzzer to the new framework (#4739)

* Port v8 fuzzer to the new framework

This commit aims to improve the support for the new "meta" differential
fuzzer added in #4515 by ensuring that a

Port v8 fuzzer to the new framework (#4739)

* Port v8 fuzzer to the new framework

This commit aims to improve the support for the new "meta" differential
fuzzer added in #4515 by ensuring that all existing differential fuzzing
is migrated to this new fuzzer. This PR includes features such as:

* The V8 differential execution is migrated to the new framework.
* `Config::set_differential_config` no longer force-disables wasm
features, instead allowing them to be enabled as per the fuzz input.
* `DiffInstance::{hash, hash}` was replaced with
`DiffInstance::get_{memory,global}` to allow more fine-grained
assertions.
* Support for `FuncRef` and `ExternRef` have been added to `DiffValue`
and `DiffValueType`. For now though generating an arbitrary
`ExternRef` and `FuncRef` simply generates a null value.
* Arbitrary `DiffValue::{F32,F64}` values are guaranteed to use
canonical NaN representations to fix an issue with v8 where with the
v8 engine we can't communicate non-canonical NaN values through JS.
* `DiffEngine::evaluate` allows "successful failure" for cases where
engines can't support that particular invocation, for example v8 can't
support `v128` arguments or return values.
* Smoke tests were added for each engine to ensure that a simple wasm
module works at PR-time.
* Statistics printed from the main fuzzer now include percentage-rates
for chosen engines as well as percentage rates for styles-of-module.

There's also a few small refactorings here and there but mostly just
things I saw along the way.

* Update the fuzzing README

show more ...

[fuzz] Add a meta-differential fuzz target (#4515)

* [fuzz] Add `Module` enum, refactor `ModuleConfig`

This change adds a way to create either a single-instruction module or a
regular (big) `was

[fuzz] Add a meta-differential fuzz target (#4515)

* [fuzz] Add `Module` enum, refactor `ModuleConfig`

This change adds a way to create either a single-instruction module or a
regular (big) `wasm-smith` module. It has some slight refactorings in
preparation for the use of this new code.

* [fuzz] Add `DiffValue` for differential evaluation

In order to evaluate functions with randomly-generated values, we needed
a common way to generate these values. Using the Wasmtime `Val` type is
not great because we would like to be able to implement various traits
on the new value type, e.g., to convert `Into` and `From` boxed values
of other engines we differentially fuzz against. This new type,
`DiffValue`, gives us a common ground for all the conversions and
comparisons between the other engine types.

* [fuzz] Add interface for differential engines

In order to randomly choose an engine to fuzz against, we expect all of
the engines to meet a common interface. The traits in this commit allow
us to instantiate a module from its binary form, evaluate exported
functions, and (possibly) hash the exported items of the instance.

This change has some missing pieces, though:
- the `wasm-spec-interpreter` needs some work to be able to create
instances, evaluate a function by name, and expose exported items
- the `v8` engine is not implemented yet due to the complexity of its
Rust lifetimes

* [fuzz] Use `ModuleFeatures` instead of existing configuration

When attempting to use both wasm-smith and single-instruction modules,
there is a mismatch in how we communicate what an engine must be able to
support. In the first case, we could use the `ModuleConfig`, a wrapper
for wasm-smith's `SwarmConfig`, but single-instruction modules do not
have a `SwarmConfig`--the many options simply don't apply. Here, we
instead add `ModuleFeatures` and adapt a `ModuleConfig` to that.
`ModuleFeatures` then becomes the way to communicate what features an
engine must support to evaluate functions in a module.

* [fuzz] Add a new fuzz target using the meta-differential oracle

This change adds the `differential_meta` target to the list of fuzz
targets. I expect that sometime soon this could replace the other
`differential*` targets, as it almost checks all the things those check.
The major missing piece is that currently it only chooses
single-instruction modules instead of also generating arbitrary modules
using `wasm-smith`.

Also, this change adds the concept of an ignorable error: some
differential engines will choke with certain inputs (e.g., `wasmi` might
have an old opcode mapping) which we do not want to flag as fuzz bugs.
Here we wrap those errors in `DiffIgnoreError` and then use a new helper
trait, `DiffIgnorable`, to downcast and inspect the `anyhow` error to
only panic on non-ignorable errors; the ignorable errors are converted
to one of the `arbitrary::Error` variants, which we already ignore.

* [fuzz] Compare `DiffValue` NaNs more leniently

Because arithmetic NaNs can contain arbitrary payload bits, checking
that two differential executions should produce the same result should
relax the comparison of the `F32` and `F64` types (and eventually `V128`
as well... TODO). This change adds several considerations, however, so
that in the future we make the comparison a bit stricter, e.g., re:
canonical NaNs. This change, however, just matches the current logic
used by other fuzz targets.

* review: allow hashing mutate the instance state

@alexcrichton requested that the interface be adapted to accommodate
Wasmtime's API, in which even reading from an instance could trigger
mutation of the store.

* review: refactor where configurations are made compatible

See @alexcrichton's
[suggestion](https://github.com/bytecodealliance/wasmtime/pull/4515#discussion_r928974376).

* review: convert `DiffValueType` using `TryFrom`

See @alexcrichton's
[comment](https://github.com/bytecodealliance/wasmtime/pull/4515#discussion_r928962394).

* review: adapt target implementation to Wasmtime-specific RHS

This change is joint work with @alexcrichton to adapt the structure of
the fuzz target to his comments
[here](https://github.com/bytecodealliance/wasmtime/pull/4515#pullrequestreview-1073247791).

This change:
- removes `ModuleFeatures` and the `Module` enum (for big and small
modules)
- upgrades `SingleInstModule` to filter out cases that are not valid for
a given `ModuleConfig`
- adds `DiffEngine::name()`
- constructs each `DiffEngine` using a `ModuleConfig`, eliminating
`DiffIgnoreError` completely
- prints an execution rate to the `differential_meta` target

Still TODO:
- `get_exported_function_signatures` could be re-written in terms of the
Wasmtime API instead `wasmparser`
- the fuzzer crashes eventually, we think due to the signal handler
interference between OCaml and Wasmtime
- the spec interpreter has several cases that we skip for now but could
be fuzzed with further work

Co-authored-by: Alex Crichton <[email protected]>

* fix: avoid SIGSEGV by explicitly initializing OCaml runtime first

* review: use Wasmtime's API to retrieve exported functions

Co-authored-by: Alex Crichton <[email protected]>

show more ...