en Copyright 2015 Java http://172.16.0.5:8080/history/wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs#45b60bd6 Start using `#[expect]` instead of `#[allow]` (#9696)* Start using `#[expect]` instead of `#[allow]`In Rust 1.81, our new MSRV, a new feature was added to Rust to use`#[expect]` to control lint levels. This new lint annotation willsilence a lint but will itself cause a lint if it doesn't actuallysilence anything. This is quite useful to ensure that annotations don'tget stale over time.Another feature is the ability to use a `reason` directive on theattribute with a string explaining why the attribute is there. Thisstring is then rendered in compiler messages if a warning or errorhappens.This commit migrates applies a few changes across the workspace:* Some `#[allow]` are changed to `#[expect]` with a `reason`.* Some `#[allow]` have a `reason` added if the lint conditionally fires (mostly related to macros).* Some `#[allow]` are removed since the lint doesn't actually fire.* The workspace configures `clippy::allow_attributes_without_reason = 'warn'` as a "ratchet" to prevent future regressions.* Many crates are annotated to allow `allow_attributes_without_reason` during this transitionary period.The end-state is that all crates should use`#[expect(..., reason = "...")]` for any lint that unconditionally firesbut is expected. The `#[allow(..., reason = "...")]` lint should be usedfor conditionally firing lints, primarily in macro-related code.The `allow_attributes_without_reason = 'warn'` level is intended to bepermanent but the transitionary`#[expect(clippy::allow_attributes_without_reason)]` crate annotationsto go away over time.* Fix adapter buildprtest:full* Fix one-core build of icache coherence* Use `allow` for missing_docsWork around rust-lang/rust#130021 which was fixed in Rust 1.83 and isn'tfixed for our MSRV at this time.* More MSRV compat List of files: /wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs Mon, 02 Dec 2024 17:19:20 +0000 Alex Crichton <alex@alexcrichton.com> http://172.16.0.5:8080/history/wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs#6ffcd4ea Improve stability for fuzz targets. (#3804)This commit improves the stability of the fuzz targets by ensuring the generated configs and modules are congruent, especially when the pooling allocator is being used. For the `differential` target, this means both configurations must use the same allocation strategy for now as one side generates the module that might not be compatible with another arbitrary config now that we fuzz the pooling allocator. These changes also ensure that constraints put on the config are more consistently applied, especially when using a fuel-based timeout. List of files: /wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs Tue, 15 Feb 2022 20:59:04 +0000 Peter Huene <peter@huene.dev> http://172.16.0.5:8080/history/wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs#ab1d845a Refactor fuzzing configuration and sometimes disable debug verifier. (#3664)* fuzz: Refactor Wasmtime's fuzz targets A recent fuzz bug found is related to timing out when compiling a module. This timeout, however, is predominately because Cranelift's debug verifier is enabled and taking up over half the compilation time. I wanted to fix this by disabling the verifier when input modules might have a lot of functions, but this was pretty difficult to implement. Over time we've grown a number of various fuzzers. Most are `wasm-smith`-based at this point but there's various entry points for configuring the wasm-smith module, the wasmtime configuration, etc. I've historically gotten quite lost in trying to change defaults and feeling like I have to touch a lot of different places. This is the motivation for this commit, simplifying fuzzer default configuration. This commit removes the ability to create a default `Config` for fuzzing, instead only supporting generating a configuration via `Arbitrary`. This then involved refactoring all targets and fuzzers to ensure that configuration is generated through `Arbitrary`. This should actually expand the coverage of some existing fuzz targets since `Arbitrary for Config` will tweak options that don't affect runtime, such as memory configuration or jump veneers. All existing fuzz targets are refactored to use this new method of configuration. Some fuzz targets were also shuffled around or reimplemented: * `compile` - this now directly calls `Module::new` to skip all the fuzzing infrastructure. This is mostly done because this fuzz target isn't too interesting and is largely just seeing what happens when things are thrown at the wall for Wasmtime. * `instantiate-maybe-invalid` - this fuzz target now skips instantiation and instead simply goes into `Module::new` like the `compile` target. The rationale behind this is that most modules won't instantiate anyway and this fuzz target is primarily fuzzing the compiler. This skips having to generate arbitrary configuration since wasm-smith-generated-modules (or valid ones at least) aren't used here. * `instantiate` - this fuzz target was removed. In general this fuzz target isn't too interesting in isolation. Almost everything it deals with likely won't pass compilation and is covered by the `compile` fuzz target, and otherwise interesting modules being instantiated can all theoretically be created by `wasm-smith` anyway. * `instantiate-wasm-smith` and `instantiate-swarm` - these were both merged into a new `instantiate` target (replacing the old one from above). There wasn't really much need to keep these separate since they really only differed at this point in methods of timeout. Otherwise we much more heavily use `SwarmConfig` than wasm-smith's built-in options. The intention is that we should still have basically the same coverage of fuzzing as before, if not better because configuration is now possible on some targets. Additionally there is one centralized point of configuration for fuzzing for wasmtime, `Arbitrary for ModuleConfig`. This internally creates an arbitrary `SwarmConfig` from `wasm-smith` and then further tweaks it for Wasmtime's needs, such as enabling various wasm proposals by default. In the future enabling a wasm proposal on fuzzing should largely just be modifying this one trait implementation. * fuzz: Sometimes disable the cranelift debug verifier This commit disables the cranelift debug verifier if the input wasm module might be "large" for the definition of "more than 10 functions". While fuzzing we disable threads (set them to 1) and enable the cranelift debug verifier. Coupled with a 20-30x slowdown this means that a module with the maximum number of functions, 100, gives: 60x / 100 functions / 30x slowdown = 20ms With only 20 milliseconds per function this is even further halved by the `differential` fuzz target compiling a module twice, which means that, when compiling with a normal release mode Wasmtime, if any function takes more than 10ms to compile then it's a candidate for timing out while fuzzing. Given that the cranelift debug verifier can more than double compilation time in fuzzing mode this actually means that the real time budget for function compilation is more like 4ms. The `wasm-smith` crate can pretty easily generate a large function that takes 4ms to compile, and then when that function is multiplied 100x in the `differential` fuzz target we trivially time out the fuzz target. The hope of this commit is to buy back half our budget by disabling the debug verifier for modules that may have many functions. Further refinements can be implemented in the future such as limiting functions for just the differential target as well. * Fix the single-function-module fuzz configuration * Tweak how features work in differential fuzzing * Disable everything for baseline differential fuzzing * Enable selectively for each engine afterwards * Also forcibly enable reference types and bulk memory for spec tests * Log wasms when compiling * Add reference types support to v8 fuzzer * Fix timeouts via fuel The default store has "infinite" fuel so that needs to be consumed before fuel is added back in. * Remove fuzzing-specific tests These no longer compile and also haven't been added to in a long time. Most of the time a reduced form of original the fuzz test case is added when a fuzz bug is fixed. List of files: /wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs Fri, 07 Jan 2022 21:12:25 +0000 Alex Crichton <alex@alexcrichton.com> http://172.16.0.5:8080/history/wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs#0642e62f Use wasm-smith to canonicalize NaN in differential fuzzing (#3195)* Update wasm-smith to 0.7.0 * Canonicalize NaN with wasm-smith for differential fuzzing This then also enables floating point executing in wasmi in addition to the spec interpreter. With NaN canonicalization at the wasm level this means that we should be producing deterministic results between Wasmtime and these alternative implementations. List of files: /wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs Tue, 17 Aug 2021 16:42:22 +0000 Alex Crichton <alex@alexcrichton.com> http://172.16.0.5:8080/history/wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs#33c3d00f Remove rss prediction from api_calls fuzzer (#3156)This functionality is now subsumed by the limiter built-in to all fuzzing stores, so there's no longer any need for it. It was also triggering arithmetic overflows in fuzzing, so instead of fixing I'm removing it! List of files: /wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs Fri, 06 Aug 2021 17:43:22 +0000 Alex Crichton <alex@alexcrichton.com> http://172.16.0.5:8080/history/wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs#bb85366a Enable simd fuzzing on oss-fuzz (#3152)* Enable simd fuzzing on oss-fuzz This commit generally enables the simd feature while fuzzing, which should affect almost all fuzzers. For fuzzers that just throw random data at the wall and see what sticks, this means that they'll now be able to throw simd-shaped data at the wall and have it stick. For wasm-smith-based fuzzers this commit also updates wasm-smith to 0.6.0 which allows further configuring the `SwarmConfig` after generation, notably allowing `instantiate-swarm` to generate modules using simd using `wasm-smith`. This should much more reliably feed simd-related things into the fuzzers. Finally, this commit updates wasmtime to avoid usage of the general `wasm_smith::Module` generator to instead use a Wasmtime-specific custom default configuration which enables various features we have implemented. * Allow dummy table creation to fail Tables might creation for imports may exceed the memory limit on the store, which we'll want to gracefully recover from and not fail the fuzzers. List of files: /wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs Thu, 05 Aug 2021 21:24:42 +0000 Alex Crichton <alex@alexcrichton.com> http://172.16.0.5:8080/history/wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs#a33caec9 Bump the wasm-tools crates (#3139)* Bump the wasm-tools crates Pulls in some updates here and there, mostly for updating crates to the latest version to prepare for later memory64 work. * Update lightbeam List of files: /wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs Wed, 04 Aug 2021 14:53:47 +0000 Alex Crichton <alex@alexcrichton.com> http://172.16.0.5:8080/history/wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs#824ce7bf deps: Update `Arbitrary` to 1.0; `libfuzzer-sys` to 0.4.0; `wasm-smith` to 0.4.0 List of files: /wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs Thu, 25 Feb 2021 22:35:47 +0000 Nick Fitzgerald <fitzgen@gmail.com> http://172.16.0.5:8080/history/wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs#8dd09121 Update wasm-tools dependenciesBrings in fixes for some assorted wast issues. List of files: /wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs Mon, 09 Nov 2020 15:03:36 +0000 Alex Crichton <alex@alexcrichton.com> http://172.16.0.5:8080/history/wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs#b73b8318 Replace binaryen -ttf based fuzzing with wasm-smith (#2336)This commit removes the binaryen support for fuzzing from wasmtime, instead switching over to `wasm-smith`. In general it's great to have what fuzzing we can, but our binaryen support suffers from a few issues: * The Rust crate, binaryen-sys, seems largely unmaintained at this point. While we could likely take ownership and/or send PRs to update the crate it seems like the maintenance is largely on us at this point. * Currently the binaryen-sys crate doesn't support fuzzing anything beyond MVP wasm, but we're interested at least in features like bulk memory and reference types. Additionally we'll also be interested in features like module-linking. New features would require either implementation work in binaryen or the binaryen-sys crate to support. * We have 4-5 fuzz-bugs right now related to timeouts simply in generating a module for wasmtime to fuzz. One investigation along these lines in the past revealed a bug in binaryen itself, and in any case these bugs would otherwise need to get investigated, reported, and possibly fixed ourselves in upstream binaryen. Overall I'm not sure at this point if maintaining binaryen fuzzing is worth it with the advent of `wasm-smith` which has similar goals for wasm module generation, but is much more readily maintainable on our end. Additonally in this commit I've added a fuzzer for wasm-smith's `SwarmConfig`-based fuzzer which should expand the coverage of tested modules. Closes #2163 List of files: /wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs Thu, 29 Oct 2020 15:02:59 +0000 Alex Crichton <alex@alexcrichton.com> http://172.16.0.5:8080/history/wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs#2c684104 Validate modules while translating (#2059)* Validate modules while translating This commit is a change to cranelift-wasm to validate each function body as it is translated. Additionally top-level module translation functions will perform module validation. This commit builds on changes in wasmparser to perform module validation interwtwined with parsing and translation. This will be necessary for future wasm features such as module linking where the type behind a function index, for example, can be far away in another module. Additionally this also brings a nice benefit where parsing the binary only happens once (instead of having an up-front serial validation step) and validation can happen in parallel for each function. Most of the changes in this commit are plumbing to make sure everything lines up right. The major functional change here is that module compilation should be faster by validating in parallel (or skipping function validation entirely in the case of a cache hit). Otherwise from a user-facing perspective nothing should be that different. This commit does mean that cranelift's translation now inherently validates the input wasm module. This means that the Spidermonkey integration of cranelift-wasm will also be validating the function as it's being translated with cranelift. The associated PR for wasmparser (bytecodealliance/wasmparser#62) provides the necessary tools to create a `FuncValidator` for Gecko, but this is something I'll want careful review for before landing! * Read function operators until EOF This way we can let the validator take care of any issues with mismatched `end` instructions and/or trailing operators/bytes. List of files: /wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs Mon, 05 Oct 2020 16:02:01 +0000 Alex Crichton <alex@alexcrichton.com> http://172.16.0.5:8080/history/wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs#1000f213 Update wasmparser to 0.59.0 (#2013)This commit is intended to update wasmparser to 0.59.0. This primarily includes bytecodealliance/wasm-tools#40 which is a large update to how parsing and validation works. The impact on Wasmtime is pretty small at this time, but over time I'd like to refactor the internals here to lean more heavily on that upstream wasmparser refactoring. For now, though, the intention is to get on the train of wasmparser's latest `main` branch to ensure we get bug fixes and such. As part of this update a few other crates and such were updated. This is primarily to handle the new encoding of `ref.is_null` where the type is not part of the instruction encoding any more. List of files: /wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs Mon, 13 Jul 2020 21:22:41 +0000 Alex Crichton <alex@alexcrichton.com> http://172.16.0.5:8080/history/wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs#c9a0ba81 Implement interrupting wasm code, reimplement stack overflow (#1490)* Implement interrupting wasm code, reimplement stack overflow This commit is a relatively large change for wasmtime with two main goals: * Primarily this enables interrupting executing wasm code with a trap, preventing infinite loops in wasm code. Note that resumption of the wasm code is not a goal of this commit. * Additionally this commit reimplements how we handle stack overflow to ensure that host functions always have a reasonable amount of stack to run on. This fixes an issue where we might longjmp out of a host function, skipping destructors. Lots of various odds and ends end up falling out in this commit once the two goals above were implemented. The strategy for implementing this was also lifted from Spidermonkey and existing functionality inside of Cranelift. I've tried to write up thorough documentation of how this all works in `crates/environ/src/cranelift.rs` where gnarly-ish bits are. A brief summary of how this works is that each function and each loop header now checks to see if they're interrupted. Interrupts and the stack overflow check are actually folded into one now, where function headers check to see if they've run out of stack and the sentinel value used to indicate an interrupt, checked in loop headers, tricks functions into thinking they're out of stack. An interrupt is basically just writing a value to a location which is read by JIT code. When interrupts are delivered and what triggers them has been left up to embedders of the `wasmtime` crate. The `wasmtime::Store` type has a method to acquire an `InterruptHandle`, where `InterruptHandle` is a `Send` and `Sync` type which can travel to other threads (or perhaps even a signal handler) to get notified from. It's intended that this provides a good degree of flexibility when interrupting wasm code. Note though that this does have a large caveat where interrupts don't work when you're interrupting host code, so if you've got a host import blocking for a long time an interrupt won't actually be received until the wasm starts running again. Some fallout included from this change is: * Unix signal handlers are no longer registered with `SA_ONSTACK`. Instead they run on the native stack the thread was already using. This is possible since stack overflow isn't handled by hitting the guard page, but rather it's explicitly checked for in wasm now. Native stack overflow will continue to abort the process as usual. * Unix sigaltstack management is now no longer necessary since we don't use it any more. * Windows no longer has any need to reset guard pages since we no longer try to recover from faults on guard pages. * On all targets probestack intrinsics are disabled since we use a different mechanism for catching stack overflow. * The C API has been updated with interrupts handles. An example has also been added which shows off how to interrupt a module. Closes #139 Closes #860 Closes #900 * Update comment about magical interrupt value * Store stack limit as a global value, not a closure * Run rustfmt * Handle review comments * Add a comment about SA_ONSTACK * Use `usize` for type of `INTERRUPTED` * Parse human-readable durations * Bring back sigaltstack handling Allows libstd to print out stack overflow on failure still. * Add parsing and emission of stack limit-via-preamble * Fix new example for new apis * Fix host segfault test in release mode * Fix new doc example List of files: /wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs Tue, 21 Apr 2020 18:03:28 +0000 Alex Crichton <alex@alexcrichton.com> http://172.16.0.5:8080/history/wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs#67bfeea1 fuzzing: Limit the total number of API calls generated (#1265)To avoid libfuzzer timeouts, limit the total number of API calls we generate in the `api_calls` fuzz target. We were already limiting the number of exported function calls we made, and this extends the limit to all API calls. List of files: /wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs Tue, 10 Mar 2020 16:28:00 +0000 Nick Fitzgerald <fitzgen@gmail.com> http://172.16.0.5:8080/history/wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs#4866fa0e Limit rayon to one thread during fuzzingThis should enable more deterministic execution. List of files: /wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs Sat, 29 Feb 2020 00:32:06 +0000 Nick Fitzgerald <fitzgen@gmail.com> http://172.16.0.5:8080/history/wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs#6e2bb9eb Limit the number of exported function calls we make in the API calls fuzzerThis should fix some fuzzing timeouts likehttps://bugs.chromium.org/p/oss-fuzz/issues/detail?id=20847 List of files: /wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs Fri, 28 Feb 2020 23:42:50 +0000 Nick Fitzgerald <fitzgen@gmail.com> http://172.16.0.5:8080/history/wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs#5ed9796e Implement `Arbitrary::size_hint` for `ApiCalls` List of files: /wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs Fri, 28 Feb 2020 23:20:00 +0000 Nick Fitzgerald <fitzgen@gmail.com> http://172.16.0.5:8080/history/wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs#0a020918 Don't let the API fuzz generator run wild (#959)We've got some OOM fuzz test cases getting reported, but these aren't very interesting. The OOMs, after some investigation, are confirmed to be happening because the test is simply allocating thousands of instances with massive tables, quickly exceeding the 2GB memory threshold for fuzzing. This isn't really interesting because this is expected behavior if you instantiate these sorts of modules. This commit updates the fuzz test case generator to have a "prediction" for each module how much memory it will take to instantiate it. This prediction is then used to avoid instantiating new modules if we predict that it will exceed our memory limit. The limits here are intentionally very squishy and imprecise. The goal here is to still generate lots of interesting test cases, but not ones that simply exhaust memory trivially. List of files: /wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs Thu, 20 Feb 2020 22:38:03 +0000 Alex Crichton <alex@alexcrichton.com> http://172.16.0.5:8080/history/wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs#adcc047f Update fuzz crates (#826)* deps: Update to arbitrary 0.3.x and libfuzzer-sys 0.2.0 * ci: Use cargo-fuzz 0.7.x in CI List of files: /wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs Thu, 16 Jan 2020 05:05:37 +0000 Nick Fitzgerald <fitzgen@gmail.com> http://172.16.0.5:8080/history/wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs#0cde3019 fuzzing: Add initial API call fuzzerWe only generate *valid* sequences of API calls. To do this, we keep track ofwhat objects we've already created in earlier API calls via the `Scope` struct.To generate even-more-pathological sequences of API calls, we use [swarmtesting]:> In swarm testing, the usual practice of potentially including all features> in every test case is abandoned. Rather, a large “swarm” of randomly> generated configurations, each of which omits some features, is used, with> configurations receiving equal resources.[swarm testing]: https://www.cs.utah.edu/~regehr/papers/swarm12.pdfThere are more public APIs and instance introspection APIs that we have thanthis fuzzer exercises right now. We will need a better generator of valid Wasmthan `wasm-opt -ttf` to really get the most out of those currently-unexercisedAPIs, since the Wasm modules generated by `wasm-opt -ttf` don't import andexport a huge variety of things. List of files: /wasmtime-44.0.1/crates/fuzzing/src/generators/api.rs Fri, 06 Dec 2019 23:48:46 +0000 Nick Fitzgerald <fitzgen@gmail.com>