PCC: remove proof-carrying code (for now?). (#12800)

In late 2023, we built out an experimental feature called
Proof-Carrying Code (PCC), where we attached "facts" to values in the
CLIF IR and built

PCC: remove proof-carrying code (for now?). (#12800)

In late 2023, we built out an experimental feature called
Proof-Carrying Code (PCC), where we attached "facts" to values in the
CLIF IR and built verification of these facts after lowering to
machine instructions. We also added "memory types" describing layout
of memory and a "checked" flag on memory operations such that we could
verify that any checked memory operation accessed valid memory (as
defined by memory types attached to pointer values via
facts). Wasmtime's Cranelift backend then put appropriate memory types
and facts in its IR such that all accesses to memory (aspirationally)
could be checked, taking the whole mid-end and lowering backend of
Cranelift out of the trusted core that enforces SFI.

This basically worked, at the time, for static memories; but never for
dynamic memories, and then work on the feature lost
prioritization (aka I had to work on other things) and I wasn't able
to complete it and put it in fuzzing/enable it as a production option.

Unfortunately since then it has bit-rotted significantly -- as we add
new backend optimizations and instruction lowerings we haven't kept
the PCC framework up to date.

Inspired by the discussion in #12497 I think it's time to delete
it (hopefully just "for now"?) unless/until we can build it again. And
when we do that, we should probably get it to the point of validating
robust operation on all combinations of memory configurations before
merging. (That implies a big experiment branch rather than a bunch of
eager PRs in-tree, but so it goes.) I still believe it is possible to
build this (and I have ideas on how to do it!) but not right now.

show more ...

Add `bulk_memory` to wast configuration (#12883)

Allows enabling/disabling this wasm proposal on a per-test basis.

Add support for map type (#12216)

* Add support for map type

Signed-off-by: Yordis Prieto <[email protected]>

* Add Map and MapEntry classes to support key/value pairs in component model

Th

Add support for map type (#12216)

* Add support for map type

Signed-off-by: Yordis Prieto <[email protected]>

* Add Map and MapEntry classes to support key/value pairs in component model

This commit introduces the Map and MapEntry classes, enabling the representation of map values in the component model. The Map class allows for the creation and iteration of key/value pairs, enhancing the functionality of the wasmtime component API. Additionally, the .gitignore file is updated to exclude build artifacts from the crates/c-api directory.

* Add wasm_component_model_map configuration support

* Format code

* Format C code

* Enhance component model to support HashMap<K, V> type

This commit introduces support for HashMap<K, V> in the component model, allowing maps to be represented as list<tuple<K, V>> in the canonical ABI. It includes implementations for the ComponentType, Lower, and Lift traits for HashMap, enabling type checking, lowering to flat representations, and lifting from memory. Additionally, the maximum depth for type generation in the fuzzing utility is updated to accommodate the new map type.

* Refactor component configuration to introduce map support

This commit removes the previous wasm features configuration and adds new functions for creating a map-configured engine. The `map_config` and `map_engine` functions are introduced to facilitate the use of the component model with maps in tests, ensuring that the engine is properly configured for map types in the component model.

* Add new WAST test for map types and remove map type definitions from existing tests

This commit introduces a new WAST test file specifically for testing various map types in the component model. Additionally, it removes the redundant map type definitions from the existing types.wast file to streamline the test suite.

* Update component fuzzing and dynamic tests to replace call_and_post_return with call

* Format code

* Refactor HashMap usage in typed.rs to use wasmtime_environ collections

* Fix HashMap initialization and insertion to handle potential errors in typed.rs

* Refactor HashMap handling in typed.rs to use lower_map_iter for improved iteration and memory management. Introduce new implementations for ComponentType, Lower, and Lift traits for std::collections::HashMap, enhancing support for map types in the component model.

* Fix map adapter trampoline compilation and alignment bugs

The translate_map function had two categories of bugs preventing map
adapter trampolines from working:

1. Wasm stack discipline: local_set_new_tmp emits LocalSet which pops
from the stack, but was called when the stack was empty (to
"pre-allocate" locals). Fixed by computing values first, then
calling local_set_new_tmp to consume them—matching translate_list's
pattern. Also removed an erroneous LocalTee that left an orphan
value on the stack. Affected: src_byte_len, dst_byte_len,
cur_src_ptr, cur_dst_ptr.

2. Pointer advancement: after value translation, the pointer still
points at the value start. The code only advanced by trailing
padding instead of value_size + trailing_padding, causing every
loop iteration to re-read the same memory.

Also fixes entry layout to use proper record alignment rules (entry
align = max(key_align, value_align), value at aligned offset).

* Refactor map entry layout calculations to use canonical ABI

* Remove unnecessary clone of map pairs during lowering

Val::Map already holds Vec<(Val, Val)> which derefs to &[(Val, Val)],
matching lower_map's signature directly. The intermediate Vec allocation
and deep clone of every key/value pair was redundant.

* Deduplicate map lift logic between HashMap implementations

* Deduplicate list and map sequence translation scaffolding

* Fix cargo fmt formatting issues

* Deduplicate map typecheck logic

* Deduplicate map lowering with linear_lower_map_to_flat and linear_lower_map_to_memory helpers

* Clean up lift_try_map: use drop, move TryHashMap import to module scope

* Fix CI: arbtest overflow and no-std HashMap lift_map

- component_fuzz: use saturating_sub in generate_hashable_key to prevent
underflow when fuel is 0 and Enum variant is chosen
- typed: remove incorrect ? operators in lift_map for hashbrown::HashMap
(with_capacity and insert don't return Result)

* Store map tuple layout in TypeMap

Compute map entry ABI and value offsets once during type building, and reuse that metadata in runtime map lift/lower paths instead of recalculating tuple layout at each call site.

* Refactor map ABI argument passing

Bundle map lift/lower layout and type metadata into a small MapAbi32 helper so map helper calls stay concise without changing behavior.

* Fix CI: enable component_model_map in fuzzing and handle map in arbitrary_val

The fuzzer's component_api oracle was generating map types but the engine
didn't have the map feature enabled, and arbitrary_val had no arm for
Type::Map. Enable component_model_map in the store helper (matching how
component_model_async is forced on) and implement arbitrary value generation
for map types.

---------

Signed-off-by: Yordis Prieto <[email protected]>

show more ...

fuzz: Enable winch-aarch64 (#12646)

As of https://github.com/bytecodealliance/wasmtime/pull/11051, Winch
supports all of Core Wasm on aarch64. This commit enables fuzzing
respectively.

Minimal implementation of fixed-length lists to enable wit-bindgen runtime tests (#10619)

* necessary parts to make the wit-bindgen test pass

* post-merge fixes

* fix clippy

* inline type computa

Minimal implementation of fixed-length lists to enable wit-bindgen runtime tests (#10619)

* necessary parts to make the wit-bindgen test pass

* post-merge fixes

* fix clippy

* inline type computations

* link missing functionality to an issue

* undo unintended removal of blank line

* fix logic mistake, enable feature in wasmtime

* optimized conversion

* add offset

* unify on name and implement wast option

* cargo fmt and beautify fixed_size_list_type

* add multi memory as it is needed by the test runner

* standardize on the official name (component model standard) "fixed-length lists"

changing wasm-tools (parser etc. is a different task)

* More instances of fixed size corrected to fixed length

show more ...

Migrate fuzzing to `wasmtime::error` (#12263)

* Migrate fuzzing to `wasmtime::error`

* fix

"Downgrade" threads support to tier 2, disable fuzzing (#12036)

* "Downgrade" threads support to tier 2, disable fuzzing

This commit is borne out of a fuzz bug that was opened recently. The
fuzz bu

"Downgrade" threads support to tier 2, disable fuzzing (#12036)

* "Downgrade" threads support to tier 2, disable fuzzing

This commit is borne out of a fuzz bug that was opened recently. The
fuzz bug specifically has to do with fallout from #12022, specifically
`SharedMemory` being used to allocated instead of `Memory`. In this
situation the resource limiter is no longer consulted meaning that
shared memories bypass this and aren't caught by OOM checks. This is
currently by design because `SharedMemory` instances don't know which
resource limiter to hook into per-store.

More generally though the implementation of wasm threads, while workable
in Wasmtime, has a number of known relatively large deficiencies. These
were not resolved prior to ungating the wasm proposal (that's on me) but
nevertheless the quality of implementation is not quite up to "tier 1
par" with the rest of what Wasmtime offers. Given this the threads
proposal is now downgraded to tier 2. To help minimize the impact of
this the wasm proposal is left enabled-by-default, but creation of a
`SharedMemory` in the Rust API requires opting-in via a new
`Config::shared_memory` method.

This commit shuffles around some documentation of wasm proposals to
split it into tier 1/2/3 instead of on/off-by-default and then adds a
column for whether the proposal is on-by-default.

* clangformat

* Fix tests

* Add tests for failed creation

Fix an issue where defined shared memories weren't gated

* Sync disabled threads stub

* Fix another test

prtest:full

* Fix fuzzing tests

* Fix dwarf tests

show more ...

Disable mach ports during fuzzing (#11990)

Reported [here] and confirmed locally for myself it appears that
sometimes exceptions bypass our mach port thread and go to signal
handlers instead. I've g

Disable mach ports during fuzzing (#11990)

Reported [here] and confirmed locally for myself it appears that
sometimes exceptions bypass our mach port thread and go to signal
handlers instead. I've got no idea why myself and it seems
non-deterministic. Instead of trying to bottom this out I've gone ahead
and just disabled mach ports during fuzzing. Mach ports are mainly there
to play nice with Breakpad and such which isn't a concern during
fuzzing, so using signals should work just as well.

[here]: https://github.com/bytecodealliance/wasmtime/issues/11850#issuecomment-3493237271

show more ...

Cooperative Multithreading (#11751)

* Initial work

* Almost compiling

* Partially working

* Cleanup

* Fix merge

* Cancellation and suspension refactoring

* Remove printlns

* Test with several

Cooperative Multithreading (#11751)

* Initial work

* Almost compiling

* Partially working

* Cleanup

* Fix merge

* Cancellation and suspension refactoring

* Remove printlns

* Test with several threads

* More testing

* Cancellation

* Fix cancellation for explicit suspends

* Finish cancellation test

* Store threads in the instance table

* Deletion almost there

* Tests all pass

* Tighten up task deletion

* Set thread state correctly

* Store pairs of thread and task ids

* Remove lift abi members

* Cleanup unnecessary change

* More cleanup

* Cleanup

* Revert cargo changes

* Revert cargo changes

* Comments

* Comments on test

* Update comments

* Update comments

* Add space that was removed in an earlier commit

* Cleanup

* Delete threads from the instance table

* Revert cargo file

* Remove dead code

* Clippy changes

* Clippy

* Revert unnecessary changes

* Revert unnecessary changes

* Revert unnecessary changes

* Revert unnecessary changes

* Revert unnecessary changes

* Review comments

* Review feedback

* Make thread IDs per-component-instance

* Fix config

* Tighten up completion

* Clippy

* Trigger full PR test

* Move funcref table reading

* Remove unused import

* Formatting

* Rename RemoveOnDrop

* Review feedback

* Review feedback

* Move start thread closure

* Review feedback

* Review feedback

* Correct feature for import

* Review feedback

* Disable failing tests

* Enable fixed tests

* Review feedback

* Readd tests

* Ignore task deletion test with Miri

show more ...

Update upstream wasm spec test suite submodule (#11926)

* Update upstream wasm spec test suite submodule

Juggle some configuration options here and there to accommodate all of
the new tests.

* Fix

Update upstream wasm spec test suite submodule (#11926)

* Update upstream wasm spec test suite submodule

Juggle some configuration options here and there to accommodate all of
the new tests.

* Fix wasmtime-fuzzing compile

* Fix miri script for table.wast

* Fix a miri provenance issue

show more ...

Disable single-pass regalloc in fuzzing and add docs (#11864)

This commit disables the single-pass algorithm when fuzzing due
to #11850 which has discovered a few issues with it. Additionally
docume

Disable single-pass regalloc in fuzzing and add docs (#11864)

This commit disables the single-pass algorithm when fuzzing due
to #11850 which has discovered a few issues with it. Additionally
documentation has been added to the configuration option indicating that
it's not recommended for security-sensitive applications to clarify its
implementation status.

show more ...

docs: fix typo (#11509)

* Update store.rs

* Update types.rs

* Update config.rs

Re-enable single-pass regalloc in fuzzing (#11721)

Follow-up to #11712

Put each component in its own store in wast testing (#11686)

This commit refactors the `wasmtime-wast` crate to place each component
in its own store. This is in contrast to core wasm testing where

Put each component in its own store in wast testing (#11686)

This commit refactors the `wasmtime-wast` crate to place each component
in its own store. This is in contrast to core wasm testing where all
instances get placed in the same store. The reason for this is that
components, in particular async state, is only fully defined so long as
a trap doesn't happen and once a trap happens it's expected the entire
store is torn down. This removes the need for the store to keep track of
exactly which instance all async tasks are associated with to ensure
that once any of them trap only the precise set of tasks necessary are
torn down. Instead Wasmtime can rely on embedders throwing away
`Store<T>` entirely.

The refactoring here provides a hook in `WastContext` to configure the
store but otherwise a new store is manufactured for all component
instances. Core wasm is unaffected. Tests are also unaffected as
same-store behavior isn't actually required by any tests.

show more ...

Disable `SinglePass` regalloc when fuzzing (#11546)

OSS-Fuzz found #11544 and #11545 overnight so disable fuzzing for now
until those are fixed to avoid hindering too much progress while fuzzing.

Pull in new regalloc2 with fastalloc fixes for exceptions, and re-enable and add to testing. (#11533)

* Revert "Cranelift/Wasmtime: disable fastalloc (single-pass) allocator for now. (#10554)"

This

Pull in new regalloc2 with fastalloc fixes for exceptions, and re-enable and add to testing. (#11533)

* Revert "Cranelift/Wasmtime: disable fastalloc (single-pass) allocator for now. (#10554)"

This reverts commit d52e23b09191185996792b8ef18e5fca2865ca43.

* Upgrade to regalloc2 0.13.1.

Pulls in bytecodealliance/regalloc2#233 to update fastalloc to support
the looser constraints needed by exception-related changes.

* cargo-vet update.

show more ...

Exceptions: add basic support for fuzzing. (#11511)

* Enable use of exceptions in our fuzz module generator.

This patch was provided by Alex in #11505; it is thus

Co-authored-by: Alex Crichton <al

Exceptions: add basic support for fuzzing. (#11511)

* Enable use of exceptions in our fuzz module generator.

This patch was provided by Alex in #11505; it is thus

Co-authored-by: Alex Crichton <[email protected]>

* Add ExnRef comparison to differential result checking.

---------

Co-authored-by: Alex Crichton <[email protected]>

show more ...

WebAssembly exception-handling support. (#11326)

* WebAssembly exception-handling support.

This PR introduces support for the [Wasm exception-handling proposal],
which introduces a conventional try

WebAssembly exception-handling support. (#11326)

* WebAssembly exception-handling support.

This PR introduces support for the [Wasm exception-handling proposal],
which introduces a conventional try/catch mechanism to WebAssembly. The
PR supports modules that use `try_table` to register handlers for a
lexical scope; and provides `throw` and `throw_ref` that allocate (in
the first case) and throw exception objects.

This PR builds on top of the work in #10510 for Cranelift-level
exception support, #10919 for an unwinder, and #11230 for exception
objects built on top of GC, in addition a bunch of smaller fix and
enabling PRs around those.

[Wasm exception-handling proposal]: https://github.com/WebAssembly/exception-handling/

prtest:full

* Permit UnwindToWasm to have unused fields in Pulley builds (for now).

* Resolve miri-caught reborrowing issue.

* Ignore exceptions tests in miri for now (Pulley not supported).

* Use wasmtime_test on exceptions tests.

* Get tests passing on pulley platforms

* Add a check to `supports_host` for the generated test and assert
failure also when that is false.
* Remove `pulley_unsupported` test as it falls out of `#[wasmtime_test]`
* Remove `exceptions_store` helper as it falls out of `#[wasmtime_test]`
* Remove miri annotations as they fall out of `#[wasmtime_test]`

* Remove dead import

* Skip some unsupported tests entirely in `#[wasmtime_test]`

If the selected compiler doesn't support the host at all then there's no
need to run it. Actually running it could misinterpret `CraneliftNative`
as "run with pulley" otherwise, so avoid such false negatives.

* Cranelift: dynamic contexts: account for outgoing-args area.

---------

Co-authored-by: Alex Crichton <[email protected]>

show more ...

Add a configuration knob for `PAGEMAP_SCAN` (#11433)

* Add a configuration knob for `PAGEMAP_SCAN`

This commit adds `PoolingAlloationConfig::pagemap_scan` and additionally
adds `-Opooling-pagemap-s

Add a configuration knob for `PAGEMAP_SCAN` (#11433)

* Add a configuration knob for `PAGEMAP_SCAN`

This commit adds `PoolingAlloationConfig::pagemap_scan` and additionally
adds `-Opooling-pagemap-scan` to configure on the CLI. This is the same
tri-state configuration option as `MpkEnable` so that enum was renamed
to just `Enabled` and repurposed for both options.

This then additionally turns the option off-by-default instead of the
previous on-by-default-if-able-to to enable more slowly rolling out this
feature.

* Fix broken test

show more ...

Wasmtime: Add (optional) bottom-up function inlining to Wasm compilation (#11283)

* Wasmtime: Add (optional) bottom-up function inlining to Wasm compilation

This commit plumbs together two pieces o

Wasmtime: Add (optional) bottom-up function inlining to Wasm compilation (#11283)

* Wasmtime: Add (optional) bottom-up function inlining to Wasm compilation

This commit plumbs together two pieces of recently-added infrastructure:

1. function inlining in Cranelift, and
2. the parallel bottom-up inlining scheduler in Wasmtime.

Sprinkle some very simple inlining heuristics on top, and this gives us function
inlining in Wasm compilation.

The default Wasmtime configuration does not enable inlining, and when we do
enable it, we only enable it for cross-component calls by default (since
presumably the toolchain that produced a particular core Wasm module, like LLVM,
already performed any inlining that was beneficial within that module, but that
toolchain couldn't know how that Wasm module would be getting linked together
with other modules via component composition, and so it could not have done any
cross-component inlining). For what it is worth, there is a config knob to
enable intra-module function inlining, but this is primarily for use by our
fuzzers, so that they can easily excercise and explore this new inlining
functionality.

All this plumbing required some changes to the `wasmtime_environ::Compiler`
trait, since Winch cannot do inlining but Cranelift can. This is mostly
encapsulated in the new `wasmtime_environ::InliningCompiler` trait, for the most
part. Additionally, we take care not to construct the call graph, or any other
data structures required only by the inliner and not regular compilation, both
when using Winch and when using Cranelift with inlining disabled.

Finally, we add a `disas` test to verify that we successfully inline a series of
calls from a function in one component, to a cross-component adapter function,
to a function in another component. Most test coverage is expected to come from
our fuzzing, however.

* Fix dead code warning when not `cfg(feature = "component-model")`

* fix winch trampoline compilation

* Move CLI options to codegen

* Move parameters into struct

* Use an index set for call-graph construction

* Smuggle inlining heuristic options through cranelift flags

* Remove old CLI flags

* set tunables before settings

* Only configure inlining options for cranelift in fuzzing

show more ...

Increase minimum available stacks when fuzzing wast tests (#11263)

This fixes a fuzz failure with the new async tests which require more
than one stack.

Remove "unaligned" memory from fuzzing (#11147)

This commit removes the configuration knob used during fuzzing to use a
custom host memory which is always unaligned on the host. The original
intenti

Remove "unaligned" memory from fuzzing (#11147)

This commit removes the configuration knob used during fuzzing to use a
custom host memory which is always unaligned on the host. The original
intention for this was to help catch issues related to alignment in the
x64 backend about, for example, performing unaligned SSE loads correctly
(as opposed to accidentally faulting on unaligned addresses).

As shown in the test failures of #11142, however, this is technically UB
for other parts of Wasmtime that assume the heap addresses are always
aligned. For example Wasmtime will create safe references in to a GC
heap and GC heaps are also allocated with this same allocator, meaning
that Rust-safe references are unaligned (which is UB).

In practice I'm not aware of any actual issues this configuration option
has ever discovered, and we've otherwise discovered alignment issues via
normal fuzzing as well. Given that I think it's best to just jettison
this entirely and stop trying to support it and/or tweak configuration
to only use it when supported or similar.

show more ...

Propagate the `component-model-gc` feature from the fuzz config to `wasmtime::Config` (#11087)

Fixes https://oss-fuzz.com/testcase-detail/6524398096154624

Initial component model and GC support in fused adapters (#11020)

* Initial component model and GC support in fused adapters

This lays down the initial infrastructure for support for GC in our fuse

Initial component model and GC support in fused adapters (#11020)

* Initial component model and GC support in fused adapters

This lays down the initial infrastructure for support for GC in our fused
adapters for the component model. We keep track of whether each lifted/lowered
function wants args/results as GC values or in linear memory. We additionally
plumb through the core function types of the functions being lifted and lowered
for (eventual) use with GC adapters.

Ultimately, this commit is enough to fuse together lifted and lowered functions
where one or both are using the GC variant of the canonical ABI. Attempting to
actually pass arguments will hit `todo!()`s. The work of implementing those
`todo!()`s is left to future commits.

* Address review feedback

* More review feedback

* Fix wasmtime-fuzzing build

* Fix the `factc` example program

* cargo fmt

* Fix fact-valid-module fuzzer

show more ...

winch(aarch64) Run integration tests in winch-aarch64 (#11031)

* winch(aarch64) Run integration tests in winch-aarch64

This commit closes
https://github.com/bytecodealliance/wasmtime/issues/8321

W

winch(aarch64) Run integration tests in winch-aarch64 (#11031)

* winch(aarch64) Run integration tests in winch-aarch64

This commit closes
https://github.com/bytecodealliance/wasmtime/issues/8321

With https://github.com/bytecodealliance/wasmtime/pull/11013, Winch
passes all spec tests for Core Wasm on Aarch64.

* Run `cargo fmt`

* Add an `integration` option to `TestConfig`

`wast::TestConfig` is shared across spec tests and integration tests,
this commits adds a flag to identify if the config is for integration
test purposes making it easier to contextually decide if tests are
expected to fail or pass depending on the Wasm configuration options.

* Use `spec_test` instead of `integration`

To better differentiate spec tests from integration tests

* Ensure only aarch64 / x64 when testing Winch

* Revert directive for typed_v128 test

Put back `#[wasmtime_test(wasm_features(simd),
strategies(not(Winch)))]`; the reason why this tests shouldn't run
with Winch is because it requires AVX, but we can have a generic rule
at the test config level since not all SIMD based tests require AVX,
it really depends on which SIMD instructions are used.

show more ...