Do not re-export `anyhow!` in the `wasmtime::prelude` (#12298)

We are trying to move to `format_err!` instead.

Add some more metrics to the pooling allocator (#11789)

* Add some more metrics to the pooling allocator

This commit adds a few more metrics to the `PoolingAllocatorMetrics`
type along the lines of

Add some more metrics to the pooling allocator (#11789)

* Add some more metrics to the pooling allocator

This commit adds a few more metrics to the `PoolingAllocatorMetrics`
type along the lines of accounting for more items as well as the unused
slots in the pooling allocator. Notably the count of unused memory and
table slots is exposed along with the number of bytes which are kept
resident in these slots despite them not being in use. This involved a
bit of plumbing to thread around the number of bytes that are actually
kept resident to some more locations but is otherwise a pretty
straightforward plumbing of accounting information we already had
internally.

* Fix configured build

* Add some docs

* Only run new test on Linux

* Try to fix ASAN

prtest:full

* Shrink pooling size in tests

show more ...

Remove unsoundness of widening store borrows (#11481)

* Remove unsoundness of widening store borrows

This commit removes preexisting unsoundness in Wasmtime where a
`&mut StoreOpaque` borrow was "

Remove unsoundness of widening store borrows (#11481)

* Remove unsoundness of widening store borrows

This commit removes preexisting unsoundness in Wasmtime where a
`&mut StoreOpaque` borrow was "widened" into encompassing the limiter on
the `T` in `StoreInner<T>`, for example, by using the self-pointer
located in an instance or the store. This fix is done by threading
`&mut StoreOpaque` as a parameter separately from a
`StoreResourceLimiter`. This means that various callers now take a new
`Option<&mut StoreResourceLimiter<'_>>` parameter in various locations.

Closes #11409

* Fix gc-less build

show more ...

Make table/memory creation async functions (#11470)

* Make core instance allocation an `async` function

This commit is a step in preparation for #11430, notably core instance
allocation, or `Store

Make table/memory creation async functions (#11470)

* Make core instance allocation an `async` function

This commit is a step in preparation for #11430, notably core instance
allocation, or `StoreOpaque::allocate_instance` is now an `async fn`.
This function does not actually use the `async`-ness just yet so it's a
noop from that point of view, but this propagates outwards to enough
locations that I wanted to split this off to make future changes more
digestable.

Notably some creation functions here such as making an `Instance`,
`Table`, or `Memory` are refactored internally to use this new `async`
function. Annotations of `assert_ready` or `one_poll` are used as
appropriate as well.

For reference this commit was benchmarked with our `instantiation.rs`
benchmark in the pooling allocator and shows no changes relative to the
original baseline from before-`async`-PRs.

* Make table/memory creation `async` functions

This commit is a large-ish refactor which is made possible by the many
previous refactorings to internals w.r.t. async-in-Wasmtime. The end
goal of this change is that table and memory allocation are both `async`
functions. Achieving this, however, required some refactoring to enable
it to work:

* To work with `Send` neither function can close over `dyn VMStore`.
This required changing their `Option<&mut dyn VMStore>` arugment to
`Option<&mut StoreResourceLimiter<'_>>`
* Somehow a `StoreResourceLimiter` needed to be acquired from an
`InstanceAllocationRequest`. Previously the store was stored here as
an unsafe raw pointer, but I've refactored this now so
`InstanceAllocationRequest` directly stores `&StoreOpaque` and
`Option<&mut StoreResourceLimiter>` meaning it's trivial to acquire
them. This additionally means no more `unsafe` access of the store
during instance allocation (yay!).
* Now-redundant fields of `InstanceAllocationRequest` were removed since
they can be safely inferred from `&StoreOpaque`. For example passing
around `&Tunables` is now all gone.
* Methods upwards from table/memory allocation to the
`InstanceAllocator` trait needed to be made `async`. This includes new
`#[async_trait]` methods for example.
* `StoreOpaque::ensure_gc_store` is now an `async` function. This
internally carries a new `unsafe` block carried over from before with
the raw point passed around in `InstanceAllocationRequest`. A future
PR will delete this `unsafe` block, it's just temporary.

I attempted a few times to split this PR up into separate commits but
everything is relatively intertwined here so this is the smallest
"atomic" unit I could manage to land these changes and refactorings.

* Shuffle `async-trait` dep

* Fix configured build

show more ...

Use RAII guards in a few locations instead of closures (#11459)

This represents no functional difference from what happens today but in
the future all of these functions will become `async` function

Use RAII guards in a few locations instead of closures (#11459)

This represents no functional difference from what happens today but in
the future all of these functions will become `async` functions where
RAII guards will then be required, so this goes ahead and switches them
to RAII guards to split out the diff.

show more ...

Add support for the Linux PAGEMAP_SCAN ioctl (#11372)

* WIP: use the pagemap_scan ioctl to selectively reset an instance's dirty pages

* Less hacky, and supporting tables, too

* Bugfixes

* WIP: m

Add support for the Linux PAGEMAP_SCAN ioctl (#11372)

* WIP: use the pagemap_scan ioctl to selectively reset an instance's dirty pages

* Less hacky, and supporting tables, too

* Bugfixes

* WIP: memcpy instead of pread

* Refactor support for pagemap

* Don't hold a raw pointer to the original data, plumb through an `Arc`
to have a safe reference instead.
* Update pagemap bindings to latest version of abstraction written.
* Include pagemap-specific tests.
* Use a `PageMap` structure created once-per-pool instead of a
static-with-a-file.
* Refactor to use the "pagemap path" unconditionally which blends in the
keep_resident bits.

* Improve safety documentation

prtest:full

* Fix some lints

* Skip ioctl tests when it's not supported

* Fix a memory leak by moving impls around

* Fix no vm build

* Review comments

* Add more pagemap-specific documentation

* Add more docs, refactor implementation slightly

* Improve `category_*` docs

Basically forward to the Linux kernel source itself.

* Fix compile

* Make pagemap integration resilient across forks

* Fix non-pooling-allocator-build

* Fix portability issues of new test

* Actually use config on macos

---------

Co-authored-by: Till Schneidereit <[email protected]>

show more ...

Reset fewer bytes when resetting tables (#11341)

This commit changes the resetting of tables back to all-null in
`TablePool::reset_table_pages_to_zero`. Previously the full capacity of
the table was

Reset fewer bytes when resetting tables (#11341)

This commit changes the resetting of tables back to all-null in
`TablePool::reset_table_pages_to_zero`. Previously the full capacity of
the table was reset back to zero, depending on the configuration of the
pooling allocator. Now the table's dynamic size is taken into account as
the size of the allocation to reset. This should help resetting fewer
bytes and making `keep_resident` options more applicable for example
because tiny tables, as can show up in components, don't need a large
memset followed by a decommit and can instead just use a tiny memset.

show more ...

Deny `unsafe_op_in_unsafe_fn` in `wasmtime::vm::instance` (#11267)

Turn the lint on and add some safety comments where appropriate.

cc #11180

Enable `allow_attributes_without_reason` (#11195)

* Enable `allow_attributes_without_reason`

This commit enables the `clippy::allow_attributes_without_reason` for
the `wasmtime` crate which previou

Enable `allow_attributes_without_reason` (#11195)

* Enable `allow_attributes_without_reason`

This commit enables the `clippy::allow_attributes_without_reason` for
the `wasmtime` crate which previously forcibly allowed it. The reason
this was allowed was that when the workspace was first migrated the
Wasmtime crate had too many instances that I was willing to fix. I've
now come back around and tried to fix everything.

In short: ideally delete `#[allow]`, otherwise use `#[expect]`,
otherwise use `#[allow]`.

prtest:full

* Adjust some directives

* Fix some warnings

* Fix stack switching size tests on unix

* Don't have a conditional `Drop` impl

* Force `testing_freelist` method to be used

Too lazy to write `#[cfg]`, but not too lazy to write a test.

show more ...

Fix Typos in Documentation Comments (#11140)

* Update table_pool.rs

* Update unix.rs

* Update interpreter.rs

Stack switching: Infrastructure and runtime support (#10388)

* [pr1] base

* prtest:full

* make sure to use ControlFlow result in trace_suspended_continuation

* stack-switching: cleanup: remove st

Stack switching: Infrastructure and runtime support (#10388)

* [pr1] base

* prtest:full

* make sure to use ControlFlow result in trace_suspended_continuation

* stack-switching: cleanup: remove stray c-api changes

These are remnants of unrelated wasmfx wasmtime experiments, possibly
suitable for later submission against upstream.

* stack-switching: reuse async_stack_size

* stack-switching: delete delete_me debugging

* stack-switching: address feedback in environ::types

* stack-switching: remove unused code from vmoffsets

* stack-switching: drop dependency on std

* stack-switching: add compilation checks to ci matrix

* stack-switching: remove debug_println cruft

* stack-switching: export environ consts consistently

* stack-switching: export vm pub items consistently

* table_pool: reduced capacity for large elements

VMContRef elements which takes up two words and we don't want to
double the size of all tables in order to support storing these.
This change changes the table to target storing the requested
max number of elements if they are "nominally" sized with
(potentially) reduced capacity for non-nominally sized types when
encountered.

Continuations are the only type of element which may result in
fewer table slots being available than requested.

* stack-switching: extend conditional compilation

A fair bit of the definitions for stack switching are still
enabled, but this patch takes things a bit further to avoid
compilation problems; notably, cont_new is now not compiled
in unless the feature is enabled.

* stack-switching: formatting fixes

* stack-switching: address new clippy checks

In addition, to get clippy to fully pass, plumbed in
additional config to make winch paths happy; there's no
impl for winch yet but plumbing through the feature is
required to make paths incorporating macros at various
layers satisfied (and it is expected we'll use the
features in the future).

* stack-switching: more conditional compilation fixes

* stack-switching: additional conditional compile on table builtins for continuations

* stack-switching: additional conditional compile fixes

* stack-switching: additional conditional compile in store

* stack-switching: remove overly strict assertion

* stack-switching: remove errantly dropped no_mangle in config c-api

* stack-switching: VMContObj::from_raw_parts

* stack-switching: remove duplicate async_stack_size feature check

* stack-switching: VMArray -> VMHostArray

* stack-switching: remove unnecessary clippy exception

* stack-switching: fix docs referenced VMRuntimeLimits

* stack-switching: fix doc typo

* stack-switching: follow recommendations for type casts

* stack-switching: use usize::next_multiple_of

* stack-switching: update outdated comment

* stack-switching: use feature gate instead of allow(dead_code)

* stack-switching: rework backtrace using chunks/zip

* stack-switching: move tests to footer module

This is a bit more consistent with the prevailing style
in tree and (subjectively) makes finding the tests
as a reader more straightforward.

Tests left unchanged sans some import cleanup.

* stack-swictchding: verify stack_chain offsets at runtime

* fixup! stack-switching: use feature gate instead of allow(dead_code)

* stack-switching: document continuation roots tracing using match arms

---------

Co-authored-by: Paul Osborne <[email protected]>

show more ...

Enable the `useless_conversion` Clippy lint (#10838)

* Enable the `useless_conversion` Clippy lint

We've got lots of types in Wasmtime and convert between them quite a
lot, but often over time conv

Enable the `useless_conversion` Clippy lint (#10838)

* Enable the `useless_conversion` Clippy lint

We've got lots of types in Wasmtime and convert between them quite a
lot, but often over time conversions become unnecessary through
refactorings or similar. This will hopefully enable us to clean up some
conversions as they come up to try to have as few as possible ideally.

* Review comments

show more ...

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtim

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtime to the 2024 Edition to keep
up-to-date with Rust idioms and access many of the edition features
exclusive to the 2024 edition.

prtest:full

* Reformat with the 2024 edition

show more ...

Test Pulley on CI on 32-bit architectures (#9745)

This commit extends our CI for i686 and armv7 to test the Pulley
backend, namely the full `*.wast` test suite as well as the `wasmtime`
crate itself

Test Pulley on CI on 32-bit architectures (#9745)

This commit extends our CI for i686 and armv7 to test the Pulley
backend, namely the full `*.wast` test suite as well as the `wasmtime`
crate itself. Note that many `*.wast` tests are still expected to fail
at this time.

This involved fixing a number of 32-vs-64 bit issues throughout the test
suite in various location in this commit.

show more ...

Rely on `core::error::Error` (#9702)

* Rely on `core::error::Error`

With Wasmtime's new MSRV at 1.81 this means that `core::error::Error` is
available which means that in `no_std` mode the `Error`

Rely on `core::error::Error` (#9702)

* Rely on `core::error::Error`

With Wasmtime's new MSRV at 1.81 this means that `core::error::Error` is
available which means that in `no_std` mode the `Error` trait can be
used. This has been integrated into `anyhow::Error` already upstream and
means that we can remove our own local hacks such as the `Err2Anyhow` trait.

This commit removes the `Err2Anyhow` trait and all usage, going back to
idiomatic Rust error propagation and conversion even in the `no_std`
world. This should make code more portable by default and remove some
weird idioms we had for supporting this.

prtest:full

* Add some trusted vets

* Audit object crate update

* Disable backtraces on CI

show more ...

Move memory pool over to aligned byte counts (#9668)

* Move memory pool over to aligned byte counts

Part of work to centralize memory management within Mmap instances -- some of
that work becomes e

Move memory pool over to aligned byte counts (#9668)

* Move memory pool over to aligned byte counts

Part of work to centralize memory management within Mmap instances -- some of
that work becomes easier if the byte counts are known to be aligned.

There were a few overflow cases that I added checks for.

* Address review comments

* Fix unused code warnings

show more ...

Add HostAlignedByteCount to enforce alignment at compile time (#9620)

* Add HostAlignedByteCount to enforce alignment at compile time

As part of the work to allow mmaps to be backed by other implem

Add HostAlignedByteCount to enforce alignment at compile time (#9620)

* Add HostAlignedByteCount to enforce alignment at compile time

As part of the work to allow mmaps to be backed by other implementations, I
realized that we didn't have any way to track whether a particular usize is
host-page-aligned at compile time.

Add a `HostAlignedByteCount` which tracks that a particular usize is aligned to
the host page size. This also does not expose safe unchecked arithmetic
operations, to ensure that overflows always error out.

With `HostAlignedByteCount`, a lot of runtime checks can go away thanks to the
type-level assertion.

In the interest of keeping the diff relatively small, I haven't converted
everything over yet. More can be converted over as time permits.

* Make zero-sized mprotects a no-op, add tests

show more ...

Add a type-level distinction between aligned and possibly-unaligned Mmaps (#9639)

In the wasmtime codebase there are two kinds of mmaps: those that are always
backed by anonymous memory and are alwa

Add a type-level distinction between aligned and possibly-unaligned Mmaps (#9639)

In the wasmtime codebase there are two kinds of mmaps: those that are always
backed by anonymous memory and are always aligned, and those that are possibly
file-backed and so the length might be unaligned. In
https://github.com/bytecodealliance/wasmtime/pull/9620 I accidentally mixed the
two up in a way that was a ticking time bomb.

To resolve this, add a type parameter to `Mmap` to distinguish between the
cases. All of wasmtime's users are clearly either one or the other, so it's
quite simple in the end.

show more ...

Remove the `wasmtime_environ::TablePlan` type (#9530)

* Remove the `wasmtime_environ::TablePlan` type

In my quest to simplify memory configuration and how things work
internally in Wasmtime one thi

Remove the `wasmtime_environ::TablePlan` type (#9530)

* Remove the `wasmtime_environ::TablePlan` type

In my quest to simplify memory configuration and how things work
internally in Wasmtime one thing I've identified to accomplish is the
removal of the `TablePlan` and `MemoryPlan` types. These introduce an
abstraction layer between table/memory implementations and `Tunables`
and personally I find it simpler to directly reference `Tunables` and
use that instead. The goal of this commit is to plumb `Tunables` closer
to where it's directly read by removing the "indirection" through the
`*Plan` types.

The `TablePlan` and `MemoryPlan` types are pervasively used throughout
Wasmtime so instead of having one large commit delete everything this is
instead a piecemeal approach to incrementally get towards the goal of
removal. Here just `TablePlan` is removed and `Tunables` is plumbed in a
few more places. I plan to also in the future remove `TableStyle` and
`MemoryStyle` in favor of directly reading `Tunables` but that's left
for a future commit. For now `TableStyle` persists and its usage is a
bit odd in isolation but I plan to follow this up with the removal of
`TableStyle`.

* Fix non-component-model build

show more ...

Enable a few miscellaneous Clippy lints (#9325)

* Enable `clippy::unnecessary_mut_passed`

This looks like it's good at catching refactoring issues where a mutable
borrow was once required but is no

Enable a few miscellaneous Clippy lints (#9325)

* Enable `clippy::unnecessary_mut_passed`

This looks like it's good at catching refactoring issues where a mutable
borrow was once required but is no longer necessary.

* Enable `clippy::unnecessary_fallible_conversions` by default

This can be useful because `From`-based conversions are typically more
ergonomic than fallible ones and additionally it can help make it more
clear that no error is possible in certain contexts.

* Enable `clippy:unnecessary_cast`

Looks to be useful for helping to clean up after refactorings.

* Avoid a transmute in int-to-float conversions

* Fix some lints from a rebase

show more ...

Implement the table64 extension to the memory64 proposal (#9206)

This commit implements the table64 extention in both Wasmtime and
Cranelift.

Most of the work was changing a bunch of u32 values to

Implement the table64 extension to the memory64 proposal (#9206)

This commit implements the table64 extention in both Wasmtime and
Cranelift.

Most of the work was changing a bunch of u32 values to u64/usize.
The decisions were made in align with the PR #3153 which
implemented the memory64 propsal itself.

One significant change was the introduction of `IndexType`
and `Limits` which streamline and unify the handling of limits
for both memories and tables.

The spec and fuzzing tests related to table64 are re-enabled which
provides a good coverage of the feature.

show more ...

Add `anyhow` stuff to our internal `wasmtime` crate prelude (#8804)

* Add `anyhow` stuff to our internal `wasmtime` crate prelude

We use it basically everywhere and it is annoying to have to import

Add `anyhow` stuff to our internal `wasmtime` crate prelude (#8804)

* Add `anyhow` stuff to our internal `wasmtime` crate prelude

We use it basically everywhere and it is annoying to have to import.

I also did an audit of existing `use` statements and removed the now-redundant
ones and replaced one-off imports with usage of the prelude, so that the prelude
is available by default in more places.

* Fix `cargo doc`

show more ...

Wasmtime: Implement the custom-page-sizes proposal (#8763)

* Wasmtime: Implement the custom-page-sizes proposal

This commit adds support for the custom-page-sizes proposal to Wasmtime:
https://gith

Wasmtime: Implement the custom-page-sizes proposal (#8763)

* Wasmtime: Implement the custom-page-sizes proposal

This commit adds support for the custom-page-sizes proposal to Wasmtime:
https://github.com/WebAssembly/custom-page-sizes

I've migrated, fixed some bugs within, and extended the `*.wast` tests for this
proposal from the `wasm-tools` repository. I intend to upstream them into the
proposal shortly.

There is a new `wasmtime::Config::wasm_custom_page_sizes_proposal` method to
enable or disable the proposal. It is disabled by default.

Our fuzzing config has been updated to turn this feature on/off as dictated by
the arbitrary input given to us from the fuzzer.

Additionally, there were getting to be so many constructors for
`wasmtime::MemoryType` that I added a builder rather than add yet another
constructor.

In general, we store the `log2(page_size)` rather than the page size
directly. This helps cut down on invalid states and properties we need to
assert.

I've also intentionally written this code such that supporting any power of two
page size (rather than just the exact values `1` and `65536` that are currently
valid) will essentially just involve updating `wasmparser`'s validation and
removing some debug asserts in Wasmtime.

* Update error string expectation

* Remove debug logging

* Use a right shift instead of a division

* fix error message expectation again

* remove page size from VMMemoryDefinition

* fix size of VMMemoryDefinition again

* Only dynamically check for `-1` sentinel for 1-byte page sizes

* Import functions that are used a few times

* Better handle overflows when rounding up to the host page size

Propagate errors instead of returning a value that is not actually a rounded up
version of the input.

Delay rounding up various config sizes until runtime instead of eagerly doing it
at config time (which isn't even guaranteed to work, so we already had to have a
backup plan to round up at runtime, since we might be cross-compiling wasm or
not have the runtime feature enabled).

* Fix some anyhow and nostd errors

* Add missing rounding up to host page size at runtime

* Add validate feature to wasmparser dep

* Add some new rounding in a few places, due to no longer rounding in config methods

* Avoid actually trying to allocate the whole address space in the `massive_64_bit_still_limited` test

The point of the test is to ensure that we hit the limiter, so just cancel the
allocation from the limiter, and otherwise avoid MIRI attempting to allocate a
bunch of memory after we hit the limiter.

* prtest:full

* Revert "Avoid actually trying to allocate the whole address space in the `massive_64_bit_still_limited` test"

This reverts commit ccfa34a78dd3d53e49a6158ca03077d42ce8bcd7.

* miri: don't attempt to allocate more than 4GiB of memory

It seems that rather than returning a null pointer from `std::alloc::alloc`,
miri will sometimes choose to simply crash the whole program.

* remove duplicate prelude import after rebasing

show more ...

Use bytes for maximum size of linear memory with pooling (#8628)

* Use bytes for maximum size of linear memory with pooling

This commit changes configuration of the pooling allocator to use a
byte-

Use bytes for maximum size of linear memory with pooling (#8628)

* Use bytes for maximum size of linear memory with pooling

This commit changes configuration of the pooling allocator to use a
byte-based unit rather than a page based unit. The previous
`PoolingAllocatorConfig::memory_pages` configuration option configures
the maximum size that a linear memory may grow to at runtime. This is an
important factor in calculation of stripes for MPK and is also a
coarse-grained knob apart from `StoreLimiter` to limit memory
consumption. This configuration option has been renamed to
`max_memory_size` and documented that it's in terms of bytes rather than
pages as before.

Additionally the documented constraint of `max_memory_size` must be
smaller than `static_memory_bound` is now additionally enforced as a
minor clean-up as part of this PR as well.

* Review comments

* Fix benchmark build

show more ...

Fix some typos (#8641)

* occurred

* winch typos

* tests typos

* cli typos

* fuzz typos

* examples typos

* docs typos

* crates/wasmtime typos

* crates/environ typos

* crates/cranelift typos

Fix some typos (#8641)

* occurred

* winch typos

* tests typos

* cli typos

* fuzz typos

* examples typos

* docs typos

* crates/wasmtime typos

* crates/environ typos

* crates/cranelift typos

* crates/test-programs typos

* crates/c-api typos

* crates/cache typos

* crates other typos

* cranelift/codegen/src/isa typos

* cranelift/codegen/src other typos

* cranelift/codegen other typos

* cranelift other typos

* ci js typo

* .github workflows typo

* RELEASES typo

* Fix clang-format documentation line

---------

Co-authored-by: Andrew Brown <[email protected]>

show more ...