Support store-access in `bindgen!` generated imports (#11628)

* Support store-access in `bindgen!` generated imports

This commit adds support to accessing the store in `bindgen!`-generated
import

Support store-access in `bindgen!` generated imports (#11628)

* Support store-access in `bindgen!` generated imports

This commit adds support to accessing the store in `bindgen!`-generated
import functions in traits. Since the inception of `bindgen!` this has
never been possible and access to the store requires manually working
with `Linker`, for example. This is not easy to do because it requires
surgically editing code or working around what bindings generation parts
you do want.

The implementation here is a small step away from what
component-model-async has already implemented for async functions.
Effectively it's a small extension of the `*WithStore` traits to also
have synchronous functions with `Access` parameters instead of `async`
functions with an `Accessor` parameter.

This is something we're going to want for the WASIp3 implementation
where I've noticed some resource destructors are going to want access to
the store to close out streams and such and this'll provide the bindings
necessary for that.

Closes #11590

* Update test expectations

show more ...

Enable `allow_attributes_without_reason` (#11195)

* Enable `allow_attributes_without_reason`

This commit enables the `clippy::allow_attributes_without_reason` for
the `wasmtime` crate which previou

Enable `allow_attributes_without_reason` (#11195)

* Enable `allow_attributes_without_reason`

This commit enables the `clippy::allow_attributes_without_reason` for
the `wasmtime` crate which previously forcibly allowed it. The reason
this was allowed was that when the workspace was first migrated the
Wasmtime crate had too many instances that I was willing to fix. I've
now come back around and tried to fix everything.

In short: ideally delete `#[allow]`, otherwise use `#[expect]`,
otherwise use `#[allow]`.

prtest:full

* Adjust some directives

* Fix some warnings

* Fix stack switching size tests on unix

* Don't have a conditional `Drop` impl

* Force `testing_freelist` method to be used

Too lazy to write `#[cfg]`, but not too lazy to write a test.

show more ...

Replace `GetHost` with a function pointer, add `HasData` (#10770)

* Replace `GetHost` with a function pointer, add `HasData`

This commit is a refactoring to the fundamentals of the `bindgen!` macro

Replace `GetHost` with a function pointer, add `HasData` (#10770)

* Replace `GetHost` with a function pointer, add `HasData`

This commit is a refactoring to the fundamentals of the `bindgen!` macro
and the functions that it generates. Prior to this change the
fundamental entrypoint generated by `bindgen!` was a function
`add_to_linker_get_host` which takes a value of type `G: GetHost`. This
`GetHost` implementation is effectively an alias for a closure whose
return value is able to close over the parameter given lfietime-wise.

The `GetHost` abstraction was added to Wasmtime originally to enable
using any type that implements `Host` traits, not just `&mut U` as was
originally supported. The definition of `GetHost` was _just_ right to
enable a type such as `MyThing<&mut T>` to implement `Host` and a
closure could be provided that could return it. At the time that
`GetHost` was added it was known to be problematic from an
understandability point of view, namely:

* It has a non-obvious definition.
* It's pretty advanced Rust voodoo to understand what it's actually
doing
* Using `GetHost` required lots of `for<'a> ...` in places which is
unfamiliar syntax for many.
* `GetHost` values couldn't be type-erased (e.g. put in a trait object)
as we couldn't figure out the lifetime syntax to do so.

Despite these issues it was the only known solution at hand so we landed
it and kept the previous `add_to_linker` style (`&mut T -> &mut U`) as a
convenience. While this has worked reasonable well (most folks just try
to not look at `GetHost`) it has reached a breaking point in the WASIp3
work.

In the WASIp3 work it's effectively now going to be required that the
`G: GetHost` value is packaged up and actually stored inside of
accessors provided to host functions. This means that `GetHost` values
now need to not only be taken in `add_to_linker` but additionally
provided to the rest of the system through an "accessor". This was made
possible in #10746 by moving the `GetHost` type into Wasmtime itself (as
opposed to generated code where it lived prior).

While this worked with WASIp3 and it was possible to plumb `G: GetHost`
safely around, this ended up surfacing more issues. Namely all
"concurrent" host functions started getting significantly more
complicated `where` clauses and type signatures. At the end of the day I
felt that we had reached the end of the road to `GetHost` and wanted to
search for alternatives, hence this change.

The fundamental purpose of `GetHost` was to be able to express, in a
generic fashion:

* Give me a closure that takes `&mut T` and returns `D`.
* The `D` type can close over the lifetime in `&mut T`.
* The `D` type must implement `bindgen!`-generated traits.

A realization I had was that we could model this with a generic
associated type in Rust. Rust support for generic associated types is
relatively new and not something I've used much before, but it ended up
being a perfect model for this. The definition of the new `HasData`
trait is deceptively simple:

trait HasData {
type Data<'a>;
}

What this enables us to do though is to generate `add_to_linker`
functions that look like this:

fn add_to_linker<T, D>(
linker: &mut Linker<T>,
getter: fn(&mut T) -> D::Data<'_>,
) -> Result<()>
where
D: HasData,
for<'a> D::Data<'a>: Host;

This definition here models `G: GetHost` as a literal function pointer,
and the ability to close over the `&mut T` lifetime with type (not just
`&mut U`) is expressed through the type constructor `type Data<'a>`).
Ideally we could take a generic generic associated type (I'm not even
sure what to call that), but that's not something Rust has today.

Overall this felt like a much simpler way of modeling `GetHost` and its
requirements. This plumbed well throughout the WASIp3 work and the
signatures for concurrent functions felt much more appropriate in terms
of complexity after this change. Taking this change to the limit means
that `GetHost` in its entirety could be purged since all usages of it
could be replaced with `fn(&mut T) -> D::Data<'a>`, a hopefully much
more understandable type.

This change is not all rainbows however, there are some gotchas that
remain:

* One is that all `add_to_linker` generated functions have a `D:
HasData` type parameter. This type parameter cannot be inferred and
must always be explicitly specified, and it's not easy to know what to
supply here without reading documentation. Actually supplying the type
parameter is quite easy once you know what to do (and what to fill
in), but it may involve defining a small struct with a custom
`HasData` implementation which can be non-obvious.

* Another is that the `G: GetHost` value was previously a full Rust
closure, but now it's transitioning to a function pointer. This is
done in preparation for WASIp3 work where the function needs to be
passed around, and doing that behind a generic parameter is more
effort than it's worth. This means that embedders relying on the true
closure-like nature here will have to update to using a function
pointer instead.

* The function pointer is stored in locations that require `'static`,
and while `fn(T)` might be expected to be `'static` regardless of `T`
is is, in fact, not. This means that practically `add_to_linker`
requires `T: 'static`. Relative to just before this change this is a
possible regression in functionality, but there orthogonal reasons
beyond just this that we want to start requiring `T: 'static` anyway.
That means that this isn't actually a regression relative to #10760, a
related change.

The first point is partially ameliorated with WASIp3 work insofar that
the `D` type parameter will start serving as a location to specify where
concurrent implementations are found. These concurrent methods don't
take `&mut self` but instead are implemented for `T: HasData` types. In
that sense it's more justified to have this weird type parameter, but in
the meantime without this support it'll feel a bit odd to have this
little type parameter hanging off the side.

This change has been integrated into the WASIp3 prototyping repository
with success. This has additionally been integrated into the Spin
embedding which has one of the more complicated reliances on
`*_get_host` functions known. Given that it's expected that while this
is not necessarily a trivial change to rebase over it should at least be
possible.

Finally the `HasData` trait here has been included with what I'm hoping
is a sufficient amount of documentation to at least give folks a spring
board to understand it. If folks have confusion about this `D` type
parameter my hope is they'll make their way to `HasData` which showcases
various patterns for "librarifying" host implementations of WIT
interfaces. These patterns are all used throughout Wasmtime and WASI
currently in crates and tests and such.

* Update expanded test expectations

show more ...

Update mod.rs (#10492)

Fix some more minor 2024 edition things (#9978)

* Fix some doctests to be compatible with the 2024 edition.
* Fix a `use<...>` that's an error in the 2024 edition but works in the
2021 edition.

Add async example to bindgen_examples (#9822)

* add async example

* fix bindgen bug to make CI happy

See https://github.com/bytecodealliance/wasmtime/pull/9822#issuecomment-2546851949

* update te

Add async example to bindgen_examples (#9822)

* add async example

* fix bindgen bug to make CI happy

See https://github.com/bytecodealliance/wasmtime/pull/9822#issuecomment-2546851949

* update test expectations

* remove async_trait

show more ...

Redesign how component exports work (#8786)

* Un-nest exports in a component

This commit flattens the representation of exports in a component to
make them more easily indexable without forcing tra

Redesign how component exports work (#8786)

* Un-nest exports in a component

This commit flattens the representation of exports in a component to
make them more easily indexable without forcing traversal through the
hierarchy of instance imports/exports to get there.

* Guarantee type information on component exports

Don't have it optional in some cases and present in others, instead
ensure there's type information for all component exports immediately
available.

* Refactor how component instance exports are loaded

This commit is a change to Wasmtime's public API for
`wasmtime::component::Instance` that reorganizes how component exports
are loaded. Previously there was a system where `Instance::exports()`
was called that that was sort of "iterated over" in a builder-style
pattern to acquire the actual export desired. This required lifetime
trickery for nested instances and some unfortunate API bloat. The major
downside of this approach is that it requires unconditional string
lookups at runtime for exports and additionally does not serve as a
great place to implement the semver-compatible logic of #8395. The goal
of this refactoring is to pave the way to improving this.

The new APIs for loading exports now look a bit more similar to what's
available for core modules. Notably there's a new
`Component::export_index` method which enables performing a string
lookup and returning an index. This index can in turn be passed to
`Instance::get_*` to skip the string lookup when exports are loaded. The
`Instance::exports` API is then entirely removed and dismantled.

The only piece remaining is the ability to load nested exports which is
done through an `Option` parameter to `Component::export_index`. The
way to load a nested instance is now to first lookup the instance with
`None` as this parameter an then the instance itself is `Some` to look
up an export of that instance. This removes the need for a
recursive-style lifetime-juggling API from wasmtime and in theory helps
simplify the usage of loading exports.

* Update `bindgen!` generated structures for exports

This commit updates the output of `bindgen!` to have a different setup
for exports of worlds to handle the changes from the previous commit.
This introduces new `*Pre` structures which are generated alongside the
existing `Guest` structures for example. The `*Pre` versions contain
`ComponentExportIndex` from the previous commit and serve as a path to
accelerating instantiation because all name lookups are skipped.

* Update test expectations for `bindgen!`-generated output

* Review comments

* Fix doc link

show more ...

Wasmtime: Implement the custom-page-sizes proposal (#8763)

* Wasmtime: Implement the custom-page-sizes proposal

This commit adds support for the custom-page-sizes proposal to Wasmtime:
https://gith

Wasmtime: Implement the custom-page-sizes proposal (#8763)

* Wasmtime: Implement the custom-page-sizes proposal

This commit adds support for the custom-page-sizes proposal to Wasmtime:
https://github.com/WebAssembly/custom-page-sizes

I've migrated, fixed some bugs within, and extended the `*.wast` tests for this
proposal from the `wasm-tools` repository. I intend to upstream them into the
proposal shortly.

There is a new `wasmtime::Config::wasm_custom_page_sizes_proposal` method to
enable or disable the proposal. It is disabled by default.

Our fuzzing config has been updated to turn this feature on/off as dictated by
the arbitrary input given to us from the fuzzer.

Additionally, there were getting to be so many constructors for
`wasmtime::MemoryType` that I added a builder rather than add yet another
constructor.

In general, we store the `log2(page_size)` rather than the page size
directly. This helps cut down on invalid states and properties we need to
assert.

I've also intentionally written this code such that supporting any power of two
page size (rather than just the exact values `1` and `65536` that are currently
valid) will essentially just involve updating `wasmparser`'s validation and
removing some debug asserts in Wasmtime.

* Update error string expectation

* Remove debug logging

* Use a right shift instead of a division

* fix error message expectation again

* remove page size from VMMemoryDefinition

* fix size of VMMemoryDefinition again

* Only dynamically check for `-1` sentinel for 1-byte page sizes

* Import functions that are used a few times

* Better handle overflows when rounding up to the host page size

Propagate errors instead of returning a value that is not actually a rounded up
version of the input.

Delay rounding up various config sizes until runtime instead of eagerly doing it
at config time (which isn't even guaranteed to work, so we already had to have a
backup plan to round up at runtime, since we might be cross-compiling wasm or
not have the runtime feature enabled).

* Fix some anyhow and nostd errors

* Add missing rounding up to host page size at runtime

* Add validate feature to wasmparser dep

* Add some new rounding in a few places, due to no longer rounding in config methods

* Avoid actually trying to allocate the whole address space in the `massive_64_bit_still_limited` test

The point of the test is to ensure that we hit the limiter, so just cancel the
allocation from the limiter, and otherwise avoid MIRI attempting to allocate a
bunch of memory after we hit the limiter.

* prtest:full

* Revert "Avoid actually trying to allocate the whole address space in the `massive_64_bit_still_limited` test"

This reverts commit ccfa34a78dd3d53e49a6158ca03077d42ce8bcd7.

* miri: don't attempt to allocate more than 4GiB of memory

It seems that rather than returning a null pointer from `std::alloc::alloc`,
miri will sometimes choose to simply crash the whole program.

* remove duplicate prelude import after rebasing

show more ...

Overhaul and improve documentation of `bindgen!` (#8727)

This is another take at improving the documentation for `bindgen!` in
Wasmtime. This commit takes a leaf out of the book of
bytecodealliance/

Overhaul and improve documentation of `bindgen!` (#8727)

This is another take at improving the documentation for `bindgen!` in
Wasmtime. This commit takes a leaf out of the book of
bytecodealliance/wit-bindgen#871 to organize the documentation of the
macro a bit more rather than having one giant doc block that can be
difficult to explore. The macro's documentation itself is now mostly a
reference of all the options that can be specified. There is now a new
documentation-only module which serves a few purposes:

* Individual examples are organized per-submodule to be a bit more
digestable.
* Each example has an example of the generated code in addition to the
source code used for each example.
* All examples are tested on CI to compile (none are run).

My hope is that this makes it easier to expand the docs here further
over time with niche features as they arise or with various options that
the macro has. This is one of the lynchpins of Wasmtime's support for
the component model so it seems pretty important to have a good
onboarding experience here.

Along the way I've implemented a few more niche options for the
`bindgen!` macro that I found necessary, such as configuring the
`wasmtime` crate and where it's located.

show more ...