f6775a33 - Replace `GetHost` with a function pointer, add `HasData` (#10770)

Tue, 13 May 2025 05:47:01 +0000

Replace `GetHost` with a function pointer, add `HasData` (#10770)* Replace `GetHost` with a function pointer, add `HasData`This commit is a refactoring to the fundamentals of the `bindgen!` macroand the functions that it generates. Prior to this change thefundamental entrypoint generated by `bindgen!` was a function`add_to_linker_get_host` which takes a value of type `G: GetHost`. This`GetHost` implementation is effectively an alias for a closure whosereturn value is able to close over the parameter given lfietime-wise.The `GetHost` abstraction was added to Wasmtime originally to enableusing any type that implements `Host` traits, not just `&mut U` as wasoriginally supported. The definition of `GetHost` was _just_ right toenable a type such as `MyThing<&mut T>` to implement `Host` and aclosure could be provided that could return it. At the time that`GetHost` was added it was known to be problematic from anunderstandability point of view, namely:* It has a non-obvious definition.* It's pretty advanced Rust voodoo to understand what it's actually doing* Using `GetHost` required lots of `for<'a> ...` in places which is unfamiliar syntax for many.* `GetHost` values couldn't be type-erased (e.g. put in a trait object) as we couldn't figure out the lifetime syntax to do so.Despite these issues it was the only known solution at hand so we landedit and kept the previous `add_to_linker` style (`&mut T -> &mut U`) as aconvenience. While this has worked reasonable well (most folks just tryto not look at `GetHost`) it has reached a breaking point in the WASIp3work.In the WASIp3 work it's effectively now going to be required that the`G: GetHost` value is packaged up and actually stored inside ofaccessors provided to host functions. This means that `GetHost` valuesnow need to not only be taken in `add_to_linker` but additionallyprovided to the rest of the system through an "accessor". This was madepossible in #10746 by moving the `GetHost` type into Wasmtime itself (asopposed to generated code where it lived prior).While this worked with WASIp3 and it was possible to plumb `G: GetHost`safely around, this ended up surfacing more issues. Namely all"concurrent" host functions started getting significantly morecomplicated `where` clauses and type signatures. At the end of the day Ifelt that we had reached the end of the road to `GetHost` and wanted tosearch for alternatives, hence this change.The fundamental purpose of `GetHost` was to be able to express, in ageneric fashion:* Give me a closure that takes `&mut T` and returns `D`.* The `D` type can close over the lifetime in `&mut T`.* The `D` type must implement `bindgen!`-generated traits.A realization I had was that we could model this with a genericassociated type in Rust. Rust support for generic associated types isrelatively new and not something I've used much before, but it ended upbeing a perfect model for this. The definition of the new `HasData`trait is deceptively simple: trait HasData { type Data<'a>; }What this enables us to do though is to generate `add_to_linker`functions that look like this: fn add_to_linker( linker: &mut Linker, getter: fn(&mut T) -> D::Data<'_>, ) -> Result<()> where D: HasData, for<'a> D::Data<'a>: Host;This definition here models `G: GetHost` as a literal function pointer,and the ability to close over the `&mut T` lifetime with type (not just`&mut U`) is expressed through the type constructor `type Data<'a>`).Ideally we could take a generic generic associated type (I'm not evensure what to call that), but that's not something Rust has today.Overall this felt like a much simpler way of modeling `GetHost` and itsrequirements. This plumbed well throughout the WASIp3 work and thesignatures for concurrent functions felt much more appropriate in termsof complexity after this change. Taking this change to the limit meansthat `GetHost` in its entirety could be purged since all usages of itcould be replaced with `fn(&mut T) -> D::Data<'a>`, a hopefully muchmore understandable type.This change is not all rainbows however, there are some gotchas thatremain:* One is that all `add_to_linker` generated functions have a `D: HasData` type parameter. This type parameter cannot be inferred and must always be explicitly specified, and it's not easy to know what to supply here without reading documentation. Actually supplying the type parameter is quite easy once you know what to do (and what to fill in), but it may involve defining a small struct with a custom `HasData` implementation which can be non-obvious.* Another is that the `G: GetHost` value was previously a full Rust closure, but now it's transitioning to a function pointer. This is done in preparation for WASIp3 work where the function needs to be passed around, and doing that behind a generic parameter is more effort than it's worth. This means that embedders relying on the true closure-like nature here will have to update to using a function pointer instead.* The function pointer is stored in locations that require `'static`, and while `fn(T)` might be expected to be `'static` regardless of `T` is is, in fact, not. This means that practically `add_to_linker` requires `T: 'static`. Relative to just before this change this is a possible regression in functionality, but there orthogonal reasons beyond just this that we want to start requiring `T: 'static` anyway. That means that this isn't actually a regression relative to #10760, a related change.The first point is partially ameliorated with WASIp3 work insofar thatthe `D` type parameter will start serving as a location to specify whereconcurrent implementations are found. These concurrent methods don'ttake `&mut self` but instead are implemented for `T: HasData` types. Inthat sense it's more justified to have this weird type parameter, but inthe meantime without this support it'll feel a bit odd to have thislittle type parameter hanging off the side.This change has been integrated into the WASIp3 prototyping repositorywith success. This has additionally been integrated into the Spinembedding which has one of the more complicated reliances on`*_get_host` functions known. Given that it's expected that while thisis not necessarily a trivial change to rebase over it should at least bepossible.Finally the `HasData` trait here has been included with what I'm hopingis a sufficient amount of documentation to at least give folks a springboard to understand it. If folks have confusion about this `D` typeparameter my hope is they'll make their way to `HasData` which showcasesvarious patterns for "librarifying" host implementations of WITinterfaces. These patterns are all used throughout Wasmtime and WASIcurrently in crates and tests and such.* Update expanded test expectations List of files: /wasmtime-44.0.1/crates/wasmtime/src/runtime/component/has_data.rs

Changes in has_data.rs

f6775a33 - Replace `GetHost` with a function pointer, add `HasData` (#10770)