| a631d20a | 04-Sep-2025 |
Paul Osborne <[email protected]> |
cranelift: stack-switching support (#11003)
* cranelift: stack-switching support
This initial commit represents the "pr2" base commit with minimal merge conflicts resolved. Due to OOB conflicts, t
cranelift: stack-switching support (#11003)
* cranelift: stack-switching support
This initial commit represents the "pr2" base commit with minimal merge conflicts resolved. Due to OOB conflicts, this commit is not functional as-is, but using it as a base in order to allow for easier reviewing of the delta from this commit to what will be used for the PR against upstream.
Co-authored-by: Daniel Hillerström <[email protected]> Co-authored-by: Paul Osborne <[email protected]>
* cranelift: stack-switching updates pass 1
This first set of changes updates the base pr in order to compiled and pass basic checks (compile, clippy, fmt) with the biggest part of the change being to eliminate injection of tracing/assertions in JIT'ed code.
* cranelift: stack-switching: restore original visibility for a few func_environ members
* cranelift: stack-switching conditional compilation
At this point, the only bit we really branch on is what we do in order to avoid problems tying into wasmtime_environ. This is basd on the approach and macro used by the gc code for converting presence/absence of the cranelift feature flag to cranelift compile time. This is a bit of a half-measure for now as we still compile most stack-switching code in cranelift, but this does enough to avoid causing problems with missing definitions in wasmtime_environ.
* cranelift: avoid "as" casts in stack-switching
Replace either with infallible From or fallible, panicing TryFrom alternatives where required.
* cranelift: cleanup stack-switching control_effect signatures
After removing emission of runtime trace logging and assertions, there were several unused parameters. Remove those from the ControlEffect signatures completely.
* cranelift: rename stack-switching VMArray to VMHostArray
This matches a change to the mirrored runtime type in the upstream changes.
* stack-switching: fix typo
Co-authored-by: Daniel Hillerström <[email protected]>
* stack-switching: used Index impl for get_stack_slot_data
* stack-switching: use smallvec over vec in several cases
* stack-switching: avoid resumetable naming confusion
* stack-switching: cleanup unused params from unchecked_get_continuation
The extra parameters here used to be used for emitting runtime assertions, but with those gone we just had unused params and lifetimes, clean those out.
* stack_switching: simplify store_data_entries assertion
* stack-switching: simplify translate_table_{grow,fill} control flow
* stack-switching: remove translate_resume_throw stub
There's already a stub elsewhere and this is not called, when exceptions are added and it is time to revisit, this method can be restored.
* stack-switching: compute control_context_size based on target triple
* stack-switching: VMHostArrayRef updates
Rename VMHostArray -> VMHostArrayRef Change impl to compute address with offset upfront rather than on each load.
* stack-switching: move cranelift code to live under func_environ
This matches the directory structure for gc and aids in visibility for a few members required by stack-switching code in cranelift.
* stack-switching: formatting fix
* stack-switching: reduce visibility on a few additional items
* stack-switching: simplify contobj fatptr con/de-struction
* stack-switching: add disas tests to cover new instructions
* stack-switching: fix layout of VMContObj
In the course of the various runtime updates, the layout of the runtime VMContObj got switched around. This resulted in failures when doing certain table operations on continuations.
This change fixes that layout problem and adds some tests with offsets to avoid the problem. Due to the way that we interact with the VMContObj in cranelift, we don't use these offsets outside of the tests.
* Fix formatting of merge conflict resolution
* cranelift: remove ir::function::get_stack_slot_data
This method isn't required as sized_stack_slots is already pub.
* stack-switching: reduce visibility of a couple func_environ methods
* stack-switching: define VMContObj as two words
This change migrates VMContObj and its usages in cranelift and runtime to work with the VMContObj fat pointer as two words in order to better target different architectures (still gated to x86_64 for now).
To support this, a size type was plumbed into the builtins function signature types (as is done for component types) that maps to usize.
* fixup! stack-switching: define VMContObj as two words
* stack-switching: add stub Val::ContRef
This type is not fully complete until continuation/gc integration is revisited (#10248) but without these changes, test cases are now failing on panics as we need some representation of continuation references in the runtime Val enumeration.
Runtime errors with TODO notes are added for the stubbed code paths to revisit later.
* fixup! stack-switching: add stub Val::ContRef
* fixup! stack-switching: add stub Val::ContRef
* fixup! stack-switching: define VMContObj as two words
prtest:full
* stack-switching: don't conflate host and target pointer sizes
Disas tests were failing on i686 targeting x86_64 as the size of the host pointer was leaking into what we were using to do codegen in a few paths. This patch is a bit of a hack as it seems like using a generic <T> for T: *mut u8 (as an example) is a bit questionable. To keep things small, I do a hacky typecheck to map pointers to the target pointer size here.
* stack-switching: VMHostArray entry sizes based off env PtrSize
Revisiting the previous commit with an approach that should be less brittle.
---------
Co-authored-by: Frank Emrich <[email protected]> Co-authored-by: Daniel Hillerström <[email protected]>
show more ...
|
| e1f50aad | 21-Aug-2025 |
Alex Crichton <[email protected]> |
Make table/memory creation async functions (#11470)
* Make core instance allocation an `async` function
This commit is a step in preparation for #11430, notably core instance allocation, or `Store
Make table/memory creation async functions (#11470)
* Make core instance allocation an `async` function
This commit is a step in preparation for #11430, notably core instance allocation, or `StoreOpaque::allocate_instance` is now an `async fn`. This function does not actually use the `async`-ness just yet so it's a noop from that point of view, but this propagates outwards to enough locations that I wanted to split this off to make future changes more digestable.
Notably some creation functions here such as making an `Instance`, `Table`, or `Memory` are refactored internally to use this new `async` function. Annotations of `assert_ready` or `one_poll` are used as appropriate as well.
For reference this commit was benchmarked with our `instantiation.rs` benchmark in the pooling allocator and shows no changes relative to the original baseline from before-`async`-PRs.
* Make table/memory creation `async` functions
This commit is a large-ish refactor which is made possible by the many previous refactorings to internals w.r.t. async-in-Wasmtime. The end goal of this change is that table and memory allocation are both `async` functions. Achieving this, however, required some refactoring to enable it to work:
* To work with `Send` neither function can close over `dyn VMStore`. This required changing their `Option<&mut dyn VMStore>` arugment to `Option<&mut StoreResourceLimiter<'_>>` * Somehow a `StoreResourceLimiter` needed to be acquired from an `InstanceAllocationRequest`. Previously the store was stored here as an unsafe raw pointer, but I've refactored this now so `InstanceAllocationRequest` directly stores `&StoreOpaque` and `Option<&mut StoreResourceLimiter>` meaning it's trivial to acquire them. This additionally means no more `unsafe` access of the store during instance allocation (yay!). * Now-redundant fields of `InstanceAllocationRequest` were removed since they can be safely inferred from `&StoreOpaque`. For example passing around `&Tunables` is now all gone. * Methods upwards from table/memory allocation to the `InstanceAllocator` trait needed to be made `async`. This includes new `#[async_trait]` methods for example. * `StoreOpaque::ensure_gc_store` is now an `async` function. This internally carries a new `unsafe` block carried over from before with the raw point passed around in `InstanceAllocationRequest`. A future PR will delete this `unsafe` block, it's just temporary.
I attempted a few times to split this PR up into separate commits but everything is relatively intertwined here so this is the smallest "atomic" unit I could manage to land these changes and refactorings.
* Shuffle `async-trait` dep
* Fix configured build
show more ...
|
| aa91737e | 19-Aug-2025 |
Alex Crichton <[email protected]> |
Relax the `Send` bound on fiber creation (#11454)
* Relax the `Send` bound on fiber creation
This commit removes the need for `Send` for all usages of fiber-related bits. The goal is that the futu
Relax the `Send` bound on fiber creation (#11454)
* Relax the `Send` bound on fiber creation
This commit removes the need for `Send` for all usages of fiber-related bits. The goal is that the future returned from `async` functions is `Send` if all the inputs are `Send`, but there's no actual need to require that in the function signatures themselves. This property was identified in upcoming work to make more internals of Wasmtime `async` where `async` functions are going to be used to implement the synchronous path through Wasmtime where `Send` isn't a bound today, nor ideally do we want to have it there.
The way this commit works is:
* First `make_fiber` now has a `Send` bound which makes it sound. Before #11444 it took `&mut dyn VMStore` which is never `Send` and the refactoring there missed adding this bound. This change required a few `Send` bounds in `concurrent.rs` where it's expected to have no impact as everything there is basically already `Send`.
* Next `make_fiber_unchecked` is added which is the same as `make_fiber` except without a `Send` bound. The `on_fiber` function is then updated to use this `*_unchecked` variant. This means that fibers can now be used with non-`Send` stores. Crucially though the structure of internals in play means that future produced is still only `Send` if `T: Send` meaning that there is no loss in safety.
* Next some `Send` bounds were dropped throughout async functions in Wasmtime as a proof-of-concept to require these changes to `on_fiber`. This means that these entrypoints into Wasmtime no longer require `Send`, but still naturally require `Send` if the result future is sent to various threads.
* Finally a new doctest is added to `Instance::new_async` with a `compile_fail` example to ensure that this fails. This is unfortunately a very brittle test because all we can assert is that compilation failed, not why compilation failed. That means that this is likely to regress over time, but it's the best we can do at this time.
Note that the goal here is NOT to remove `Send` bounds throughout Wasmtime. Many of them are still required, for example all the ones related to `Linker`. The realization is that we can remove some of the "edge" bounds on entrypoints where the resulting future can be conditionally `Send` depending on `T` meaning we don't actually have to write down any bounds ourselves.
This refactoring will enable future refactorings to have sync-and-async functions use the same internals and neither entrypoint needs to have a `Send` bound.
* Review comments
show more ...
|
| aef5eeb5 | 12-Aug-2025 |
Alex Crichton <[email protected]> |
Refactor internals of table initialization and management (#11416)
* Refactor const-eval to use `Val`, not `ValRaw`
This commit refactors the evaluation of constant expressions during instantiation
Refactor internals of table initialization and management (#11416)
* Refactor const-eval to use `Val`, not `ValRaw`
This commit refactors the evaluation of constant expressions during instantiation for example to use `Val` instead of `ValRaw`. Previously the usage of `ValRaw` meant that wasm was disallowed from performing a GC during const evaluation, but currently constant expressions can indeed perform a GC. The goal of this commit is to lift this limitation. This is expected to be a minor slowdown for modules that hit this path, but most modules shouldn't hit this in a hot loop since LLVM doesn't generate modules that use this branch of const eval.
The usage of `Val` brings a number of benefits and refactorings associated with it:
* Const-evaluation is generally safer than before since everything is higher-level. * GC types in const-eval were almost already using `Val` meaning that there's actually fewer conversions now. * Instantiation code was refactored to use `wasmtime::*`-API types instead of low-level VM types. This deduplicates a good deal and lifts complexity out of the raw VM bits.
Another issue that this commit fixes is to change how table initialization is modeled internally in `vm::Instance::table_init_segment`. Previously this was done by removing the tables from an instance to get a split borrow into the store and the table. This is not valid though because if, during initialization, a GC is performed then the table is not present to find roots through the table. This function is refactored to scope borrows to within a loop instead of over a loop via various refactorings and such and usage of higher level APIs. This is again, like above, expected to pessimize performance but this is also not known to be a hot path for modules at this time.
* Remove the `TableElement` type
This commit is a refactoring of how tables work within Wasmtime to avoid funneling table elements through a `TableElement` enum internally. Instead methods are "exploded" to `grow_{gc_ref,func,cont}` which means, for example, funcrefs don't need a GcStore. The main motivation for this change was to avoid the idiom where `TableElement` represents a cloned, but unrooted, GC reference.
Prior to this commit there were a number of subtle bugs in the table code for Wasmtime where write barriers were forgotten on `table.init`, `table.set` (via the embedder API), and `table.grow`. While `table.fill` correctly handled the GC references it was awkward to get everything else working consistently so I opted to remove `TableElement` entirely to make it more clear that `&VMGcRef` is ubiquitously used meaning that the write barriers, for example, are the same as other parts of the Wasmtime codebase.
This has a few extra tests for "make sure this doesn't leak" to ensure that GC works correctly with new barriers in place.
* Fix some lints and warnings
* Fix wmemcheck build
* Review comments
* Optimize const eval and global initialization
* Fix compile
* Fix lints
show more ...
|