Changes in enabled.rs

eaa4632e - Implement exception objects. (#11230)

Tue, 15 Jul 2025 17:15:35 +0000

Implement exception objects. (#11230)* WIP: Working exception objects* Clean build with gc disabled (`cargo check -p wasmtime --no-default-features --features runtime`).* Review feedback.* Stub out C-API support.* Fix Clippy complaints.* Fix dead-code warning in c-api build.* Actually fix 27->26 reserved bit rename and test.* Fix exnref doc-test.* fix fuzzing build* fix feature-flagging on Instance::id* Bless disas test diff due to reserved-bits change.* Review feedback. List of files: /wasmtime-44.0.1/crates/wasmtime/src/runtime/gc/enabled.rs

c16414fb - Introduce the `wasmtime::EqRef` type (#9285)

Thu, 19 Sep 2024 19:48:54 +0000

Introduce the `wasmtime::EqRef` type (#9285)* Introduce the `wasmtime::EqRef` typeThis commit introduces the `wasmtime::EqRef` type, which corresponds to Wasm's`(ref eq)` type, and statically represents Wasm references that can be testedfor equality.* fix no-gc builds List of files: /wasmtime-44.0.1/crates/wasmtime/src/runtime/gc/enabled.rs

c4be2d84 - Introduce `wasmtime::ArrayRef` and allocating Wasm GC arrays (#9145)

Tue, 20 Aug 2024 19:34:28 +0000

Introduce `wasmtime::ArrayRef` and allocating Wasm GC arrays (#9145)* Introduce `wasmtime::ArrayRef` and allocating Wasm GC arraysThis commit introduces the `wasmtime::ArrayRef` type and support for allocatingWasm GC arrays from the host. This commit does *not* add support for the`array.new` family of Wasm instructions; guests still cannot allocate Wasm GCobjects yet, but initial support should be pretty straightforward after thiscommit lands.The `ArrayRef` type has everything you expect from other value types in the`wasmtime` crate:* A method to get its type or check whether it matches a given type* An implementation of `WasmTy` so that it can be used with `Func::wrap`-style APIs* The ability to upcast it into an `AnyRef` and to do checked downcasts in the opposite directionThere are, additionally, methods for getting, setting, and enumerating a`ArrayRef`'s elements.Similar to how allocating a Wasm GC struct requires a `StructRefPre`, allocatinga Wasm GC array requires an `ArrayRefPre`, and this is motivated by the samereasons.* fix some doc tests and add docs for Func::wrap-style APIs* Add a comment about why we can't user `iter::repeat(elem).take(len)`* Fix some warnings in no-gc builds List of files: /wasmtime-44.0.1/crates/wasmtime/src/runtime/gc/enabled.rs

f2e689cd - Introduce `wasmtime::StructRef` and allocating Wasm GC structs (#8933)

Thu, 11 Jul 2024 22:14:56 +0000

Introduce `wasmtime::StructRef` and allocating Wasm GC structs (#8933)* Introduce `wasmtime::StructRef` and allocating Wasm GC structsThis commit introduces the `wasmtime::StructRef` type and support for allocatingWasm GC structs from the host. This commit does *not* add support for the`struct.new` family of Wasm instructions; guests still cannot allocate Wasm GCobjects yet, but initial support should be pretty straightforward after thiscommit lands.The `StructRef` type has everything you expect from other value types in the`wasmtime` crate:* A method to get its type or check whether it matches a given type* An implementation of `WasmTy` so that it can be used with `Func::wrap`-style APIs* The ability to upcast it into an `AnyRef` and to do checked downcasts in the opposite directionThere are, additionally, methods for getting, setting, and enumerating a`StructRef`'s fields.To allocate a `StructRef`, we need proof that the struct type we are allocatingis being kept alive for the duration that the allocation may live. This isrequired for many reasons, but a basic example is getting a struct instance'stype from the embedder API: this does a type-index-to-`StructType` lookup andconversion and if the type wasn't kept alive, then the type-index lookup willresult in what is logically a use-after-free bug. This won't be a problem forWasm guests (when we get around to implementing allocation for them) since theirmodule defines the type, the store holds onto its instances' modules, and theallocation cannot outlive the store. For the host, we need another method ofkeeping the object's type alive, since it might be that the host defined thetype and there is no module that also defined it, let alone such a module thatis being kept alive in the store.The solution to the struct-type-lifetime problem that this commit implements forhosts is for the store to hold a hash set of `RegisteredType`s specifically forobjects which were allocated via the embedder API. But we also don't want to doa hash lookup on every allocation, so we also implement a `StructRefPre` type. A`StructRefPre` is proof that the embedder has inserted a `StructType`'s inner`RegisteredType` into a store. Structurally, it is a pair of the struct type anda store id. All `StructRef` allocation methods require a `StructRefPre`argument, which does a fast store id check, rather than a whole hash tableinsertion.I opted to require `StructRefPre` in all allocation cases -- even though thishas the downside of always forcing callers to create one before they allocate,even if they are only allocating a single object -- because of tworeasons. First, this avoids needing to define duplicate methods, with andwithout a `StructRefPre` argument. Second, this avoids a performance footgun inthe API where users don't realize that they *can* avoid extra work by creating asingle `StructRefPre` and then using it multiple times. Anecdotally, I've heardmultiple people complain about instantiation being slower than advertised but itturns out they weren't using `InstancePre`, and I'd like to avoid that situationfor allocation if we can.* Move `allow(missing_docs)` up to `gc::disabled` module instead of each `impl`* Rename `cast` to `unchecked_cast`* fix `GcHeapOutOfMemory` error example in doc example* document additional error case for `StructRef::new`* Use `unpack` method instead of open-coding it* deallocate on failed initialization* Refactor field access methods to share more codeAnd define `fields()` in terms of `field()` rather than the other way around.* Add upcast methods from structref to anyref* Remove duplicate type checking and add clarifying comments about initializing vs writing fields* make the `PodValType` trait safe* fix benchmarks build* prtest:full* add miri ignores to new tests that call into wasm List of files: /wasmtime-44.0.1/crates/wasmtime/src/runtime/gc/enabled.rs

0fa13013 - Add `GcRuntime` and `GcCompiler` traits; `i31ref` support (#8196)

Thu, 04 Apr 2024 00:24:50 +0000

Add `GcRuntime` and `GcCompiler` traits; `i31ref` support (#8196)\### The `GcRuntime` and `GcCompiler` TraitsThis commit factors out the details of the garbage collector away from the restof the runtime and the compiler. It does this by introducing two new traits,very similar to a subset of [those proposed in the Wasm GC RFC], although notall equivalent functionality has been added yet because Wasmtime doesn'tsupport, for example, GC structs yet:[those proposed in the Wasm GC RFC]: https://github.com/bytecodealliance/rfcs/blob/main/accepted/wasm-gc.md#defining-the-pluggable-gc-interface1. The `GcRuntime` trait: This trait defines how to create new GC heaps, run collections within them, and execute the various GC barriers the collector requires. Rather than monomorphize all of Wasmtime on this trait, we use it as a dynamic trait object. This does imply some virtual call overhead and missing some inlining (and resulting post-inlining) optimization opportunities. However, it is *much* less disruptive to the existing embedder API, results in a cleaner embedder API anyways, and we don't believe that VM runtime/embedder code is on the hot path for working with the GC at this time anyways (that would be the actual Wasm code, which has inlined GC barriers and direct calls and all of that). In the future, once we have optimized enough of the GC that such code is ever hot, we have options we can investigate at that time to avoid these dynamic virtual calls, like only enabling one single collector at build time and then creating a static type alias like `type TheOneGcImpl = ...;` based on the compile time configuration, and using this type alias in the runtime rather than a dynamic trait object. The `GcRuntime` trait additionally defines a method to reset a GC heap, for use by the pooling allocator. This allows reuse of GC heaps across different stores. This integration is very rudimentary at the moment, and is missing all kinds of configuration knobs that we should have before deploying Wasm GC in production. This commit is large enough as it is already! Ideally, in the future, I'd like to make it so that GC heaps receive their memory region, rather than allocate/reserve it themselves, and let each slot in the pooling allocator's memory pool be *either* a linear memory or a GC heap. This would unask various capacity planning questions such as "what percent of memory capacity should we dedicate to linear memories vs GC heaps?". It also seems like basically all the same configuration knobs we have for linear memories apply equally to GC heaps (see also the "Indexed Heaps" section below).2. The `GcCompiler` trait: This trait defines how to emit CLIF that implements GC barriers for various operations on GC-managed references. The Rust code calls into this trait dynamically via a trait object, but since it is customizing the CLIF that is generated for Wasm code, the Wasm code itself is not making dynamic, indirect calls for GC barriers. The `GcCompiler` implementation can inline the parts of GC barrier that it believes should be inline, and leave out-of-line calls to rare slow paths.All that said, there is still only a single implementation of each of thesetraits: the existing deferred reference-counting (DRC) collector. So there is abunch of code motion in this commit as the DRC collector was further isolatedfrom the rest of the runtime and moved to its own submodule. That said, this wasnot *purely* code motion (see "Indexed Heaps" below) so it is worth not simplyskipping over the DRC collector's code in review.\### Indexed HeapsThis commit does bake in a couple assumptions that must be shared across allcollector implementations, such as a shared `VMGcHeader` that all objectsallocated within a GC heap must begin with, but the most notable andfar-reaching of these assumptions is that all collectors will use "indexedheaps".What we are calling indexed heaps are basically the three following invariants:1. All GC heaps will be a single contiguous region of memory, and all GC objects will be allocated within this region of memory. The collector may ask the system allocator for additional memory, e.g. to maintain its free lists, but GC objects themselves will never be allocated via `malloc`.2. A pointer to a GC-managed object (i.e. a `VMGcRef`) is a 32-bit offset into the GC heap's contiguous region of memory. We never hold raw pointers to GC objects (although, of course, we have to compute them and use them temporarily when actually accessing objects). This means that deref'ing GC pointers is equivalent to deref'ing linear memory pointers: we need to add a base and we also check that the GC pointer/index is within the bounds of the GC heap. Furthermore, compressing 64-bit pointers into 32 bits is a fairly common technique among high-performance GC implementations[^compressed-oops][^v8-ptr-compression] so we are in good company.3. Anything stored inside the GC heap is untrusted. Even each GC reference that is an element of an `(array (ref any))` is untrusted, and bounds checked on access. This means that, for example, we do not store the raw pointer to an `externref`'s host object inside the GC heap. Instead an `externref` now stores an ID that can be used to index into a side table in the store that holds the actual `Box` host object, and accessing that side table is always checked.[^compressed-oops]: See ["Compressed OOPs" in OpenJDK.](https://wiki.openjdk.org/display/HotSpot/CompressedOops)[^v8-ptr-compression]: See [V8's pointer compression](https://v8.dev/blog/pointer-compression).The good news with regards to all the bounds checking that this scheme impliesis that we can use all the same virtual memory tricks that linear memories useto omit explicit bounds checks. Additionally, (2) means that the sizes of GCobjects is that much smaller (and therefore that much more cache friendly)because they are only holding onto 32-bit, rather than 64-bit, references toother GC objects. (We can, in the future, support GC heaps up to 16GiB in sizewithout losing 32-bit GC pointers by taking advantage of `VMGcHeader` alignmentand storing aligned indices rather than byte indices, while still leaving thebottom bit available for tagging as an `i31ref` discriminant. Should we everneed to support even larger GC heap capacities, we could go to full 64-bitreferences, but we would need explicit bounds checks.)The biggest benefit of indexed heaps is that, because we are (explicitly orimplicitly) bounds checking GC heap accesses, and because we are not otherwisetrusting any data from inside the GC heap, we greatly reduce how badly thingscan go wrong in the face of collector bugs and GC heap corruption. We areessentially sandboxing the GC heap region, the same way that linear memory is asandbox. GC bugs could lead to the guest program accessing the wrong GC object,or getting garbage data from within the GC heap. But only garbage data fromwithin the GC heap, never outside it. The worse that could happen would be if wedecided not to zero out GC heaps between reuse across stores (which is a validtrade off to make, since zeroing a GC heap is a defense-in-depth techniquesimilar to zeroing a Wasm stack and not semantically visible in the absence ofGC bugs) and then a GC bug would allow the current Wasm guest to read old GCdata from the old Wasm guest that previously used this GC heap. But again, itcould never access host data.Taken altogether, this allows for collector implementations that are nearly freefrom `unsafe` code, and unsafety can otherwise be targeted and limited in scope,such as interactions with JIT code. Most importantly, we do not have to maintaincritical invariants across the whole system -- invariants which can't be nicelyencapsulated or abstracted -- to preserve memory safety. Such holisticinvariants that refuse encapsulation are otherwise generally a huge safetyproblem with GC implementations.\### `VMGcRef` is *NOT* `Clone` or `Copy` Anymore`VMGcRef` used to be `Clone` and `Copy`. It is not anymore. The motivation herewas to be sure that I was actually calling GC barriers at all the correctplaces. I couldn't be sure before. Now, you can still explicitly copy a raw GCreference without running GC barriers if you need to and understand why that'sokay (aka you are implementing the collector), but that is something you have toopt into explicitly by calling `unchecked_copy`. The default now is that youcan't just copy the reference, and instead call an explicit `clone` method (not*the* `Clone` trait, because we need to pass in the GC heap context to run theGC barriers) and it is hard to forget to do that accidentally. This resulted ina pretty big amount of churn, but I am wayyyyyy more confident that the correctGC barriers are called at the correct times now than I was before.\### `i31ref`I started this commit by trying to add `i31ref` support. And it grew into thewhole traits interface because I found that I needed to abstract GC barriersinto helpers anyways to avoid running them for `i31ref`s, so I figured that Imight as well add the whole traits interface. In comparison, `i31ref` support ismuch easier and smaller than that other part! But it was also difficult to pullapart from this commit, sorry about that!---------------------Overall, I know this is a very large commit. I am super happy to have somesynchronous meetings to walk through this all, give an overview of thearchitecture, answer questions directly, etc... to make review easier!prtest:full List of files: /wasmtime-44.0.1/crates/wasmtime/src/runtime/gc/enabled.rs

bd2ea901 - Define garbage collection rooting APIs (#8011)

Wed, 06 Mar 2024 00:40:02 +0000

Define garbage collection rooting APIs (#8011)* Define garbage collection rooting APIsRooting prevents GC objects from being collected while they are actively beingused.We have a few sometimes-conflicting goals with our GC rooting APIs:1. Safety: It should never be possible to get a use-after-free bug because the user misused the rooting APIs, the collector "mistakenly" determined an object was unreachable and collected it, and then the user tried to access the object. This is our highest priority.2. Moving GC: Our rooting APIs should moving collectors (such as generational and compacting collectors) where an object might get relocated after a collection and we need to update the GC root's pointer to the moved object. This means we either need cooperation and internal mutability from individual GC roots as well as the ability to enumerate all GC roots on the native Rust stack, or we need a level of indirection.3. Performance: Our rooting APIs should generally be as low-overhead as possible. They definitely shouldn't require synchronization and locking to create, access, and drop GC roots.4. Ergonomics: Our rooting APIs should be, if not a pleasure, then at least not a burden for users. Additionally, the API's types should be `Sync` and `Send` so that they work well with async Rust.For example, goals (3) and (4) are in conflict when we think about how tosupport (2). Ideally, for ergonomics, a root would automatically unroot itselfwhen dropped. But in the general case that requires holding a reference to thestore's root set, and that root set needs to be held simultaneously by all GCroots, and they each need to mutate the set to unroot themselves. That implies`Rc>` or `Arc>`! The former makes the store and GC roottypes not `Send` and not `Sync`. The latter imposes synchronization and lockingoverhead. So we instead make GC roots indirect and require passing in a storecontext explicitly to unroot in the general case. This trades worse ergonomicsfor better performance and support for moving GC and async Rust.Okay, with that out of the way, this module provides two flavors of rootingAPI. One for the common, scoped lifetime case, and another for the rare casewhere we really need a GC root with an arbitrary, non-LIFO/non-scoped lifetime:1. `RootScope` and `Rooted`: These are used for temporarily rooting GC objects for the duration of a scope. Upon exiting the scope, they are automatically unrooted. The internal implementation takes advantage of the LIFO property inherent in scopes, making creating and dropping `Rooted`s and `RootScope`s super fast and roughly equivalent to bump allocation. This type is vaguely similar to V8's [`HandleScope`]. [`HandleScope`]: https://v8.github.io/api/head/classv8_1_1HandleScope.html Note that `Rooted` can't be statically tied to its context scope via a lifetime parameter, unfortunately, as that would allow the creation and use of only one `Rooted` at a time, since the `Rooted` would take a borrow of the whole context. This supports the common use case for rooting and provides good ergonomics.2. `ManuallyRooted`: This is the fully general rooting API used for holding onto non-LIFO GC roots with arbitrary lifetimes. However, users must manually unroot them. Failure to manually unroot a `ManuallyRooted` before it is dropped will result in the GC object (and everything it transitively references) leaking for the duration of the `Store`'s lifetime. This type is roughly similar to SpiderMonkey's [`PersistentRooted`], although they avoid the manual-unrooting with internal mutation and shared references. (Our constraints mean we can't do those things, as mentioned explained above.) [`PersistentRooted`]: http://devdoc.net/web/developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/JSAPI_reference/JS::PersistentRooted.htmlAt the end of the day, both `Rooted` and `ManuallyRooted` are just taggedindices into the store's `RootSet`. This indirection allows working with Rust'sborrowing discipline (we use `&mut Store` to represent mutable access to the GCheap) while still allowing rooted references to be moved around without tying upthe whole store in borrows. Additionally, and crucially, this indirection allowsus to update the *actual* GC pointers in the `RootSet` and support moving GCs(again, as mentioned above).* Reorganize GC-related submodules in `wasmtime-runtime`* Reorganize GC-related submodules in `wasmtime`* Use `Into` for `Externref::data[_mut]` methods* Run rooting tests under MIRI* Make `into_abi` take an `AutoAssertNoGc`* Don't use atomics to update externref ref counts anymore* Try to make lifetimes/safety more-obviously correctRemove some transmute methods, assert that `VMExternRef`s are the only valid`VMGcRef`, etc.* Update extenref constructor examples* Make `GcRefImpl::transmute_ref` a non-default trait method* Make inline fast paths for GC LIFO scopes* Make `RootSet::unroot_gc_ref` an `unsafe` function* Move Hash and Eq for Rooted, move to impl methods* Remove type parameter from `AutoAssertNoGc`Just wrap a `&mut StoreOpaque` directly.* Make a bunch of internal `ExternRef` methods that deal with raw `VMGcRef`s take `AutoAssertNoGc` instead of `StoreOpaque`* Fix compile after rebase* rustfmt* revert unrelated egraph changes* Fix non-gc build* Mark `AutoAssertNoGc` methods inline* review feedback* Temporarily remove externref support from the C APIUntil we can add proper GC rooting.* Remove doxygen reference to temp deleted function* Remove need to `allow(private_interfaces)`* Fix call benchmark compilation List of files: /wasmtime-44.0.1/crates/wasmtime/src/runtime/gc/enabled.rs