Add support for map type (#12216)* Add support for map typeSigned-off-by: Yordis Prieto <[email protected]>* Add Map and MapEntry classes to support key/value pairs in component modelTh
Add support for map type (#12216)* Add support for map typeSigned-off-by: Yordis Prieto <[email protected]>* Add Map and MapEntry classes to support key/value pairs in component modelThis commit introduces the Map and MapEntry classes, enabling the representation of map values in the component model. The Map class allows for the creation and iteration of key/value pairs, enhancing the functionality of the wasmtime component API. Additionally, the .gitignore file is updated to exclude build artifacts from the crates/c-api directory.* Add wasm_component_model_map configuration support* Format code* Format C code* Enhance component model to support HashMap<K, V> typeThis commit introduces support for HashMap<K, V> in the component model, allowing maps to be represented as list<tuple<K, V>> in the canonical ABI. It includes implementations for the ComponentType, Lower, and Lift traits for HashMap, enabling type checking, lowering to flat representations, and lifting from memory. Additionally, the maximum depth for type generation in the fuzzing utility is updated to accommodate the new map type.* Refactor component configuration to introduce map supportThis commit removes the previous wasm features configuration and adds new functions for creating a map-configured engine. The `map_config` and `map_engine` functions are introduced to facilitate the use of the component model with maps in tests, ensuring that the engine is properly configured for map types in the component model.* Add new WAST test for map types and remove map type definitions from existing testsThis commit introduces a new WAST test file specifically for testing various map types in the component model. Additionally, it removes the redundant map type definitions from the existing types.wast file to streamline the test suite.* Update component fuzzing and dynamic tests to replace call_and_post_return with call* Format code* Refactor HashMap usage in typed.rs to use wasmtime_environ collections* Fix HashMap initialization and insertion to handle potential errors in typed.rs* Refactor HashMap handling in typed.rs to use lower_map_iter for improved iteration and memory management. Introduce new implementations for ComponentType, Lower, and Lift traits for std::collections::HashMap, enhancing support for map types in the component model.* Fix map adapter trampoline compilation and alignment bugsThe translate_map function had two categories of bugs preventing mapadapter trampolines from working:1. Wasm stack discipline: local_set_new_tmp emits LocalSet which pops from the stack, but was called when the stack was empty (to "pre-allocate" locals). Fixed by computing values first, then calling local_set_new_tmp to consume them—matching translate_list's pattern. Also removed an erroneous LocalTee that left an orphan value on the stack. Affected: src_byte_len, dst_byte_len, cur_src_ptr, cur_dst_ptr.2. Pointer advancement: after value translation, the pointer still points at the value start. The code only advanced by trailing padding instead of value_size + trailing_padding, causing every loop iteration to re-read the same memory.Also fixes entry layout to use proper record alignment rules (entryalign = max(key_align, value_align), value at aligned offset).* Refactor map entry layout calculations to use canonical ABI* Remove unnecessary clone of map pairs during loweringVal::Map already holds Vec<(Val, Val)> which derefs to &[(Val, Val)],matching lower_map's signature directly. The intermediate Vec allocationand deep clone of every key/value pair was redundant.* Deduplicate map lift logic between HashMap implementations* Deduplicate list and map sequence translation scaffolding* Fix cargo fmt formatting issues* Deduplicate map typecheck logic* Deduplicate map lowering with linear_lower_map_to_flat and linear_lower_map_to_memory helpers* Clean up lift_try_map: use drop, move TryHashMap import to module scope* Fix CI: arbtest overflow and no-std HashMap lift_map- component_fuzz: use saturating_sub in generate_hashable_key to prevent underflow when fuel is 0 and Enum variant is chosen- typed: remove incorrect ? operators in lift_map for hashbrown::HashMap (with_capacity and insert don't return Result)* Store map tuple layout in TypeMapCompute map entry ABI and value offsets once during type building, and reuse that metadata in runtime map lift/lower paths instead of recalculating tuple layout at each call site.* Refactor map ABI argument passingBundle map lift/lower layout and type metadata into a small MapAbi32 helper so map helper calls stay concise without changing behavior.* Fix CI: enable component_model_map in fuzzing and handle map in arbitrary_valThe fuzzer's component_api oracle was generating map types but the enginedidn't have the map feature enabled, and arbitrary_val had no arm forType::Map. Enable component_model_map in the store helper (matching howcomponent_model_async is forced on) and implement arbitrary value generationfor map types.---------Signed-off-by: Yordis Prieto <[email protected]>
show more ...
perf(wave): avoid Vec allocation in make_tuple (#12464)
cranelift: stack-switching support (#11003)* cranelift: stack-switching supportThis initial commit represents the "pr2" base commit withminimal merge conflicts resolved. Due to OOB conflicts, t
cranelift: stack-switching support (#11003)* cranelift: stack-switching supportThis initial commit represents the "pr2" base commit withminimal merge conflicts resolved. Due to OOB conflicts, thiscommit is not functional as-is, but using it as a base inorder to allow for easier reviewing of the delta fromthis commit to what will be used for the PR against upstream.Co-authored-by: Daniel Hillerström <[email protected]>Co-authored-by: Paul Osborne <[email protected]>* cranelift: stack-switching updates pass 1This first set of changes updates the base pr in order tocompiled and pass basic checks (compile, clippy, fmt) withthe biggest part of the change being to eliminate injectionof tracing/assertions in JIT'ed code.* cranelift: stack-switching: restore original visibility for a few func_environ members* cranelift: stack-switching conditional compilationAt this point, the only bit we really branch on is what wedo in order to avoid problems tying into wasmtime_environ.This is basd on the approach and macro used by the gc code forconverting presence/absence of the cranelift feature flag tocranelift compile time. This is a bit of a half-measure for nowas we still compile most stack-switching code in cranelift, butthis does enough to avoid causing problems with missing definitionsin wasmtime_environ.* cranelift: avoid "as" casts in stack-switchingReplace either with infallible From or fallible, panicingTryFrom alternatives where required.* cranelift: cleanup stack-switching control_effect signaturesAfter removing emission of runtime trace logging and assertions,there were several unused parameters. Remove those from theControlEffect signatures completely.* cranelift: rename stack-switching VMArray to VMHostArrayThis matches a change to the mirrored runtime type inthe upstream changes.* stack-switching: fix typoCo-authored-by: Daniel Hillerström <[email protected]>* stack-switching: used Index impl for get_stack_slot_data* stack-switching: use smallvec over vec in several cases* stack-switching: avoid resumetable naming confusion* stack-switching: cleanup unused params from unchecked_get_continuationThe extra parameters here used to be used for emitting runtimeassertions, but with those gone we just had unused paramsand lifetimes, clean those out.* stack_switching: simplify store_data_entries assertion* stack-switching: simplify translate_table_{grow,fill} control flow* stack-switching: remove translate_resume_throw stubThere's already a stub elsewhere and this is not called, whenexceptions are added and it is time to revisit, this methodcan be restored.* stack-switching: compute control_context_size based on target triple* stack-switching: VMHostArrayRef updatesRename VMHostArray -> VMHostArrayRefChange impl to compute address with offset upfront rather thanon each load.* stack-switching: move cranelift code to live under func_environThis matches the directory structure for gc and aids in visibilityfor a few members required by stack-switching code in cranelift.* stack-switching: formatting fix* stack-switching: reduce visibility on a few additional items* stack-switching: simplify contobj fatptr con/de-struction* stack-switching: add disas tests to cover new instructions* stack-switching: fix layout of VMContObjIn the course of the various runtime updates, the layout of theruntime VMContObj got switched around. This resulted in failureswhen doing certain table operations on continuations.This change fixes that layout problem and adds some tests withoffsets to avoid the problem. Due to the way that we interactwith the VMContObj in cranelift, we don't use these offsets outsideof the tests.* Fix formatting of merge conflict resolution* cranelift: remove ir::function::get_stack_slot_dataThis method isn't required as sized_stack_slots is already pub.* stack-switching: reduce visibility of a couple func_environ methods* stack-switching: define VMContObj as two wordsThis change migrates VMContObj and its usages in cranelift and runtimeto work with the VMContObj fat pointer as two words in order to bettertarget different architectures (still gated to x86_64 for now).To support this, a size type was plumbed into the builtins functionsignature types (as is done for component types) that maps tousize.* fixup! stack-switching: define VMContObj as two words* stack-switching: add stub Val::ContRefThis type is not fully complete until continuation/gc integrationis revisited (#10248) but without these changes, test cases arenow failing on panics as we need some representation ofcontinuation references in the runtime Val enumeration.Runtime errors with TODO notes are added for the stubbedcode paths to revisit later.* fixup! stack-switching: add stub Val::ContRef* fixup! stack-switching: add stub Val::ContRef* fixup! stack-switching: define VMContObj as two wordsprtest:full* stack-switching: don't conflate host and target pointer sizesDisas tests were failing on i686 targeting x86_64 as the size of thehost pointer was leaking into what we were using to do codegenin a few paths. This patch is a bit of a hack as it seems likeusing a generic <T> for T: *mut u8 (as an example) is a bitquestionable. To keep things small, I do a hacky typecheck to mappointers to the target pointer size here.* stack-switching: VMHostArray entry sizes based off env PtrSizeRevisiting the previous commit with an approach that should beless brittle.---------Co-authored-by: Frank Emrich <[email protected]>Co-authored-by: Daniel Hillerström <[email protected]>
Implement exception objects. (#11230)* WIP: Working exception objects* Clean build with gc disabled (`cargo check -p wasmtime --no-default-features --features runtime`).* Review feedback.* St
Implement exception objects. (#11230)* WIP: Working exception objects* Clean build with gc disabled (`cargo check -p wasmtime --no-default-features --features runtime`).* Review feedback.* Stub out C-API support.* Fix Clippy complaints.* Fix dead-code warning in c-api build.* Actually fix 27->26 reserved bit rename and test.* Fix exnref doc-test.* fix fuzzing build* fix feature-flagging on Instance::id* Bless disas test diff due to reserved-bits change.* Review feedback.
Enable `allow_attributes_without_reason` (#11195)* Enable `allow_attributes_without_reason`This commit enables the `clippy::allow_attributes_without_reason` forthe `wasmtime` crate which previou
Enable `allow_attributes_without_reason` (#11195)* Enable `allow_attributes_without_reason`This commit enables the `clippy::allow_attributes_without_reason` forthe `wasmtime` crate which previously forcibly allowed it. The reasonthis was allowed was that when the workspace was first migrated theWasmtime crate had too many instances that I was willing to fix. I'venow come back around and tried to fix everything.In short: ideally delete `#[allow]`, otherwise use `#[expect]`,otherwise use `#[allow]`.prtest:full* Adjust some directives* Fix some warnings* Fix stack switching size tests on unix* Don't have a conditional `Drop` impl* Force `testing_freelist` method to be usedToo lazy to write `#[cfg]`, but not too lazy to write a test.
update `component-model-async` plumbing (#11123)* [DO NOT MERGE] update `component-model-async` plumbingThis pulls in the latest Component Model async ABI code from the`wasip3-prototyping` repo,
update `component-model-async` plumbing (#11123)* [DO NOT MERGE] update `component-model-async` plumbingThis pulls in the latest Component Model async ABI code from the`wasip3-prototyping` repo, including various API refactors and spec updates.This includes all the changes to the `wasmtime` crate from `wasip3-prototyping`_except_ that the `concurrent` submodule and child submodules contain onlynon-functional stubs. For that reason, and the fact thate.g. `Func::call_async` is now implemented in terms of `Func::call_concurrent`,most of the component model tests are failing. This commit is not meant to bemerged as-is; a follow-up commit (to be PR'd separately) will contain the real`concurrent` implementation, at which point the tests will pass again. I'msplitting these into separate PRs to make review easier.Signed-off-by: Joel Dice <[email protected]>* Undo wit-bindgen changesNo longer necessary after other refactors* Move back to crates.io-based wit-bindgen* Undo upgrade of http-body-util(deferred for future PR)* Add back in arbitrary use of asyncLooks like it may have been lost by accident* Make imports more conventional for Wasmtime* Some minor changes* Privatize a component field* Cut down a bit on #[cfg]* Undo a no-longer-necessary `pub`* add doc comments for `{Future,Stream,ErrorContext}Any`Signed-off-by: Joel Dice <[email protected]>* rename `concurrent` stub module to `concurrent_disabled`...and avoid panicking in the stubs.Signed-off-by: Joel Dice <[email protected]>* fix test regressionSigned-off-by: Joel Dice <[email protected]>* revert `call_async` and `post_return_impl` changesThese will need to wait until the `component-model-async` feature is fullyimplemented.Signed-off-by: Joel Dice <[email protected]>* remove unused structSigned-off-by: Joel Dice <[email protected]>* add `Options::callback` fieldThis isn't used yet, but will be used when the real `component-model-async`implementation is merged.Signed-off-by: Joel Dice <[email protected]>* Remove no-longer-needed feature* Trim reexports from WasmtimeSome of these are no longer needed or can be avoided with small changes.Some deps are likely needed in the next commit but they'll be best addedthere.* Update test expectations* More trimming of Cargo.toml* Defer `*Buffer` traits to next PRNot needed for this PR I believe.* Use conventional Wasmtime imports + remove dummy_waker* Reduce duplication in `*_disabled`* Remove some unncessary bounds* Remove some `for<'a>` bounds where unnecessary* Remove another bound* Defer more functions to the next PR`drop_fibers` is different in the next PR, so defer it to then.* Remove some reexports no longer necessaryBindings generation changed awhile back so these aren't needed, deferthe implementations to the next PR.* Remove unnecessary dropThis was already moved to `run_manual_drop_routines`* Defer a `pub(crate)` to a future PR* Expand comments in traphandlers* Defer some types to the next PR* Update linker documentation* Add `Send`/`Sync` bounds to `ComponentType`This commit is extracted from from review of #11123 and #11127. Whilenot literally present in those PRs it's my own personal conclusion thatit's best to just go ahead and add these bounds at the "base" of thecomponent trait hierarchy. The current implementation in #11123 addsbounds in many locations and this would remove the need to add boundseverywhere and instead have everything inherited through the `Lift` and`Lower` traits.This raises the question of: why? The main conclusion that I've reachedleading to this change is that Wasmtime currently will store `R`, areturn value, on the stack during the lowering process back into linearmemory. This might involve allocation, however, meaning that wasm can beinvoked and a context switch could happen. For Wasmtime's `unsafe impl`of `Send` and `Sync` on fibers to be sound it requires that thisstack-local variable is also `Send` and `Sync` as it's an entirelyuser-provided type. Thus I've concluded that for results it's alwaysrequired for these to be both `Send` and `Sync` (or at the very least,`Send`).Given that I've gone ahead and updated to require both `Send` and `Sync`for both params and results. This is not expected to actually have anyimpact in practice since all primitives are already `Send`/`Sync` (minus`Rc` impls all removed here) and all `bindgen!`-generated types arecompositions of `Send`/`Sync` primitives meaning that they're also`Send` and `Sync`.* Remove some now-unnecessary bounds* Fix build after #11160* Remove some now-unnecessary duplicate bounds* Uncomment test that now works* Undo accidental doc wrap* Clarify comment on `Value` typesDon't leave `TODO` in public-facing documentation ideally* Actually resolve the conflict (forgot to commit)* Avoid returning boxed futures in APIs* Defer making constructors more public to a future PR* Make a method name more conventional* Refactor `linear_lift_into_from_memory`* Drop the `max_count` parameter in favor of slicing the `WasmList` itself. Avoids situations such as what happens if `max_count` is larger than the length of the list.* Don't have the default implementation collect to a vector and then push all that onto a different vector. Instead push each item individually through `extend`.* De-indent a block of code added* Remove unsafety from `prepare_call`Mostly move the parameters themselves to the closure to avoid rawpointers/drop/etc.This will have the consequence of in the future `call_async` is going tonow require `Params: 'static` but that seems more-or-less inevitable atthis point.* Go back to returning box, alas.* Apply same treatment to lift functionMake it a closure and reduce some levels of indirection of the variousfunctions in play.* Refactor `lower_params` to require less context.Relax the bounds on the closure specified since it's immediately calledand then additionally take out parameters/captures that the closure cancarry itself.* Don't pass extraneous `Instance` parameterThis can now be inferred from `Func`.* Clean up some SAFETY comments* Generalize the signature of `lift_results`* Move `lift_results` function to `Func`Also rename the lift/lower helpers to `with_{lift,lower}_context`* Remove parameter from `with_lift_context`Like `with_lower_context` this is fine to capture in the closure passedin.* Simplify the dynamic lifting logicDon't call `with_lift_context` in two locations, only call it once witha dynamic parameter.* Refactor away the `Func::lift_results_sync` helper* Use `with_lift_context` in `call_raw`* Simplify a call to `Func::call_unchecked_raw`* Ungate `with_lift_context` to fix non-cm-async build* Fix compile (bad cherry-pick conflict resolution)* Use `with_lower_context` in `call_raw`Trying to unify the async/concurrent paths as much as possible.* Move params out of `call_raw`Let closures capture the params, no need to thread it through as anunnecessary argument.* Clean up unsafety in `Func::call_raw`* Accurately mark `call_raw` itself as `unsafe`, then document why callers should be safe.* Don't have one large `unsafe` block in `call_raw`, instead split it up with separate safety comments.* Move a one-off type definition closer to its use* Avoid intermediate allocations in dynamic calls* Simplify a future-return site* Deduplicate checking parameter count* Simplify a future invocation with `?`* Simplify a variable declaration* Simplify some function signatures* Remove outdated safety comment* Refactor to not require `Params: 'static` on `call_async`* Remove no-longer-necessary SAFETY comment* Fix typos* Add a fast-path with no `Box` for sync host functionsSpeeds up host calls by ~20% and puts them back on parity with thebeforehand numbers Wasmtime has.* Synchronize signatures of async/concurrent dynamic callsUse slices for both instead of vecs for one and slices for the other.Required some slight rejiggering. Apparently one can solve a closureproblem with another closure, then one surely has no more closureproblems.* Fix non-cm-async build---------Signed-off-by: Joel Dice <[email protected]>Co-authored-by: Alex Crichton <[email protected]>
Update nightly used in CI (#10957)A new lint was added to rustc so this updates the nightly used in CI andthen additionally fixes the lints that are firing.
Update Wasmtime to the 2024 Rust Edition (#10806)* Update Wasmtime to the 2024 Rust EditionNow that our MSRV supports the 2024 edition it's possible to make thisswitch. This commit moves Wasmtim
Update Wasmtime to the 2024 Rust Edition (#10806)* Update Wasmtime to the 2024 Rust EditionNow that our MSRV supports the 2024 edition it's possible to make thisswitch. This commit moves Wasmtime to the 2024 Edition to keepup-to-date with Rust idioms and access many of the edition featuresexclusive to the 2024 edition.prtest:full* Reformat with the 2024 edition
wasmtime-cli: support `run --invoke` for components using wave (#10054)* wip* invoke works!!!* code motion* more context in errors* fix test of invoke* Invoke wave enhancements (#10511)
wasmtime-cli: support `run --invoke` for components using wave (#10054)* wip* invoke works!!!* code motion* more context in errors* fix test of invoke* Invoke wave enhancements (#10511)* wip* wasmtime::component::Component: add iterator of exports* components_rec* exports_rec gives fully qualified name* invoke works!!!* code motion* more context in errors* fix test of invoke* Finalized enhancements for --invoke: error messages* Testing and documenting --invoke* Update if else re: invoke* Updating to fix truncation possibilities in unwrap_tuple* Add clippy annotation to resolve CI error and leave original code (that makes 2 i64 out of 1 i128 and discards extra bits).* Format (rustfmt --edition 2021) Rust files in this PR.* Removing duplicate code missed from previous conflict resolution* Add more verbose documentation* Add more verbose documentation---------Co-authored-by: Pat Hickey <[email protected]>* Convert docs and error trapping to single quote approach (#10533)* Convert docs and error trapping to single quote approach* Adjust the error message a little* Respond to review (#10544)* cli: search component for export items---------Co-authored-by: Timothy McCallum <[email protected]>
Update wasm-tools crates (#9952)* Update wasm-tools cratesPull in recent updates and denying memory64 support in components duringvalidation.* Fix renamings in wave* Fix some bindgen tests
Upstream wasm-wave instances in wasmtime (#8872)* wasmtime: add wasm-wave instances behind an off-by-default wave feature* cargo vet: trust the usuals, audit beef, exempt logos family* correct
Upstream wasm-wave instances in wasmtime (#8872)* wasmtime: add wasm-wave instances behind an off-by-default wave feature* cargo vet: trust the usuals, audit beef, exempt logos family* correct unrelated comment* wasmtime::component::wasm_wave re-exports the crate* convenience functions: component::Val::{to_wave, from_wave}.* fix componentfunc params