| 1b59b579 | 09-Mar-2026 |
Yordis Prieto <[email protected]> |
Add support for map type (#12216)
* Add support for map type
Signed-off-by: Yordis Prieto <[email protected]>
* Add Map and MapEntry classes to support key/value pairs in component model
Th
Add support for map type (#12216)
* Add support for map type
Signed-off-by: Yordis Prieto <[email protected]>
* Add Map and MapEntry classes to support key/value pairs in component model
This commit introduces the Map and MapEntry classes, enabling the representation of map values in the component model. The Map class allows for the creation and iteration of key/value pairs, enhancing the functionality of the wasmtime component API. Additionally, the .gitignore file is updated to exclude build artifacts from the crates/c-api directory.
* Add wasm_component_model_map configuration support
* Format code
* Format C code
* Enhance component model to support HashMap<K, V> type
This commit introduces support for HashMap<K, V> in the component model, allowing maps to be represented as list<tuple<K, V>> in the canonical ABI. It includes implementations for the ComponentType, Lower, and Lift traits for HashMap, enabling type checking, lowering to flat representations, and lifting from memory. Additionally, the maximum depth for type generation in the fuzzing utility is updated to accommodate the new map type.
* Refactor component configuration to introduce map support
This commit removes the previous wasm features configuration and adds new functions for creating a map-configured engine. The `map_config` and `map_engine` functions are introduced to facilitate the use of the component model with maps in tests, ensuring that the engine is properly configured for map types in the component model.
* Add new WAST test for map types and remove map type definitions from existing tests
This commit introduces a new WAST test file specifically for testing various map types in the component model. Additionally, it removes the redundant map type definitions from the existing types.wast file to streamline the test suite.
* Update component fuzzing and dynamic tests to replace call_and_post_return with call
* Format code
* Refactor HashMap usage in typed.rs to use wasmtime_environ collections
* Fix HashMap initialization and insertion to handle potential errors in typed.rs
* Refactor HashMap handling in typed.rs to use lower_map_iter for improved iteration and memory management. Introduce new implementations for ComponentType, Lower, and Lift traits for std::collections::HashMap, enhancing support for map types in the component model.
* Fix map adapter trampoline compilation and alignment bugs
The translate_map function had two categories of bugs preventing map
adapter trampolines from working:
1. Wasm stack discipline: local_set_new_tmp emits LocalSet which pops
from the stack, but was called when the stack was empty (to
"pre-allocate" locals). Fixed by computing values first, then
calling local_set_new_tmp to consume them—matching translate_list's
pattern. Also removed an erroneous LocalTee that left an orphan
value on the stack. Affected: src_byte_len, dst_byte_len,
cur_src_ptr, cur_dst_ptr.
2. Pointer advancement: after value translation, the pointer still
points at the value start. The code only advanced by trailing
padding instead of value_size + trailing_padding, causing every
loop iteration to re-read the same memory.
Also fixes entry layout to use proper record alignment rules (entry
align = max(key_align, value_align), value at aligned offset).
* Refactor map entry layout calculations to use canonical ABI
* Remove unnecessary clone of map pairs during lowering
Val::Map already holds Vec<(Val, Val)> which derefs to &[(Val, Val)],
matching lower_map's signature directly. The intermediate Vec allocation
and deep clone of every key/value pair was redundant.
* Deduplicate map lift logic between HashMap implementations
* Deduplicate list and map sequence translation scaffolding
* Fix cargo fmt formatting issues
* Deduplicate map typecheck logic
* Deduplicate map lowering with linear_lower_map_to_flat and linear_lower_map_to_memory helpers
* Clean up lift_try_map: use drop, move TryHashMap import to module scope
* Fix CI: arbtest overflow and no-std HashMap lift_map
- component_fuzz: use saturating_sub in generate_hashable_key to prevent
underflow when fuel is 0 and Enum variant is chosen
- typed: remove incorrect ? operators in lift_map for hashbrown::HashMap
(with_capacity and insert don't return Result)
* Store map tuple layout in TypeMap
Compute map entry ABI and value offsets once during type building, and reuse that metadata in runtime map lift/lower paths instead of recalculating tuple layout at each call site.
* Refactor map ABI argument passing
Bundle map lift/lower layout and type metadata into a small MapAbi32 helper so map helper calls stay concise without changing behavior.
* Fix CI: enable component_model_map in fuzzing and handle map in arbitrary_val
The fuzzer's component_api oracle was generating map types but the engine
didn't have the map feature enabled, and arbitrary_val had no arm for
Type::Map. Enable component_model_map in the store helper (matching how
component_model_async is forced on) and implement arbitrary value generation
for map types.
---------
Signed-off-by: Yordis Prieto <[email protected]>
show more ...
|
| b4475438 | 30-Jul-2025 |
Joel Dice <[email protected]> |
refactor `{Stream,Future}|{Reader,Writer}` APIs and internals (#11325)
* refactor `{Stream,Future}|{Reader,Writer}` APIs and internals
This makes a several changes to how `{Stream,Future}|{Reader,W
refactor `{Stream,Future}|{Reader,Writer}` APIs and internals (#11325)
* refactor `{Stream,Future}|{Reader,Writer}` APIs and internals
This makes a several changes to how `{Stream,Future}|{Reader,Writer}` work to
make them more efficient and, in some ways, more ergonomic:
- The background tasks have been removed, allowing reads and writes to complete without task context switching. We now only allocate and use oneshot channels lazily when the other end is not yet ready; this improves real world performance benchmarks (e.g. wasi-http request handling) considerably.
- Instances of `{Stream,Future}Reader` can now be lifted and lowered directly; no need for `Host{Stream,Future}` anymore.
- The type parameter for `Stream{Reader,Writer}` no longer refers to the buffer type -- just the payload type (i.e. `StreamReader<u8>` instead of `StreamReader<Vec<u8>>`), meaning any buffer type may be used for a given read or write operation. This also means the compiler needs help with type inference less often when calling `Instance::stream`.
- Instances of `{Stream,Future}|{Reader,Writer}` now require access to the store in order to be disposed of properly. I've added RAII wrapper structs (`WithAccessor[AndValue]`) to help with this, and also updated `Store::drop` and `Instance::run_concurrent` to ensure the store thread-local is set when dropping futures closing over `&Accessor`s.
- In order to ensure that resources containing `{Stream,Future}|{Reader,Writer}` instances are disposed of properly, I've added `LinkerInstance::resource_concurrent` and have updated `wasmtime-wit-bindgen` to use it. This gives resource drop functions access to a `StoreContextMut` via an `Accessor`, allowing the stream and future handles to be disposed of.
- In order to make this work, I had to change `Accessor::instance` from a `Instance` to an `Option<Instance>`, which is awkward but temporary since we're planning to remove `Accessor::instance` entirely once we've moved concurrent state from `ComponentInstance` to `Store`.
That problem of disposal is definitely the most awkward part of all this. In
simple cases, it's easy enough to ensure that read and write handles are
disposed of properly, but both `wasmtime-wasi` and `wasmtime-wasi-http` have
some pretty complicated functions where handles are passed between tasks and/or
stored inside resources, so it can be tricky to ensure proper disposal on all
code paths. I'm open to ideas for improving this, but I suspect we'll need new
Rust language features (e.g. linear types) to make it truly ergonomic, robust,
and efficient.
While testing the above, I discovered an issue with `Instance::poll_until` such
that it would prematurely give up and return a "deadlock" trap error, believing
that there was no further work to do, even though the future passed to it was
ready to resolve the next time it was polled. I've fixed this by polling it one
last time and only trapping if it returns pending.
Note that I've moved a few associated functions from `ConcurrentState` to
`Instance` (e.g. `guest_drop_writable` and others) since they now need access to
the store; they're unchanged otherwise. Apologies for the diff noise.
Finally, I've tweaked how `wasmtime serve` to poll the guest for content before
handing the response to Hyper, which helps performance by ensuring the first
content chunk can be sent with the same TCP packet as the beginning of the
response.
Signed-off-by: Joel Dice <[email protected]>
fix wasi p3 build and test failures
Signed-off-by: Joel Dice <[email protected]>
use `ManuallyDrop` instead of `Option` in `Dropper`
This allows us to drop its `value` field in-place, i.e. without moving it,
thereby upholding the `Pin` guarantee.
Signed-off-by: Joel Dice <[email protected]>
address review comments
- Remove `DropWithStoreAndValue` and friends; go back to taking a `fn() -> T` parameter in `Instance::future` instead
- Make `DropWithStore::drop[_with]` take `&mut self` instead of `self`
- Make `WithAccessor` and `DropWithStore` private
- Instead, I've added public `Guarded{Stream,Future}{Reader,Writer}` types for RAII
- and also `{Stream,Future}{Reader,Writer}::close[_with]` methods
- Use RAII in `FutureReader::read` and `FutureWriter::write` to ensure handles are dropped if the `Future` is dropped
Signed-off-by: Joel Dice <[email protected]>
* lower host stream/future writes in background task
This avoids unsoundness due to guest realloc calls while there are host embedder
frames on the stack.
Signed-off-by: Joel Dice <[email protected]>
* fix `tcp.rs` regressions
Signed-off-by: Joel Dice <[email protected]>
---------
Signed-off-by: Joel Dice <[email protected]>
show more ...
|