[44.0.0] Merged backports for security advisories (#13007)

* fix(environ): repair unsound StringPool::try_clone()

The 43.0 release introduced a soundness bug in StringPool::try_clone(): the
cloned

[44.0.0] Merged backports for security advisories (#13007)

* fix(environ): repair unsound StringPool::try_clone()

The 43.0 release introduced a soundness bug in StringPool::try_clone(): the
cloned map retains &'static str keys pointing into the original pool's
strings storage. Once the original Linker is dropped those keys dangle.

Cloning a Linker, then dropping the original one, leaves a linker whose
registered imports could no longer be found, causing instantiation to
fail with "unknown import".

Signed-off-by: Flavio Castelli <[email protected]>

* Fix pooling allocator predicate to reset VM permissions

This commit fixes a mistake that was introduced in #9583 where the logic
to reset a linear memory slot in the pooling allocator used the wrong
predicate. Specifically VM permissions must be reset if virtual memory
can be relied on at all, and the preexisting predicate of
`can_elide_bounds_check` was an inaccurate representation of this. The
correct predicate to check is `can_use_virtual_memory`.

* winch: Fix the type of the `table.size` output register

This commit corrects the tagged size of the output of the `table.size`
instruction. Previously this was hardcoded as a 32-bit integer instead
of consulting the table's index type to use the
index-type-sized-register instead.

* winch: Fix a host panic when executing `table.fill`

This commit fixes a possible panic when a Winch-compiled module executes
the `table.fill` instruction. Refactoring in #11254 updated Cranelift
but forgot to update Winch meaning that Winch's indices were still using
the module-level indices instead of the `DefinedTableIndex` space. This
adds some tests and updates Winch's translation to use preexisting
helpers.

* x64: Fix `f64x2.splat` without SSE3

Don't sink a load into `pshufd` which loads 16 bytes, instead force
`put_in_xmm` to ensure only 8 bytes are loaded.

* Properly verify alignment in string transcoding

This commit updates string transcoding between guest modules to properly
verify alignment. Previously alignment was only verified on the first
allocation, not reallocations, which is not spec-compliant. This
additionally fixes a possible host panic when dealing with unaligned
pointers.

* Fix type confusion in AArch64 amode RegScaled folding

* winch: Add add_uextend to perform explicit extension when needed.

This commit fixes an out-of-bounds access caused by the lack zero
extension in the code responsible for calculating the heap address for
loads/stores.

This issue manifests in aarch64 (unlike x64) given that no automatic
extension is performed, resulting in an out-of-bounds access.

An alternative approach is to emit an extend for the index, however
this approach is preferred given that it gives the MacroAssembler
layer better control of how to lower addition, e.g., in aarch64 we can
inline the desired extension in a single instruction.

* winch: Correctly type the result of table.grow

This commit fixes an out-of-bounds access caused by the lack of type
narrowing from the `table.grow` builtin. Without explicit narrowing,
the type is treated as 64-bit value, which could cause issues when
paired with loads/stores.

* Review comments

* Properly handle table index types

Only narrow when dealing with the 64-bit pointer/32-bit tables

* Fix panic with out-of-bounds flags in `Value`

This commit fixes a panic when a component model `Value` is lifted from
a flags value which specifies out-of-bounds bits as 1. This is specified
in the component model to ignore the out-of-bounds bits, which `flags!`
correctly did (and thus `bindgen!`), but `Value` treated out-of-bounds
bits as a panic due to indexing an array.

* Fix bounds checks in FACT's `string_to_compact` method

We need to bounds check the source byte length, not the number of code units.

* Add missing realloc validation in string transcoding

This commit adds a missing validation that a return value of `realloc`
is inbounds during string transcoding. This was accidentally missing on
the transcoding path from `utf8` to `latin1+utf16` which meant that a
nearly-raw pointer could get passed to the host to perform the
transcode.

* winch: Refine zero extension heuristic

This commit refines the zero extension heuristic such that it
unconditionally emits a zero extension when dealing with 32-bit
heaps. This eliminates any ambiguity related to the value of the
memory indices across ISAs.

* Fix failure on 32-bit

* Fix miri test

---------

Signed-off-by: Flavio Castelli <[email protected]>
Co-authored-by: Flavio Castelli <[email protected]>
Co-authored-by: Shun Kashiwa <[email protected]>
Co-authored-by: Saúl Cabrera <[email protected]>
Co-authored-by: Nick Fitzgerald <[email protected]>

show more ...

Verify alignment of returned component-model strings (#12897)

* Verify alignment of returned component-model strings

The lifting code erroneously forgot to check for this. There's no actual
consequ

Verify alignment of returned component-model strings (#12897)

* Verify alignment of returned component-model strings

The lifting code erroneously forgot to check for this. There's no actual
consequence to this in Wasmtime per-se, but it's required in the
component model spec to trap, so a trap is added here.

* Fix tests

* Optimize alignment check

* Fix build

show more ...

Handle OOM in the rest of Wasmtime's non-component, -async, -compilation APIs (#12858)

* Handle OOM in more places in the public API

A bunch of random places:

* Add: `Trap::try_new` to handle OOM

Handle OOM in the rest of Wasmtime's non-component, -async, -compilation APIs (#12858)

* Handle OOM in more places in the public API

A bunch of random places:

* Add: `Trap::try_new` to handle OOM while creating traps
* Use: `TryVec` inside `Func::call_impl_do_call` and `wasm_val_raw_storage` to
hold the args and rets
* Add: `Instance::try_exports` for iterating over an instance's exports while
handling OOM
* `Linker:try_get`, like `Linker::get` but handling OOM
* `Linker:try_get_by_import`, like `Linker::get_by_import` but handling OOM
* Use `try_new` to box things in `SharedMemory::new`
* Use `TryVec` instead of `Vec` in our dynamic tables

* Add OOM tests for most of Wasmtime's public API

Excludes component-, async-, and compilation-related APIs.

* address review feedback

* fix test compilation

* fix c-api

show more ...

Fix bounds check for `error-context.debug-message` (#12879)

* Fix bounds check for `error-context.debug-message`

This fixes a possible subsequent panic when lowering the message itself.

* Review c

Fix bounds check for `error-context.debug-message` (#12879)

* Fix bounds check for `error-context.debug-message`

This fixes a possible subsequent panic when lowering the message itself.

* Review comments

show more ...

Adjust fuel consumption of maps (#12875)

Consume fuel based on the host's representation of memory rather than
the guest's. This matches what arrays do, for example.

Fix a stacked borrows violation in futures/streams copy (#12872)

* Fix a stacked borrows violation in futures/streams copy

This commit fixes an issue in the copy implementation for
futures/streams

Fix a stacked borrows violation in futures/streams copy (#12872)

* Fix a stacked borrows violation in futures/streams copy

This commit fixes an issue in the copy implementation for
futures/streams related to component-model-async. Specifically the added
tests here tripped an error in Miri which is related to intra-component
copies and stacked borrows. This refactoring ends up using `copy_within`
for the intra-component case and `copy_to_nonoverlapping` for the
inter-component case to resolve this issue. This commit additionally
refactors the validation/checks to happen in one location instead of in
a few to ensure that all validation is done up-front in a shared manner.

* Fix CI

* Fix expected error message

* Review comments

show more ...

Fix state of futures/streams after cancellation (#12881)

* Fix state of futures/streams after cancellation

This fixes two related but distinct issues with respect to delivering
events to stream/fut

Fix state of futures/streams after cancellation (#12881)

* Fix state of futures/streams after cancellation

This fixes two related but distinct issues with respect to delivering
events to stream/future handles. First when delivering an event to a
future or a stream the shared code is now more unified into one path.
This fixes an issue with futures where they would always have `done`
flagged as `false` accidentally. This then fixes an additional issue
where this `on_delivery` function wasn't invoked when futures/streams
had their operations cancelled.

* Fix CI

show more ...

Fix double read/writes on stream/future handles (#12873)

* Fix double read/writes on stream/future handles

This should result in a first-class trap, not a bug.

* Fix CI

Fix a copy/paste in an error message (#12880)

Shouldn't show up anyway, but still a minor issue.

Remove some more panics in `concurrent.rs` (#12874)

Downgrade some panics to `bail_bug!` or `?` where appropriate by
propagating `Result<T>` in a few more locations.

Fix error message for out-of-bounds variant discriminants (#12885)

Fixes a minor mistake when loading a `Val` from memory with an
out-of-bounds discriminant.

Fix a panic in `Bytes{,Mut}` `StreamProducer` impls (#12878)

This commit fixes the logic of these `impl`s to match the `Vec`-style
blocks to relinquish the entire buffer to Wasmtime immediately. Thi

Fix a panic in `Bytes{,Mut}` `StreamProducer` impls (#12878)

This commit fixes the logic of these `impl`s to match the `Vec`-style
blocks to relinquish the entire buffer to Wasmtime immediately. This
fixes an issue where `split_off` is called with too large a value which
can panic.

show more ...

Fix a copy/paste typo in `StreamAny` (#12871)

When closing, close a stream, not a future. Fixes the test added here as
well.

Fix logic and double-drop UB in `WriteBuffer` impls (#12870)

For component-model-async this fixes logic and undefined behavior issues
in the implementation of the `WriteBuffer` trait for various typ

Fix logic and double-drop UB in `WriteBuffer` impls (#12870)

For component-model-async this fixes logic and undefined behavior issues
in the implementation of the `WriteBuffer` trait for various types.
Notably all types didn't respect the `unsafe` contract of the
`WriteBuffer::take` method by limiting the number of results to what was
requested. This flawed detail, when coupled with other implementation
details, led to both incorrect results and UB in the case that
`VecBuffer<T>` was used where `T` had a `Drop` implementation. This is
not a security issue because component-model-async is not yet Tier 1 and
thus is exempt from the security issue process, but this is nonetheless
something needed to be fixed.

show more ...

Debugging: add integration test with LLDB and some minor tweaks. (#12856)

This PR adds:

- An integration-test that runs LLDB against the Wasmtime CLI to
verify basic debugging functionality, simi

Debugging: add integration test with LLDB and some minor tweaks. (#12856)

This PR adds:

- An integration-test that runs LLDB against the Wasmtime CLI to
verify basic debugging functionality, similar to the existing
native-debug tests.

- A CI job that runs the above in CI.

- Some minor tweaks to the gdbstub debugger design:
- Rather than the initial single-step to get to the first Wasm
instruction where module(s) will be instantiated into the store
and visible to the debugger, we pre-register modules with the
store eagerly. This avoids the slightly hacky flow and also is a
preparation step for `wasmtime serve` debugging, where we can't
single-step into execution eagerly (because execution doesn't
start at all until an HTTP request arrives).
- Add a separate message-printing path for "debugger info messages",
allowing us to print the "debugger is listening on <PORT>" message
without inheriting stderr for the whole debugger component
environment. This message is necessary for the above integration
test (it parses the message to determine when the debuggee is ready).

show more ...

Use `TryPrimaryMap` in an instance's `OwnedImports` (#12847)

Use `TryBTreeMap` in the store's `ModuleRegistry` (#12846)

Lift/lower refactors inspired by `map<K, V>` (#12753)

In reading over #12216 I found a few related but also somewhat
orthogonal refactors that I wanted to implement, and thus this commit.
Changes he

Lift/lower refactors inspired by `map<K, V>` (#12753)

In reading over #12216 I found a few related but also somewhat
orthogonal refactors that I wanted to implement, and thus this commit.
Changes here are:

* Handling pointer pairs for lists/maps/strings is now more uniform with
a single set of functions doing the lift/lower to flat/memory. Less
"FIXME 4311" spread throughout effectively.

* Conditional defines and organization of the `HashMap`-related impls
were tweaked, for example impls on `HashMap` are now unconditional as
well as `TryHashMap`. Some internals were refactored to in theory
reduce complexity, but this is also subjective.

* Lifting a `map<K, V>` now consumes fuel, a recent change, to ensure
that resource exhaustion in the host is limited.

show more ...

fix: correct various typos (#12807)

Signed-off-by: Ho Kim <[email protected]>

cm-async: Fix cancelling a completed host task (#12797)

Incorrect behavior on Wasmtime's part led to erroneously raising a trap
in the guest about the terminal status of a task being delivered
alrea

cm-async: Fix cancelling a completed host task (#12797)

Incorrect behavior on Wasmtime's part led to erroneously raising a trap
in the guest about the terminal status of a task being delivered
already. This commit refactors some internals to ensure that cancelling
a completed host task is not a failure, it instead just returns that the
host task is indeed completed.

show more ...

Consume hostcall fuel when buffering stream data in the host (#12767)

For guest-to-guest communication stream reads/writes rendezvousing
together will currently copy data through `Val`. This is expe

Consume hostcall fuel when buffering stream data in the host (#12767)

For guest-to-guest communication stream reads/writes rendezvousing
together will currently copy data through `Val`. This is expected to
become more optimized in the future, but for now this needs to consume
the concept of "hostcall fuel" introduced in #12652 to ensure that the
guest can't exhaust memory in the host. This additionally tweaks some
hostcall fuel calculations to more accurately reflect the size of values
on the host, notably by using `size_of::<Thing>()` on the host rather
than the size in the guest.

Closes #12674

show more ...

Enable limiting wasip3 resource limits (#12761)

* Enable limiting wasip3 resource limits

This commit adds a new `Store::concurrent_resource_table` method which
enables getting a handle to the under

Enable limiting wasip3 resource limits (#12761)

* Enable limiting wasip3 resource limits

This commit adds a new `Store::concurrent_resource_table` method which
enables getting a handle to the underlying `ResourceTable` used by the
concurrent implementation of component-model-async. This can in turn be
used to set the max capacity on the table and limit the guest usage of
the table.

Closes #11552

* Adjust features

* Fix imports

show more ...

Set current thread before lowering stream/future reads (#12736)

* Set thread before lowering stream/future reads

* Restore thread

* Make CurrentThread pub(crate)

* fmt

* Remove with_thread

* Re

Set current thread before lowering stream/future reads (#12736)

* Set thread before lowering stream/future reads

* Restore thread

* Make CurrentThread pub(crate)

* fmt

* Remove with_thread

* Remove DS_Store

* Add test

* Check return values

* Move string data

show more ...

refactor: clarify error msg. for de-allocated `ResourceAny` (#12751)

* test: illustrate #12465

* refactor: clarify error msg. for de-allocated `ResourceAny`

The error message used to state that th

refactor: clarify error msg. for de-allocated `ResourceAny` (#12751)

* test: illustrate #12465

* refactor: clarify error msg. for de-allocated `ResourceAny`

The error message used to state that this happens because the type of
the resource changed, but actually this can also happen with the very
same type if the resource was already de-allocated. Clarifying the error
message might safe some `wasmtime` users some long debugging sessions.

See #12465.

show more ...

Add support for map type (#12216)

* Add support for map type

Signed-off-by: Yordis Prieto <[email protected]>

* Add Map and MapEntry classes to support key/value pairs in component model

Th

Add support for map type (#12216)

* Add support for map type

Signed-off-by: Yordis Prieto <[email protected]>

* Add Map and MapEntry classes to support key/value pairs in component model

This commit introduces the Map and MapEntry classes, enabling the representation of map values in the component model. The Map class allows for the creation and iteration of key/value pairs, enhancing the functionality of the wasmtime component API. Additionally, the .gitignore file is updated to exclude build artifacts from the crates/c-api directory.

* Add wasm_component_model_map configuration support

* Format code

* Format C code

* Enhance component model to support HashMap<K, V> type

This commit introduces support for HashMap<K, V> in the component model, allowing maps to be represented as list<tuple<K, V>> in the canonical ABI. It includes implementations for the ComponentType, Lower, and Lift traits for HashMap, enabling type checking, lowering to flat representations, and lifting from memory. Additionally, the maximum depth for type generation in the fuzzing utility is updated to accommodate the new map type.

* Refactor component configuration to introduce map support

This commit removes the previous wasm features configuration and adds new functions for creating a map-configured engine. The `map_config` and `map_engine` functions are introduced to facilitate the use of the component model with maps in tests, ensuring that the engine is properly configured for map types in the component model.

* Add new WAST test for map types and remove map type definitions from existing tests

This commit introduces a new WAST test file specifically for testing various map types in the component model. Additionally, it removes the redundant map type definitions from the existing types.wast file to streamline the test suite.

* Update component fuzzing and dynamic tests to replace call_and_post_return with call

* Format code

* Refactor HashMap usage in typed.rs to use wasmtime_environ collections

* Fix HashMap initialization and insertion to handle potential errors in typed.rs

* Refactor HashMap handling in typed.rs to use lower_map_iter for improved iteration and memory management. Introduce new implementations for ComponentType, Lower, and Lift traits for std::collections::HashMap, enhancing support for map types in the component model.

* Fix map adapter trampoline compilation and alignment bugs

The translate_map function had two categories of bugs preventing map
adapter trampolines from working:

1. Wasm stack discipline: local_set_new_tmp emits LocalSet which pops
from the stack, but was called when the stack was empty (to
"pre-allocate" locals). Fixed by computing values first, then
calling local_set_new_tmp to consume them—matching translate_list's
pattern. Also removed an erroneous LocalTee that left an orphan
value on the stack. Affected: src_byte_len, dst_byte_len,
cur_src_ptr, cur_dst_ptr.

2. Pointer advancement: after value translation, the pointer still
points at the value start. The code only advanced by trailing
padding instead of value_size + trailing_padding, causing every
loop iteration to re-read the same memory.

Also fixes entry layout to use proper record alignment rules (entry
align = max(key_align, value_align), value at aligned offset).

* Refactor map entry layout calculations to use canonical ABI

* Remove unnecessary clone of map pairs during lowering

Val::Map already holds Vec<(Val, Val)> which derefs to &[(Val, Val)],
matching lower_map's signature directly. The intermediate Vec allocation
and deep clone of every key/value pair was redundant.

* Deduplicate map lift logic between HashMap implementations

* Deduplicate list and map sequence translation scaffolding

* Fix cargo fmt formatting issues

* Deduplicate map typecheck logic

* Deduplicate map lowering with linear_lower_map_to_flat and linear_lower_map_to_memory helpers

* Clean up lift_try_map: use drop, move TryHashMap import to module scope

* Fix CI: arbtest overflow and no-std HashMap lift_map

- component_fuzz: use saturating_sub in generate_hashable_key to prevent
underflow when fuel is 0 and Enum variant is chosen
- typed: remove incorrect ? operators in lift_map for hashbrown::HashMap
(with_capacity and insert don't return Result)

* Store map tuple layout in TypeMap

Compute map entry ABI and value offsets once during type building, and reuse that metadata in runtime map lift/lower paths instead of recalculating tuple layout at each call site.

* Refactor map ABI argument passing

Bundle map lift/lower layout and type metadata into a small MapAbi32 helper so map helper calls stay concise without changing behavior.

* Fix CI: enable component_model_map in fuzzing and handle map in arbitrary_val

The fuzzer's component_api oracle was generating map types but the engine
didn't have the map feature enabled, and arbitrary_val had no arm for
Type::Map. Enable component_model_map in the store helper (matching how
component_model_async is forced on) and implement arbitrary value generation
for map types.

---------

Signed-off-by: Yordis Prieto <[email protected]>

show more ...