Add wasi:http support to the C API (#12950)

Remove need for explicit `Config::async_support` knob (#12371)

* Refactor component model host function definitions

Push the `async`-ness down one layer.

* Remove need for explicit `Config::async

Remove need for explicit `Config::async_support` knob (#12371)

* Refactor component model host function definitions

Push the `async`-ness down one layer.

* Remove need for explicit `Config::async_support` knob

This commit is an attempt to step towards reconciling "old async" and
"new async" in Wasmtime. The old async style is the original async
support in Wasmtime with `call_async`, `func_wrap_async`, etc, where the
main property is that the store is "locked" during an async operation.
Put another way, a store can only execute at most one async operation at
a time. This is in contrast to "new async" support in Wasmtime with the
component-model-async (WASIp3) support, where stores can have more than
one async operation in flight at once.

This commit does not fully reconcile these differences, but it does
remove one hurdle along the way: `Config::async_support`. Since the
beginning of Wasmtime this configuration knob has existed to explicitly
demarcate a config/engine/store as "this thing requires `async` stuff
internally." This has started to make less and less sense over time
where the line between sync and async has become more murky with WASIp3
where the two worlds comingle. The goal of this commit is to deprecate
`Config::async_support` and make the function not actually do anything.

In isolation this can't simply be done, however, because there are many
load-bearing aspects of Wasmtime that rely on this `async_support` knob.
For example once epochs + yielding are enabled it's required that all
Wasm is executed on a fiber lest it hit an epoch and not know how to
yield. That means that this commit is not a simple removal of
`async_support` but instead a refactoring/rearchitecting of how async is
used internally within Wasmtime. The high-level ideas within Wasmtime
now are:

* A `Store` has a "requires async" boolean stored within it.
* All configuration options which end up requiring async, such as
yielding with epochs, turn this boolean on.
* Creation of host functions which use async
(e.g. `func_wrap_{async,concurrent}`) will also turn this option on.
* Synchronous API entrypoints into Wasmtime ensure that this boolean is
disabled.
* Asynchronous APIs are usable at any time.

This means that the concept of an async store vs a sync store is now
gone. All stores are equally capable of executing sync/async, and the
change now is that dynamically some stores will require that async is
used with certain configuration. Additionally all panicking conditions
around `async_support` have been converted to errors instead. All
relevant APIs already returned an error and things are murky enough now
that it's not necessarily trivial to get this right at the embedder
level. In the interest of avoiding panics all detected async mismatches
are now first-class `wasmtime::Error` values.

The end result of this commit is that `Config::async_support` is a
deprecated `#[doc(hidden)]` function that does nothing. While many
internal changes happened as well as having new tests for all this sort
of behavior this is not expected to have a great impact on external
consumers. In general a deletion of `async_support(true)` is in theory
all that's required. This is intended to make it easier to think about
async/sync/etc in the future with WASIp3 and eventually reconcile
`func_wrap_async` and `func_wrap_concurrent` for example. That's left
for future refactorings however.

prtest:full

* Review comments

* Fix CI failures

show more ...

Fill out more of the C++ API for components (#11889)

* capi: Add a C++ API for `component::Linker`

Just a bare-bones API for now with functionality that's possible to fill
out. Notably instantiatio

Fill out more of the C++ API for components (#11889)

* capi: Add a C++ API for `component::Linker`

Just a bare-bones API for now with functionality that's possible to fill
out. Notably instantiation and defining functions is not yet possible in
C++. Some more feature-parity is also added with the core linkers as
well.

* capi: Add bindings for component instances

* capi: Delete wasip2.h header file

Upon reflection I realize that this is not actually necessary and is
otherwise a duplicate of the functionality in `wasi.h`. All of the
functionality in `wasi.h` is already supported to power WASIp2-defined
APIs in a linker, which is enabled by the `WasiView` trait redirecting
to the `WasiView for WasiP1Ctx` implementation. This is similar to how
the `wasmtime` CLI works where a P1 context is always created and then
it's conditionally used for either core wasm or components.

Effectively this is a deletion of duplicate functionality in the C API
but no underlying functionality is lost. Translating information to
preexisting WASI calls will work the same as using the wasip2 APIs before.

* capi: Start bindings for component functions

* capi: Bind component values in the C++ API

* capi: Finish bindings for component functions/linkers

All the pieces are now in place to use the C++ API in testing.

* Fix compilation of CLI

* Fix compile on MSVC

* Try again to fix msvc compat

* Fix `get_f32` and `get_f64` return values

* Fix signed return values

* Remove no-longer-needed `capi_transfer` function

* Try to fix msvc again

* Remove std::optional constructor of `Val`

* Fix another namespace clash on msvc

* One day I surely may understand a fraction of either C++ or MSVC, but
today is not that day.

* Document internal macro

* Add other internal docs

show more ...

Rename "preview{0,1}" in `wasmtime-wasi` to "p{0,1}" (#11380)

* Rename "preview{0,1}" in `wasmtime-wasi` to "p{0,1}"

This commit renames the `preview1` module and features to `p1` and does
the same

Rename "preview{0,1}" in `wasmtime-wasi` to "p{0,1}" (#11380)

* Rename "preview{0,1}" in `wasmtime-wasi` to "p{0,1}"

This commit renames the `preview1` module and features to `p1` and does
the same for `preview0`. This additionally cleans up the test suite a
bit to share more code amongst all the implementaitons and to also move
the p1 tests out of the p2 folder.

This additionally adds a `p2` feature to the `wasmtime-wasi` crate but
it does not currently gate the `p2` module because that'll require some
more refactoring an annotations to get that working.

* Fix build of the CLI

* Fix build of the C API

* Fix bench-api build

* Fix build of examples

* More renamings

show more ...

Unify WASIp{2,3} context structures (#11370)

This removes `wasmtime_wasi::p{2,3}::{WasiCtx, WasiCtxBuilder,
WasiView}` in favor of only having `wasmtime_wasi::{WasiCtx,
WasiCtxBuilder, WasiView}` in

Unify WASIp{2,3} context structures (#11370)

This removes `wasmtime_wasi::p{2,3}::{WasiCtx, WasiCtxBuilder,
WasiView}` in favor of only having `wasmtime_wasi::{WasiCtx,
WasiCtxBuilder, WasiView}` instead. Conceptually these revisions of WASI
all provide the same functionality just with a different veneer that the
component model offers, so having only one way to configure host-side
behavior will make it easier to both organize implementations internally
(e.g. more sharing of code) as well as for embedders to configure (only
one context to create/manage).

show more ...

Simplify WASI internal implementations (#11365)

* Simplify WASI internal implementations

This commit migrates the WASIp2 implementation to be closer to the
upcoming WASIp3 implementation in terms o

Simplify WASI internal implementations (#11365)

* Simplify WASI internal implementations

This commit migrates the WASIp2 implementation to be closer to the
upcoming WASIp3 implementation in terms of how things are implemented
internally. Previously the way things worked with WASIp2 is:

* Embedders call `add_to_linker` with `T: WasiView`
* Internally `add_to_linker` is called which creates `WasiImpl<&mut T>`
* All internal implementations were `impl<T> Host for WasiImpl<T> where T: WasiView`
* A forwarding impl of `impl<T: WasiView> WasiView for &mut T` was
required

While this all worked it's a bit complicated for a few reasons:

1. Dealing with generically named structures like `WasiImpl` (or
`IoImpl` or `WasiHttpImpl`) is a bit baroque and not always obvious
as to what's going on.
2. The extra layer of generics in `impl<T> Host for WasiImpl<T>` adds a
layer of conceptual indirection which is non-obvious.
3. Other WASI proposal implementations do not use this strategy and
instead use "view" types or `impl Host for TheType` for example.
4. Internal incantations of `add_to_linker` had to deal with mixtures of
`IoImpl` and `WasiImpl` and aligning everything just right.
5. An extra layer of generics on all impls meant that everything was
generic meaning that `wasmtime-wasi`-the-crate didn't generate much
code, causing longer codegen times for consumers.

The goal of this commit is to migrate towards the style of what WASIp3
is prototyping for how impls are modeled. This is done to increase the
amount of code that can be shared between WASIp2 and WASIp3. This has a
number of benefits such as being easier to understand and also being
more modular where `wasi:clocks` implementations of traits don't require
filesystem context to be present (as is the case today). This in theory
helps a more mix-and-match paradigm of blending together various bits
and pieces of `wasmtime-wasi` implementations.

Concretely the changes made here are:

* `WasiView` no longer inherits from `IoView`, they're unrelated traits
now.
* `WasiView` now returns `WasiViewCtx<'a>` which has `ctx: &'a mut WasiCtx`
and `table: &'a mut ResourceTable`. That means it basically does the
same thing before but in a slightly different fashion.
* Implementations of `Host` traits are now directly for
`WasiCtxView<'_>` and don't involve any generics at all. These are
hopefully easier to understand and also better from a
codegen/compile-time perspective.
* Embedders no longer need to implement `IoView` directly and instead
fold that functionality into `WasiView`.
* `WasiHttpView` no longer inherits from `IoView` and instead has a
direct `fn table` method. Additionally `WasiHttpImpl` no longer embeds
`IoImpl` inside of it.
* Host traits for `wasi:io` are now implemented directly for
`ResourceTable` instead of `IoImpl<T>`.

The immediate goal of this refactoring is to enable more sharing along
the lines of #11362. This was not possible prior because WASIp3 requires
a simultaneous borrow on the table/ctx while the trait hierarchy
previously gave you one-or-the-other. With this new organization it will
be possible to get both at the same time meaning more
structure/contexts/etc can be shared between implementations.

prtest:full

* CI fixes

* More CI fixes

* More CI fixes

show more ...

c-api: component-model: Resource table, WASI (#11055)

* c-api: component-model: Resource table, WASI

* WASIP2 context builder

* Add include

* Rename function

* Add a simple test

* Add comments

c-api: component-model: Resource table, WASI (#11055)

* c-api: component-model: Resource table, WASI

* WASIP2 context builder

* Add include

* Rename function

* Add a simple test

* Add comments prtest:full

show more ...

Enable the `useless_conversion` Clippy lint (#10838)

* Enable the `useless_conversion` Clippy lint

We've got lots of types in Wasmtime and convert between them quite a
lot, but often over time conv

Enable the `useless_conversion` Clippy lint (#10838)

* Enable the `useless_conversion` Clippy lint

We've got lots of types in Wasmtime and convert between them quite a
lot, but often over time conversions become unnecessary through
refactorings or similar. This will hopefully enable us to clean up some
conversions as they come up to try to have as few as possible ideally.

* Review comments

show more ...

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtim

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtime to the 2024 Edition to keep
up-to-date with Rust idioms and access many of the edition features
exclusive to the 2024 edition.

prtest:full

* Reformat with the 2024 edition

show more ...

Reuse Wasm linear memories code for GC heaps (#10503)

* Reuse code for Wasm linear memories for GC heaps

Instead of bespoke code paths and structures for Wasm GC, this commit makes it
so that we no

Reuse Wasm linear memories code for GC heaps (#10503)

* Reuse code for Wasm linear memories for GC heaps

Instead of bespoke code paths and structures for Wasm GC, this commit makes it
so that we now reuse VM structures like `VMMemoryDefinition` and bounds-checking
logic. Notably, we also reuse all the associated bounds-checking optimizations
and, when possible, virtual-memory techniques to completely elide them.

Furthermore, this commit adds support for growing GC heaps, reusing the
machinery for growing memories, and makes it so that GC heaps always start out
empty. This allows us to properly delay allocating the GC heap's storage until a
GC object is actually allocated.

Fixes #9350

* fix c api compilation

* use assert_contains

* remove no-longer-necessary extra memory config from limiter tests

* Helper for retry-after-maybe-async-gc in libcalls

* Clean up some comments

* fix wasmtime-fuzzing and no-gc compilation

* fix examples

* fix no-gc+compiler build

* fix build without pooling allocator

* fix +cranelift +gc-drc -gc-null builds

* fix table hash key stability test

* fix oracle usage of `ExternRef::new`

* fix +gc -gc-null -gc-drc build

* fix wasmtime-fuzzing

* make `StorePtr` wrap a `NonNull`

* Fix some doc tests

* Remove some unnecessary retry helpers now that `FooRef::new` will auto-gc

* fix things after rebase

* Reorganize collection/growth methods for GC heap

* rename BoundsCheck variants

* fix cfg'ing of gc only code

* Fix doc tests

* fix one more gc cfg

* disable GC heap OOM test on non-64-bit targets

show more ...

Enable `unsafe-attr-outside-unsafe` 2024 edition lint (#9964)

* Enable `unsafe-attr-outside-unsafe` 2024 edition lint

This commit enables the `unsafe-attr-outside-unsafe` lint in rustc used
in tran

Enable `unsafe-attr-outside-unsafe` 2024 edition lint (#9964)

* Enable `unsafe-attr-outside-unsafe` 2024 edition lint

This commit enables the `unsafe-attr-outside-unsafe` lint in rustc used
in transitioning to the 2024 edition. This requires that the
`#[no_mangle]` attribute is replaced in favor of `#[unsafe(no_mangle)]`.
This mostly affects the C API of wasmtime and most of the changes here
are a simple search/replace.

* Another attribute update

* Fix command adapter build

show more ...

Implement the table64 extension to the memory64 proposal (#9206)

This commit implements the table64 extention in both Wasmtime and
Cranelift.

Most of the work was changing a bunch of u32 values to

Implement the table64 extension to the memory64 proposal (#9206)

This commit implements the table64 extention in both Wasmtime and
Cranelift.

Most of the work was changing a bunch of u32 values to u64/usize.
The decisions were made in align with the PR #3153 which
implemented the memory64 propsal itself.

One significant change was the introduction of `IndexType`
and `Limits` which streamline and unify the handling of limits
for both memories and tables.

The spec and fuzzing tests related to table64 are re-enabled which
provides a good coverage of the feature.

show more ...

Enforce `uninlined_format_args` for the workspace (#9065)

* Enforce `uninlined_format_args` for the workspace

* fix: failing `Monolith Checks` job

* fix: formatting

wasmtime-c-api: switch from wasi-common to wasmtime-wasi (#8066)

* wasmtime-c-api: switch from using wasi-common to wasmtime-wasi

* Fix WasiP1Ctx references, and stop eagerly opening dirs for preop

wasmtime-c-api: switch from wasi-common to wasmtime-wasi (#8066)

* wasmtime-c-api: switch from using wasi-common to wasmtime-wasi

* Fix WasiP1Ctx references, and stop eagerly opening dirs for preopens

* Add OutputFile to skip async writes in stdout/stderr

---------

Co-authored-by: Trevor Elliott <[email protected]>

show more ...

c-api: Better differentiate between `wasm.h` and `wasmtime.h` APIs (#8344)

This renames some types and adds some type aliases to help us better distinguish
between `wasm.h` APIs and `wasmtime.h` API

c-api: Better differentiate between `wasm.h` and `wasmtime.h` APIs (#8344)

This renames some types and adds some type aliases to help us better distinguish
between `wasm.h` APIs and `wasmtime.h` APIs, primarily for `Store`-related
types. In general, `WasmFoo` is related to `wasm.h` and `WasmtimeFoo` is related
to `wasmtime.h`.

* `StoreRef` -> `WasmStoreRef`
* Introduce the `WasmStore[Data]` and `WasmStoreContext[Mut]` aliases
* `StoreData` -> `WasmtimeStoreData`
* `CStoreContext[Mut]` -> `WasmtimeStoreContext[Mut]`
* Introduce the `Wasmtime{Store,Caller}` aliases

show more ...

Make wasi-common self-contained, deprecate exports from wasmtime-wasi (#7881)

* WIP: try to make wasi-common self contained.

* rebase: cargo.lock

* remove all dependencies between wasi-common and

Make wasi-common self-contained, deprecate exports from wasmtime-wasi (#7881)

* WIP: try to make wasi-common self contained.

* rebase: cargo.lock

* remove all dependencies between wasi-common and wasmtime-wasi

* use wasi-common directly throughout tests, benches, examples, cli run

* wasi-threads: use wasi-common's maybe_exit_on_error in spawned thread

not a very modular design, but at this point wasi-common and
wasi-threads are forever wed

* fix wasmtime's docs

* re-introduce wasmtime-wasi's exports of wasi-common definitions behind deprecated

* factor out determining i32 process exit code

and remove libc dep because rustix provides the same constant

* commands/run: inline the logic about aborting on trap

since this is the sole place in the codebase its used

* Add high-level summary to wasi-common's top-level doc comment.

* c-api: fix use of wasi_cap_std_sync => wasi_common::sync, wasmtime_wasi => wasi_common

* fix tokio example

* think better of combining downcast and masking into one method

* fix references to wasmtime_wasi in docs

prtest:full

* benches: use wasi-common

* cfg-if around use of rustix::process because that doesnt exist on windows

* wasi-common: include tests, caught by verify-publish

* fix another bench

* exit requires wasmtime dep. caught by verify-publish.

show more ...

c-api: support yielding in wasmtime_store_epoch_deadline_callback (#7476)

* c-api: reuse CallbackDataPtr

Signed-off-by: Tyler Rockwood <[email protected]>

* c-api: support yielding in epoch de

c-api: support yielding in wasmtime_store_epoch_deadline_callback (#7476)

* c-api: reuse CallbackDataPtr

Signed-off-by: Tyler Rockwood <[email protected]>

* c-api: support yielding in epoch deadline callback

Signed-off-by: Tyler Rockwood <[email protected]>

* Use triple slash doc comments

prtest:full

* c-api: guard yield behind async cfg flag

Signed-off-by: Tyler Rockwood <[email protected]>

* c-api: prevent dead code warnings with no default features

Signed-off-by: Tyler Rockwood <[email protected]>

---------

Signed-off-by: Tyler Rockwood <[email protected]>

show more ...

Switch to simpler fuel APIs (#7298)

In an effort to simplify the many fuel related APIs, simplify the
interface here to a single counter with get and set methods.
Additionally the async yield is red

Switch to simpler fuel APIs (#7298)

In an effort to simplify the many fuel related APIs, simplify the
interface here to a single counter with get and set methods.
Additionally the async yield is reduced to an interval of the total fuel
instead of injecting fuel, so it's easy to still reason about how much
fuel is left even with yielding turned on.

Internally this works by keeping two counters - one the VM uses to
increment towards 0 for fuel, the other to track how much is in
"reserve". Then when we're out of gas, we pull from the reserve to
refuel and continue. We use the reserve in two cases: one for overflow
of the fuel (which is an i64 and the API expresses fuel as u64) and the
other for async yieling, which then the yield interval acts as a cap to
how much we can refuel with.

This also means that `get_fuel` can return the full range of `u64`
before this change it could only return up to `i64::MAX`. This is
important because this PR is removing the functionality to track fuel
consumption, and this makes the API less error prone for embedders to
track consumption themselves.

Careful to note that the VM counter that is stored as `i64` can be
positive if an instruction "costs" multiple units of fuel when the fuel
ran out.

prtest:full

Signed-off-by: Tyler Rockwood <[email protected]>

show more ...

Support reset_fuel in store APIs (#7240)

* Support set_fuel in store APIs

Fixes: https://github.com/bytecodealliance/wasmtime/issues/5109

Signed-off-by: Tyler Rockwood <[email protected]>

* r

Support reset_fuel in store APIs (#7240)

* Support set_fuel in store APIs

Fixes: https://github.com/bytecodealliance/wasmtime/issues/5109

Signed-off-by: Tyler Rockwood <[email protected]>

* rename set_fuel to reset_fuel

To make it more clear that consumed fuel is being reset.

Signed-off-by: Tyler Rockwood <[email protected]>

* update out of date documentation for fuel in C API

Signed-off-by: Tyler Rockwood <[email protected]>

---------

Signed-off-by: Tyler Rockwood <[email protected]>

show more ...

Allow async yield from epoch interruption callback (#6464)

* Allow async yield from epoch interruption callback

When an epoch interruption deadline arrives, previously it was possible
to yield to t

Allow async yield from epoch interruption callback (#6464)

* Allow async yield from epoch interruption callback

When an epoch interruption deadline arrives, previously it was possible
to yield to the async executor, or to invoke a callback on the wasm
stack, but not both. This changes the API to allow callbacks to run and
then request yielding to the async executor.

* Fix Wasmtime C API implementation

show more ...

Expose remaining fuel (#6445)

* Expose fuel_remaining for store

* Add tests for fuel_remaining

add C API for epoch_deadline_callback and wasmtime_error_t creation (#6359)

* add C API for epoch_dead_callback and wasmtime_error_t creation

* revise C API wasmtime_error_new msg encoding

Expose wasmtime_store_limiter in the c-api (#5761)

c-api: refactor definitions with macroes (#4416)

Signed-off-by: spacewander <[email protected]>

Delete historical interruptable support in Wasmtime (#3925)

* Delete historical interruptable support in Wasmtime

This commit removes the `Config::interruptable` configuration along with
the `In

Delete historical interruptable support in Wasmtime (#3925)

* Delete historical interruptable support in Wasmtime

This commit removes the `Config::interruptable` configuration along with
the `InterruptHandle` type from the `wasmtime` crate. The original
support for adding interruption to WebAssembly was added pretty early on
in the history of Wasmtime when there was no other method to prevent an
infinite loop from the host. Nowadays, however, there are alternative
methods for interruption such as fuel or epoch-based interruption.

One of the major downsides of `Config::interruptable` is that even when
it's not enabled it forces an atomic swap to happen when entering
WebAssembly code. This technically could be a non-atomic swap if the
configuration option isn't enabled but that produces even more branch-y
code on entry into WebAssembly which is already something we try to
optimize. Calling into WebAssembly is on the order of a dozens of
nanoseconds at this time and an atomic swap, even uncontended, can add
up to 5ns on some platforms.

The main goal of this PR is to remove this atomic swap on entry into
WebAssembly. This is done by removing the `Config::interruptable` field
entirely, moving all existing consumers to epochs instead which are
suitable for the same purposes. This means that the stack overflow check
is no longer entangled with the interruption check and perhaps one day
we could continue to optimize that further as well.

Some consequences of this change are:

* Epochs are now the only method of remote-thread interruption.
* There are no more Wasmtime traps that produces the `Interrupted` trap
code, although we may wish to move future traps to this so I left it
in place.
* The C API support for interrupt handles was also removed and bindings
for epoch methods were added.
* Function-entry checks for interruption are a tiny bit less efficient
since one check is performed for the stack limit and a second is
performed for the epoch as opposed to the `Config::interruptable`
style of bundling the stack limit and the interrupt check in one. It's
expected though that this is likely to not really be measurable.
* The old `VMInterrupts` structure is renamed to `VMRuntimeLimits`.

show more ...