Remove need for explicit `Config::async_support` knob (#12371)

* Refactor component model host function definitions

Push the `async`-ness down one layer.

* Remove need for explicit `Config::async

Remove need for explicit `Config::async_support` knob (#12371)

* Refactor component model host function definitions

Push the `async`-ness down one layer.

* Remove need for explicit `Config::async_support` knob

This commit is an attempt to step towards reconciling "old async" and
"new async" in Wasmtime. The old async style is the original async
support in Wasmtime with `call_async`, `func_wrap_async`, etc, where the
main property is that the store is "locked" during an async operation.
Put another way, a store can only execute at most one async operation at
a time. This is in contrast to "new async" support in Wasmtime with the
component-model-async (WASIp3) support, where stores can have more than
one async operation in flight at once.

This commit does not fully reconcile these differences, but it does
remove one hurdle along the way: `Config::async_support`. Since the
beginning of Wasmtime this configuration knob has existed to explicitly
demarcate a config/engine/store as "this thing requires `async` stuff
internally." This has started to make less and less sense over time
where the line between sync and async has become more murky with WASIp3
where the two worlds comingle. The goal of this commit is to deprecate
`Config::async_support` and make the function not actually do anything.

In isolation this can't simply be done, however, because there are many
load-bearing aspects of Wasmtime that rely on this `async_support` knob.
For example once epochs + yielding are enabled it's required that all
Wasm is executed on a fiber lest it hit an epoch and not know how to
yield. That means that this commit is not a simple removal of
`async_support` but instead a refactoring/rearchitecting of how async is
used internally within Wasmtime. The high-level ideas within Wasmtime
now are:

* A `Store` has a "requires async" boolean stored within it.
* All configuration options which end up requiring async, such as
yielding with epochs, turn this boolean on.
* Creation of host functions which use async
(e.g. `func_wrap_{async,concurrent}`) will also turn this option on.
* Synchronous API entrypoints into Wasmtime ensure that this boolean is
disabled.
* Asynchronous APIs are usable at any time.

This means that the concept of an async store vs a sync store is now
gone. All stores are equally capable of executing sync/async, and the
change now is that dynamically some stores will require that async is
used with certain configuration. Additionally all panicking conditions
around `async_support` have been converted to errors instead. All
relevant APIs already returned an error and things are murky enough now
that it's not necessarily trivial to get this right at the embedder
level. In the interest of avoiding panics all detected async mismatches
are now first-class `wasmtime::Error` values.

The end result of this commit is that `Config::async_support` is a
deprecated `#[doc(hidden)]` function that does nothing. While many
internal changes happened as well as having new tests for all this sort
of behavior this is not expected to have a great impact on external
consumers. In general a deletion of `async_support(true)` is in theory
all that's required. This is intended to make it easier to think about
async/sync/etc in the future with WASIp3 and eventually reconcile
`func_wrap_async` and `func_wrap_concurrent` for example. That's left
for future refactorings however.

prtest:full

* Review comments

* Fix CI failures

show more ...

Wasmtime: generalize `async_stack_zeroing` knob to cover initialization (#10027)

* Wasmtime: generalize `async_stack_zeroing` knob to cover initialization

This commit moves the knob from the `Pooli

Wasmtime: generalize `async_stack_zeroing` knob to cover initialization (#10027)

* Wasmtime: generalize `async_stack_zeroing` knob to cover initialization

This commit moves the knob from the `PoolingInstanceAllocatorConfig` to the
regular `Config` and now controls both whether stacks are zeroed before reuse
and whether they are zeroed before the initial use. The latter doesn't matter
usually, since anonymous mmaps are already zeroed so we don't have to do
anything there, but for no-std environments it is the difference between
manually zeroing the stack or simply using unininitialized memory.

* Fix CLI and test builds

* fix default config value

* fix some more tests

show more ...

Use cmake to build wasmtime-c-api (#9031)

* Use cmake to build wasmtime-c-api

* Properly expose features when building via cmake

* Install all headers to same directory

* Add vets

* attempt to f

Use cmake to build wasmtime-c-api (#9031)

* Use cmake to build wasmtime-c-api

* Properly expose features when building via cmake

* Install all headers to same directory

* Add vets

* attempt to fix ci

* Run all tests on CI

prtest:full

* Set CARGO_BUILD_TARGET; add CMakeLists to package

* Update comment on github action

* Attempt to fix android build

* Fix source dir modifications of c-api build

* Re-add BINARY_DIR option

* Fix build

* Move header installation to a cmake script

Try to avoid dealing with cmake configuration/platforms/etc.

* Tweak build of headers

* Install headers in build dir for examples

* Add cmake files to dist, fix header install dir

---------

Co-authored-by: Alex Crichton <[email protected]>

show more ...

Refactor installation of C API and features supported (#8642)

* Refactor installation of C API and features supported

This commit overhauls and refactors the management of the building of
the C AP

Refactor installation of C API and features supported (#8642)

* Refactor installation of C API and features supported

This commit overhauls and refactors the management of the building of
the C API. Instead of this being script-based it's now done entirely
through CMake and CMake is the primary focus for building the C API. For
example building the C API release artifacts is now done through CMake
instead of through a shell script's `cargo build` and manually moving
artifacts.

The benefits that this brings are:

* The C API now properly hides symbols in its header files that weren't
enabled at build time. This is done through a new build-time generated
`conf.h` templated on a `conf.h.in` file in the source tree.

* The C API's project now supports enabling/disabling Cargo features to
have finer-grained support over what's included (plus auto-management
of the header file).

* Building the C API and managing it is now exclusively done through
CMake. For example invoking `doxygen` now lives in CMake, installation
lives there, etc.

The `CMakeLists.txt` file for the C API is overhauled in the process of
doing this. The build now primarily matches on the Rust target being
built rather than the host target. Additionally installation will now
install both the static and shared libraries instead of just one.
Additionally during this refactoring various bits and pieces of
Android-specific code were all removed. Management of the C toolchain
feels best left in scope of the caller (e.g. configuring `CC_*` env vars
and such) rather than here.

prtest:full

* Don't use `option` for optional strings

* Invert release build check

Also adjust some indentation

* Fix more indentation

* Remove no-longer-used variable

* Reduce duplication in feature macro

show more ...

Fix some typos (#8641)

* occurred

* winch typos

* tests typos

* cli typos

* fuzz typos

* examples typos

* docs typos

* crates/wasmtime typos

* crates/environ typos

* crates/cranelift typos

Fix some typos (#8641)

* occurred

* winch typos

* tests typos

* cli typos

* fuzz typos

* examples typos

* docs typos

* crates/wasmtime typos

* crates/environ typos

* crates/cranelift typos

* crates/test-programs typos

* crates/c-api typos

* crates/cache typos

* crates other typos

* cranelift/codegen/src/isa typos

* cranelift/codegen/src other typos

* cranelift/codegen other typos

* cranelift other typos

* ci js typo

* .github workflows typo

* RELEASES typo

* Fix clang-format documentation line

---------

Co-authored-by: Andrew Brown <[email protected]>

show more ...

add clang format (#7601)

* add clang-format

We chose WebKit style because out of all the builtin styles it seems the
closest to what already exists in wasmtime.

Signed-off-by: Tyler Rockwood <rock

add clang format (#7601)

* add clang-format

We chose WebKit style because out of all the builtin styles it seems the
closest to what already exists in wasmtime.

Signed-off-by: Tyler Rockwood <[email protected]>

* c-api: don't reorder headers

The order here matters

Signed-off-by: Tyler Rockwood <[email protected]>

* c-api: apply clang-format

Signed-off-by: Tyler Rockwood <[email protected]>

* fiber: apply clang-format

Signed-off-by: Tyler Rockwood <[email protected]>

* runtime: apply clang-format

Signed-off-by: Tyler Rockwood <[email protected]>

* examples: apply clang format

Signed-off-by: Tyler Rockwood <[email protected]>

* tests: apply clang-format

Signed-off-by: Tyler Rockwood <[email protected]>

* ci: add clang-format checks

Signed-off-by: Tyler Rockwood <[email protected]>

* clang-format: keep braces on the same line

This is more the existing style

Signed-off-by: Tyler Rockwood <[email protected]>

* remove clang-format

Just use the tool defaults (LLVM)

Signed-off-by: Tyler Rockwood <[email protected]>

* Fix ci name

Signed-off-by: Tyler Rockwood <[email protected]>

* manually reformat a couple of comments

prtest:full

Signed-off-by: Tyler Rockwood <[email protected]>

* disable formatting for doc-wasm.h

Signed-off-by: Tyler Rockwood <[email protected]>

* manually reformat wasmtime.h

Signed-off-by: Tyler Rockwood <[email protected]>

* disable formatting

To prevent a link from being broken

Signed-off-by: Tyler Rockwood <[email protected]>

* examples: fixing build commands

Signed-off-by: Tyler Rockwood <[email protected]>

* fix parameter comment

Signed-off-by: Tyler Rockwood <[email protected]>

---------

Signed-off-by: Tyler Rockwood <[email protected]>

show more ...

Fix header for async fuel yielding (#7414)

This returns `Option<Box<wasmtime_error_t>>` on the Rust side.

Signed-off-by: Tyler Rockwood <[email protected]>

Switch to simpler fuel APIs (#7298)

In an effort to simplify the many fuel related APIs, simplify the
interface here to a single counter with get and set methods.
Additionally the async yield is red

Switch to simpler fuel APIs (#7298)

In an effort to simplify the many fuel related APIs, simplify the
interface here to a single counter with get and set methods.
Additionally the async yield is reduced to an interval of the total fuel
instead of injecting fuel, so it's easy to still reason about how much
fuel is left even with yielding turned on.

Internally this works by keeping two counters - one the VM uses to
increment towards 0 for fuel, the other to track how much is in
"reserve". Then when we're out of gas, we pull from the reserve to
refuel and continue. We use the reserve in two cases: one for overflow
of the fuel (which is an i64 and the API expresses fuel as u64) and the
other for async yieling, which then the yield interval acts as a cap to
how much we can refuel with.

This also means that `get_fuel` can return the full range of `u64`
before this change it could only return up to `i64::MAX`. This is
important because this PR is removing the functionality to track fuel
consumption, and this makes the API less error prone for embedders to
track consumption themselves.

Careful to note that the VM counter that is stored as `i64` can be
positive if an instruction "costs" multiple units of fuel when the fuel
ran out.

prtest:full

Signed-off-by: Tyler Rockwood <[email protected]>

show more ...

c-api: expose memory_init_cow (#7227)

* c-api: expose memory_init_cow

Signed-off-by: Tyler Rockwood <[email protected]>

* c-api: fix type in async.h

I really wish these were autogenerated :/

c-api: expose memory_init_cow (#7227)

* c-api: expose memory_init_cow

Signed-off-by: Tyler Rockwood <[email protected]>

* c-api: fix type in async.h

I really wish these were autogenerated :/

Signed-off-by: Tyler Rockwood <[email protected]>

---------

Signed-off-by: Tyler Rockwood <[email protected]>

show more ...

Introduce API for custom stack memory (#7209)

* Introduce StackMemory and StackMemoryCreator

This allows custom implementations of stack memory to be plugged into
the async functionality for wasmti

Introduce API for custom stack memory (#7209)

* Introduce StackMemory and StackMemoryCreator

This allows custom implementations of stack memory to be plugged into
the async functionality for wasmtime. Currently, stacks are always
mmapped, and this custom allocator allows embedders to use any memory
they would like.

The new APIs are also exposed in the C api.

This has no effect on windows, as our hands are tied with fibers there.

Signed-off-by: Tyler Rockwood <[email protected]>

* Add test for custom host memory

Signed-off-by: Tyler Rockwood <[email protected]>

* fix allocator test

Signed-off-by: Tyler Rockwood <[email protected]>

* Address review comments

Signed-off-by: Tyler Rockwood <[email protected]>

* Fix lint warnings

prtest:full

Signed-off-by: Tyler Rockwood <[email protected]>

* fix windows lint warning

prtest:full

Signed-off-by: Tyler Rockwood <[email protected]>

---------

Signed-off-by: Tyler Rockwood <[email protected]>

show more ...

c-api: Support InstancePre (#7140)

* c-api: Support InstancePre

Signed-off-by: Tyler Rockwood <[email protected]>

* c-api: Consolidate all the instance_pre functionality

Signed-off-by: Tyler

c-api: Support InstancePre (#7140)

* c-api: Support InstancePre

Signed-off-by: Tyler Rockwood <[email protected]>

* c-api: Consolidate all the instance_pre functionality

Signed-off-by: Tyler Rockwood <[email protected]>

* c-api: Add async instantiate support to pre instances

Signed-off-by: Tyler Rockwood <[email protected]>

* c-api: Add star to comment for doxygen

prtest:full

Signed-off-by: Tyler Rockwood <[email protected]>

---------

Signed-off-by: Tyler Rockwood <[email protected]>

show more ...

Async support in the C API (#7106)

* c-api: Add a feature for async

Signed-off-by: Tyler Rockwood <[email protected]>

* c-api: Add support for async config

Signed-off-by: Tyler Rockwood <rock

Async support in the C API (#7106)

* c-api: Add a feature for async

Signed-off-by: Tyler Rockwood <[email protected]>

* c-api: Add support for async config

Signed-off-by: Tyler Rockwood <[email protected]>

* c-api: Add support for calling async functions

Signed-off-by: Tyler Rockwood <[email protected]>

* c-api: Add ability to yield execution of Wasm in a store

Signed-off-by: Tyler Rockwood <[email protected]>

* c-api: Introduce wasmtime_linker_instantiate_async

Signed-off-by: Tyler Rockwood <[email protected]>

* c-api: Support defining async host functions

Signed-off-by: Tyler Rockwood <[email protected]>

* gitignore: ignore cmake cache for examples

Signed-off-by: Tyler Rockwood <[email protected]>

* examples: Add example of async API in C

Signed-off-by: Tyler Rockwood <[email protected]>

* c-api: Consolidate async functionality into a single place

Put all the async stuff in it's own header and own rust source file

Also remove the wasmtime_async_continuation_new function, users can just
allocate it directly.

Signed-off-by: Tyler Rockwood <[email protected]>

* c-api: Make async function safe

Signed-off-by: Tyler Rockwood <[email protected]>

* c-api: Remove wasmtime_call_future_get_results

Signed-off-by: Tyler Rockwood <[email protected]>

* c-api: Simplify CHostCallFuture

Move the result translation and hostcall_val_storage usage into an async
function

Signed-off-by: Tyler Rockwood <[email protected]>

* c-api: Simplify C continuation implementation

Remove the caller, which means that we don't need another struct for the
future implementation.

Signed-off-by: Tyler Rockwood <[email protected]>

* c-api: Improve async.h documentation

Signed-off-by: Tyler Rockwood <[email protected]>

* c-api: Cleanup from previous changes

Signed-off-by: Tyler Rockwood <[email protected]>

* examples: Fix example

Signed-off-by: Tyler Rockwood <[email protected]>

* c-api: Simplify continuation callback

This gives more duality with calling an async function and also means
that the implementation can pretty much mirror the sync version.

Signed-off-by: Tyler Rockwood <[email protected]>

* c-api: Fix async.h documentation

Signed-off-by: Tyler Rockwood <[email protected]>

* c-api: Fix documentation for async.h

Signed-off-by: Tyler Rockwood <[email protected]>

* c-api: Review feedback

Signed-off-by: Tyler Rockwood <[email protected]>

* examples: Downgrade async.cpp example to C++11

Signed-off-by: Tyler Rockwood <[email protected]>

* c-api: initialize continuation with a panic callback

Signed-off-by: Tyler Rockwood <[email protected]>

* prtest:full

Signed-off-by: Tyler Rockwood <[email protected]>

---------

Signed-off-by: Tyler Rockwood <[email protected]>

show more ...