Do not re-export `anyhow!` in the `wasmtime::prelude` (#12298)

We are trying to move to `format_err!` instead.

Add `Config::without_compiler` (#12089)

* Add `Config::without_compiler`

This provides us an allocation-free path for constructing a `Config`.

Why not rely on builds when `cfg(not(any(feature = "c

Add `Config::without_compiler` (#12089)

* Add `Config::without_compiler`

This provides us an allocation-free path for constructing a `Config`.

Why not rely on builds when `cfg(not(any(feature = "cranelift", feature =
"winch")))`? Because we need to test our various OOM-handling and
allocation-free code paths in this workspace, and without some way to create a
config without a compiler otherwise, cargo's feature resolver will enable those
features in the workspace, enabling the compiler in the config, but we don't
intend to make compiler configurations handle OOM.

* Quiet unused mut warnings

* Move default compiler flags into builder

show more ...

`CodeBuilder` APIs for defining compile-time builtins (#11918)

* `CodeBuilder` APIs for defining compile-time builtins

Compile-time builtins, as described in [the
RFC](https://github.com/bytecodeal

`CodeBuilder` APIs for defining compile-time builtins (#11918)

* `CodeBuilder` APIs for defining compile-time builtins

Compile-time builtins, as described in [the
RFC](https://github.com/bytecodealliance/rfcs/blob/main/accepted/wasmtime-compile-time-builtins.md),
are effectively the sum of three parts:

1. Function inlining
2. Unsafe intrinsics
3. Component composition

The first two have already been implemented in Wasmtime. This commit implements
the final part, leveraging `wasm-compose` to link host-defined compile-time
builtin components with guest-defined main components. It exposes Wasmtime's
unsafe intrinsics only to the host-defined compile-time builtins, not the
guest-defined main Wasm component.

Why `wasm-compose` and not `wac`? Because it is in the same repo as the rest of
the `wasm-tools` crates, and therefore it is easy to depend on without bringing
in duplicate copies of that family of crates into our workspace and builds. Also
its programmatic API is somewhat easier to use, and is not spread across
multiple crates.

* Fix unused lifetime in `cfg(not(feature = "compile-time-builtins"))`

* Fix an unused warning when `cfg(not(feature = "compile-time-builtins"))`

* Add cargo vet audit for `fixedbitset` version `0.4.2`

* Add cargo vet audit for `bitmaps` version 2.10.0

* Add cargo vet audit for `sized-chunks` diff `0.6.5 -> 0.7.0`

* Add/tweak cargo vet exemptions for some deps

These all have >10,000 daily downloads, and so are okay to exempt per our
policy:
https://docs.wasmtime.dev/contributing-coding-guidelines.html#policy-for-adding-cargo-vet-entries

* Add cargo vet audit for `wasm-compose` diff `0.236.0 -> 0.238.1`

Don't know why I have to do this certification even though we have a wildcard
audit for this crate because it is authored by the Bytecode Alliance...

* Fix visibility of type

* Move compile-time builtins methods to module to cut down on `cfg`s

* Skip checking `heck` in `cargo deny`

Until https://github.com/bytecodealliance/wasm-tools/pull/2359 is merged,
released, and updated in this tree.

* Always read compile-time inputs

* Tighten up parse loop and level tracking

* as_deref instead of clone

* Remove 'b lifetime from CodeBuilder

* fix `cfg(not(feature = "compile-time-builtins"))` build

* Ignore tests that run wasm when in MIRI

show more ...

Implement unsafe intrinsics for compile-time builtins (#11825)

* Implement unsafe intrinsics for compile-time builtins

This commit adds the extremely unsafe
`wasmtime::CodeBuilder::expose_unsafe_in

Implement unsafe intrinsics for compile-time builtins (#11825)

* Implement unsafe intrinsics for compile-time builtins

This commit adds the extremely unsafe
`wasmtime::CodeBuilder::expose_unsafe_intrinsics` method. When enabled, the Wasm
being compiled is given access to special imports that correspond to direct,
unchecked and unsandboxed, native load and store operations. These intrinsics
are intended to be used for implementing fast, inline-able versions of WASI
interfaces that are special-cased to a particular host embedding, for example.

Compile-time builtins, as originally described in [the
RFC](https://github.com/bytecodealliance/rfcs/pull/43), are basically made up of
three parts:

1. A function inliner
2. Unsafe intrinsics
3. Component composition to encapsulate the usage of unsafe intrinsics in a safe
interface

Part (1) has been implemented in Wasmtime and Cranelift for a little while
now (see `wasmtime::Config::compiler_inlining`). This commit is part (2). After
this commit lands, part (3) can be done with `wac` and `wasm-compose`, although
follow up work is required to make the developer experience nicer and more
integrated into Wasmtime so that the APIs can look like those proposed in the
RFC.

* fill out some more docs

* fix non component model builds

* start filling out the doc example

* Factor abi params/returns out; truncate/extend pointers

* Compile unsafe intrinsics on winch as well

* prtest:full

* have the macro define the signature

* ignore tests in MIRI because MIRI can't compile Wasm

* juggle pointer provenance in `Store::data[_mut]`

* add a test for store data provenance and also fix it

* use `VmPtr` for the store data pointer

* finish writing unsafe intrinsics example

* fix up docs and rules around only accessing data from `T` in a `Store<T>`

* Only reserve space for the intrinsics' `VMFuncRef`s if they are in use

* use dangling pointers instead of options

* Rename `StoreInner::data` to `data_no_provenance` and fix some accesses to use the method accessors

* Add comments about the provenance juggling inside `StoreInner::data[_mut]`

* only compile intrinsics that are used

Turns out we don't need to add phases, we already have the info available to do
this.

* fix duplicate symbol names

show more ...

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtim

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtime to the 2024 Edition to keep
up-to-date with Rust idioms and access many of the edition features
exclusive to the 2024 edition.

prtest:full

* Reformat with the 2024 edition

show more ...

Wasmtime: support a notion of "custom code publisher". (#9778)

* Wasmtime: support a notion of "custom code publisher".

In some `no_std` environments, virtual memory usage is *generally*
prohibited

Wasmtime: support a notion of "custom code publisher". (#9778)

* Wasmtime: support a notion of "custom code publisher".

In some `no_std` environments, virtual memory usage is *generally*
prohibited for performance-predictability reasons, but the MMU
hardware is still in use for permissions (e.g., `W^X`
write-xor-execute). Occasional changes to page mapping permissions are
thus necessary when new modules are loaded dynamically, and are
acceptable in that context. Wasmtime needs a way to support
"publishing" code (making it executable) in such environments.

Rather than try to segment the `signals-based-traps` divide further,
and piece out the code-publishing parts from the heap parts, and
backdoor a path to `mprotect` in an otherwise `no_std` build, in this
PR I have opted to add a trait an impl of which the embedder can
provide to the `Config` to implement custom actions for "code
publish". This otherwise operates properly in a
no-`signals-based-traps` environment, e.g., the module backing memory
itself is regularly allocated rather than mmap'd (but is now aligned
to the degree requested by the trait impl).

* Review feedback.

* Plumb through custom alignment for runtime code generation

* Add a test for custom code memory.

show more ...

Update handling of wasm features and `*.wast` testing (#9158)

* Update handling of wasm features and `*.wast` testing

This commit is an attempt at making it easier to manage the set of
variables we

Update handling of wasm features and `*.wast` testing (#9158)

* Update handling of wasm features and `*.wast` testing

This commit is an attempt at making it easier to manage the set of
variables we have in play with `*.wast` testing and wasm features.
The Cranelift and Winch backends support different sets of wasm features
at this time and historically Cranelift has also had
architecture-specific support for wasm features. This is hoped to help
simplify management of all of this in the future in addition to making
`*.wast` testing more robust. Notable changes here are:

* A `Config` no longer tracks a `WasmFeatures` but instead only tracks
explicitly disabled/enabled features. The final `WasmFeatures` is then
computed given the result of compiler configuration.
* Default-enabled features now start from nothing instead of
`wasmparser`'s defaults to avoid future breakage there.
* Each compiler configuration and/or backend now has a listed set of
"this feature may panic" wasm features. These are all disabled by
default and if explicitly enabled a `Config::validate` error is
returned.
* All `*.wast` tests are now run through both Cranelift and Winch. Tests
are now asserted to either fail or pass depending on configuration and
the whole test will fail if the result doesn't match the expectation.
* Many `gc` proposal tests which are now passing are flagged as passing
now. Failing `winch` tests are now explicitly listed instead of using
a litany of patterns.

* Change panicking features for winch

* Fix builds without a compiler

show more ...

Refactor use of `CodeBuilder` on the CLI (#9125)

* Refactor use of `CodeBuilder` on the CLI

This commit refactors `wasmtime run` and `wasmtime compile` to
unconditionally use `CodeBuilder` internal

Refactor use of `CodeBuilder` on the CLI (#9125)

* Refactor use of `CodeBuilder` on the CLI

This commit refactors `wasmtime run` and `wasmtime compile` to
unconditionally use `CodeBuilder` internally. This will in theory help
out in the future if more debug-related options are added to
`CodeBuilder` for example. This refactoring required some changes to
`CodeBuilder` to be able to support a query about whether the internal
bytes were a component or a module. The text format is now converted to
binary immediately when supplied rather than during the compilation
phase. This in turn required some API changes to make the selection of
supporting the text format a compile-time choice of method rather than a
runtime value.

* Fix compile

* Fix no-cranelift build of CLI

show more ...

Add `anyhow` stuff to our internal `wasmtime` crate prelude (#8804)

* Add `anyhow` stuff to our internal `wasmtime` crate prelude

We use it basically everywhere and it is annoying to have to import

Add `anyhow` stuff to our internal `wasmtime` crate prelude (#8804)

* Add `anyhow` stuff to our internal `wasmtime` crate prelude

We use it basically everywhere and it is annoying to have to import.

I also did an audit of existing `use` statements and removed the now-redundant
ones and replaced one-off imports with usage of the prelude, so that the prelude
is available by default in more places.

* Fix `cargo doc`

show more ...

Add support for `#![no_std]` to the `wasmtime` crate (#8533)

* Always fall back to custom platform for Wasmtime

This commit updates Wasmtime's platform support to no longer require an
opt-in `RUSTF

Add support for `#![no_std]` to the `wasmtime` crate (#8533)

* Always fall back to custom platform for Wasmtime

This commit updates Wasmtime's platform support to no longer require an
opt-in `RUSTFLAGS` `--cfg` flag to be specified. With `no_std` becoming
officially supported this should provide a better onboarding experience
where the fallback custom platform is used. This will cause linker
errors if the symbols aren't implemented and searching/googling should
lead back to our docs/repo (eventually, hopefully).

* Change Wasmtime's TLS state to a single pointer

This commit updates the management of TLS to rely on just a single
pointer rather than a pair of a pointer and a `bool`. Additionally
management of the TLS state is pushed into platform-specific modules to
enable different means of managing it, namely the "custom" platform now
has a C function required to implement TLS state for Wasmtime.

* Delay conversion to `Instant` in atomic intrinsics

The `Duration` type is available in `no_std` but the `Instant` type is
not. The intention is to only support the `threads` proposal if `std` is
active but to assist with this split push the `Duration` further into
Wasmtime to avoid using a type that can't be mentioned in `no_std`.

* Gate more parts of Wasmtime on the `profiling` feature

Move `serde_json` to an optional dependency and gate the guest profiler
entirely on the `profiling` feature.

* Refactor conversion to `anyhow::Error` in `wasmtime-environ`

Have a dedicated trait for consuming `self` in addition to a
`Result`-friendly trait.

* Gate `gimli` in Wasmtime on `addr2line`

Cut down the dependency list if `addr2line` isn't enabled since then
the dependency is not used. While here additionally lift the version
requirement for `addr2line` up to the workspace level.

* Update `bindgen!` to have `no_std`-compatible output

Pull most types from Wasmtime's `__internal` module as the source of
truth.

* Use an `Option` for `gc_store` instead of `OnceCell`

No need for synchronization here when mutability is already available in
the necessary contexts.

* Enable embedder-defined host feature detection

* Add `#![no_std]` support to the `wasmtime` crate

This commit enables compiling the `runtime`, `gc`, and `component-model`
features of the `wasmtime` crate on targets that do not have `std`. This
tags the crate as `#![no_std]` and then updates everything internally to
import from `core` or `alloc` and adapt for the various idioms. This
ended up requiring some relatively extensive changes, but nothing too
too bad in the grand scheme of things.

* Require `std` for the perfmap profiling agent

prtest:full

* Fix build on wasm

* Fix windows build

* Remove unused import

* Fix Windows/Unix build without `std` feature

* Fix some doc links

* Remove unused import

* Fix build of wasi-common in isolation

* Fix no_std build on macos

* Re-fix build

* Fix standalone build of wasmtime-cli-flags

* Resolve a merge conflict

* Review comments

* Remove unused import

show more ...

Use rustdoc's `doc_auto_cfg` feature instead of `doc_cfg` (#8532)

This commit removes all our `#[cfg_attr(..., doc(cfg(...)))]`
annotations throughout Wasmtime and `wasmtime-wasi`. These are all
rep

Use rustdoc's `doc_auto_cfg` feature instead of `doc_cfg` (#8532)

This commit removes all our `#[cfg_attr(..., doc(cfg(...)))]`
annotations throughout Wasmtime and `wasmtime-wasi`. These are all
replaced with `feature(doc_auto_cfg)` which automatically infers the
attribute to show rather than requiring us to duplicate it.
Spot-checking the docs this looks just-as-readable while being much
easier to maintain over time.

show more ...

Add initial support for DWARF Fission (#8055)

* add cloning for String attributes

* use into_owned instead of to_vec to avoid a clone if possible.

* runs, but does not substitute

* show vars from

Add initial support for DWARF Fission (#8055)

* add cloning for String attributes

* use into_owned instead of to_vec to avoid a clone if possible.

* runs, but does not substitute

* show vars from c program, start cleanup

* tiday

* resolve conflicts

* remove WASI folder

* Add module_builder

add dwarf_package to state for cache

* move dwarf loading to module_environ.rs

pass the dwarf as binary as low as module_environ.rs

* pass dwarf package rather than add to debug_info

* tidy option/result nested if

* revert some toml and whitespace.

* add features cranelift,winch to module_builder and compute_artifacts

remove some `use`s

* address some feedback

remove unused 'use's

* address some feedback

remove unused 'use's

* move wat feature condition to cover whole method.

* More feedback

Another try at wat feature move

* Another try at wat feature move

* change gimli exemption version

add typed-arena exemption

* add None for c-api

* move `use` to #cfg

* fix another config build

* revert unwanted code deletion

* move inner function closer to use

* revert extra param to Module::new

* workaround object crate bug.

* add missing parameter

* add missing parameter

* Merge remote-tracking branch 'origin/main' into dwarf-att-string

# Conflicts:
# crates/wasmtime/src/engine.rs
# crates/wasmtime/src/runtime/module.rs
# src/common.rs

* remove moduke

* use common gimli version of 28.1

* remove wasm feature, revert gimli version

* remove use of object for wasm dwarf

* remove NativeFile workaround, add feature for dwp loading

* sync winch signature

* revert bench api change

* add dwarf for no cache feature

* put back merge loss of module kind

* remove param from docs

* add dwarf fission lldb test

* simplify and include test source

* clang-format

* address feedback, remove packages

add docs
simplify return type

* remove Default use on ModuleTypesBuilder

* Remove an `unwrap()` and use `if let` instead

* Use `&[u8]` instead of `&Vec<u8>`

* Remove an `unwrap()` and return `None` instead

* Clean up some code in `transform_dwarf`

* Clean up some code in `replace_unit_from_split_dwarf`

* Clean up some code in `split_unit`

* Minor refactorings and documentation in `CodeBuilder`

* Restrict visibility of `dwarf_package_binary`

* Revert supply-chain folder changes

* Fix compile error on nightly

* prtest:full

* prtest:full

* prtest:full

* prtest:full

* prtest:full

* prtest:full

* prtest:full

* prtest:full

* use lldb 15

* prtest:full

* prtest:full

* load dwp when loading wasm bytes with path

* correct source file name

* remove debug

---------

Co-authored-by: Alex Crichton <[email protected]>

show more ...

Fix rustdoc warnings on Nightly (#8258)

* Fix rustdoc warnings on Nightly

I noticed during a failed doc build of another PR we've got a number of
warnings being emitted, so resolve all those here.

Fix rustdoc warnings on Nightly (#8258)

* Fix rustdoc warnings on Nightly

I noticed during a failed doc build of another PR we've got a number of
warnings being emitted, so resolve all those here.

* Fix more warnings

* Fix rebase conflicts

show more ...

Add a `compile` feature to `wasmtime-environ` (#8250)

* Add a `compile` feature to `wasmtime-environ`

This commit adds a compile-time feature to remove some dependencies of
the `wasmtime-environ` c

Add a `compile` feature to `wasmtime-environ` (#8250)

* Add a `compile` feature to `wasmtime-environ`

This commit adds a compile-time feature to remove some dependencies of
the `wasmtime-environ` crate. This compiles out support for compiling
modules/components and makes the crate slimmer in terms of amount of
code compiled along with its dependencies. Much of this should already
have been statically removed by native linkers so this likely won't have
any compile-size impact, but it's a nice-to-have in terms of
organization.

This has a fair bit of shuffling around of code, but apart from
renamings and movement there are no major changes here.

* Fix compile issue

* Gate `ModuleTranslation` and its methods on `compile`

* Fix doc link

* Fix doc link

show more ...

Fix compilation cache for components (#8251)

This commit fixes a mistake in #8181 which meant that the caching for
components was no longer working. The mistake is fixed in this commit as
well as a

Fix compilation cache for components (#8251)

This commit fixes a mistake in #8181 which meant that the caching for
components was no longer working. The mistake is fixed in this commit as
well as a new test being added too.

show more ...

Add a `CodeBuilder` type to the `wasmtime` crate (#8181)

* Add a `ModuleBuilder` type to the `wasmtime` crate

This commit is extracted from #8055 and adds a new `ModuleBuilder` type
which is intend

Add a `CodeBuilder` type to the `wasmtime` crate (#8181)

* Add a `ModuleBuilder` type to the `wasmtime` crate

This commit is extracted from #8055 and adds a new `ModuleBuilder` type
which is intended to be able to further configure compilation beyond
what the constructors of `Module` already provide. For example in #8055
knobs will be added to process `*.dwp` files for more debuginfo
processing.

Co-authored-by: yowl00 <[email protected]>

* Fix build

* Review feedback

* Rename ModuleBuilder to CodeBuilder

* Fix doc errors

---------

Co-authored-by: yowl00 <[email protected]>

show more ...