only remove `_initialize` function from module snapshots (#12540)

* only remove `_initialize` function from module snapshots

Removing `_initialize` from snapshots of wit-component-produced componen

only remove `_initialize` function from module snapshots (#12540)

* only remove `_initialize` function from module snapshots

Removing `_initialize` from snapshots of wit-component-produced components leads
to invalid inputs because it breaks the alias `wit-component` creates.
Moreover, removing that function is redundant because it is only called from a
start function which is also removed during snapshotting.

Ideally, removing `_initialize` and any aliases which point to it would be taken
care of a general-purpose component dead code removal tool, but no such tool
exists yet. In the meantime, leaving it there creates a bit of code bloat but
is otherwise harmless.

* add wizer test for `_initialize` in components

show more ...

wasmtime-wizer: make rayon optional (#12313)

* wasmtime-wizer: make rayon optional

when compiling wasmtime-wizer to wasm32, rayon detects that and tries to
use wasm-bindgen APIs

the new rayon feat

wasmtime-wizer: make rayon optional (#12313)

* wasmtime-wizer: make rayon optional

when compiling wasmtime-wizer to wasm32, rayon detects that and tries to
use wasm-bindgen APIs

the new rayon feature flag allows users to choose if snapshotting should
be parallel or sequential

* add the new wasmtime-wizer rayon feature flag to CI micro_checks

show more ...

wasmtime-wizer: add type_hints to global_get (#12314)

InstanceState::global_get now takes a parameter to tell the system what
type this global expects

in JS, WebAssembly.Instance always returns num

wasmtime-wizer: add type_hints to global_get (#12314)

InstanceState::global_get now takes a parameter to tell the system what
type this global expects

in JS, WebAssembly.Instance always returns number which has to be
converted for snapshotting

show more ...

Migrate Wizer to `wasmtime::error` (#12266)

Update wasm-tools/wit-bindgen dependencies (#12121)

This pulls in wasm-tools support for custom-descriptors which required
adding more match arms in a few locations. Wasmtime doesn't support
this, h

Update wasm-tools/wit-bindgen dependencies (#12121)

This pulls in wasm-tools support for custom-descriptors which required
adding more match arms in a few locations. Wasmtime doesn't support
this, however, so it should all be rejected during validation.

show more ...

wizer: Don't copy all memory out of a store (#11891)

Instead of copying all memory out of a store only copy the contents of
the data segments that are created. This refactors the snapshotting
proces

wizer: Don't copy all memory out of a store (#11891)

Instead of copying all memory out of a store only copy the contents of
the data segments that are created. This refactors the snapshotting
process to work with data segments purely with ranges first and then
copy out the data at the end once all the ranges have settled. This
additionally, while refactoring, adds 64-bit memory support to
snapshotting by ensuring that data segments have the right type of
offset annotated.

show more ...

Merge component-init into wasmtime-wizer (#11878)

* Merge component-init into wasmtime-wizer

This commit is the next phase of merging the wizer and component-init
repositories into Wasmtime. This d

Merge component-init into wasmtime-wizer (#11878)

* Merge component-init into wasmtime-wizer

This commit is the next phase of merging the wizer and component-init
repositories into Wasmtime. This does not take the same approach as
merging wizer where the git histories were merged, but instead this
takes an entirely different approach for component-init. Effectively I
read the code in component-init and copied over the spirit of the code
into the wasmtime-wizer crate. Very little was literally copied over due
to such large changes in the internals of organization and
implementation. The main goal of merging this repository is to replace
the core wasm tracking of state in component-init with what
wasmtime-wizer already has. Sort of like the runtime in the `wasmtime`
crate the goal is to build component support entirely on top of core
support to avoid duplicating anything.

The general strategy for pre-init components is that unlike core wasm
where more exports are added a component has a new module and new
accessor functions injected for all state found in the component. For
example an `i32` global results in a `(func (result s32))` in WIT.
Memories result in `(func (result (list u8)))`. This "accessor module"
is synthetically built during the instrumentation pass of Wizer and then
used to acquire snapshot results afterwards.

All of this support is added in a new `component` submodule of the
`wasmtime-wizer` crate. This submodule has the same structure as its
core wasm counterpart at the root of the crate, but the internal
implementations are entirely different. Anything encountering a core
wasm module delegates to the core wasm support in the root of the crate
for Wizer.

There is one new limitation at this time over what component-init
supports which is that nested components are not supported just yet.
Currently in component-init nested components are copied over as-is
which ends up producing a faulty initialization if the components
actually have core wasm instances associated with them. For now this
implementation sidesteps this by forbidding nested components entirely.
To support wizening the output of `wasm-tools component {new,link}`,
however, some support for nested components will be required. I plan on
adding that in a follow-up commit.

Testing of component-init is pretty light right now so this commit
copies over a few `*.wat` tests "in spirit" but does not literally copy
over the preexisting tests. There are a few tests in component-init
which initialize the real output of `wasm-tools component new` which may
want to be migrated eventually but my hope is that this repo can stick
to smaller more focused tests for now.

One large-ish change made to `wasmtime-wizer` during this merge was to
change snapshotting functionality to being an `async` function. This is
required for components because reading state requires invoking a
function, which in the context of the `wasmtime run` subcommand is
always done in "async mode". This meant that the `async` propagates
outwards to much of `wasmtime-wizer`, even the core wasm traits. My hope
is that this isn't a big issue as the CLI can deal with it and embedders
can throw an `async` in there.

Overall this is intended to be a mostly-complete skeleton plus basic
functionality for component-init. I have not done thorough testing with
real-world components just yet (e.g. componentize-py) so there will
likely be follow-up PRs to address various inevitable shortcomings I've
introduced in this merge.

* Deduplicate some debug asserts

* Fix wizer benches/fuzzing

* Use `wizer-initialize` for core wasm and components

* Only copy out nonzero contents of memory, not all memory

* Fix a rebase conflict

* Revert "Only copy out nonzero contents of memory, not all memory"

This reverts commit 865146cc459826945cd22f4a41f1137d3e8c1c46.

show more ...

wizer: Disallow GC mutations (#11877)

Explicitly disallow mutation-related GC instructions in Wizer. This
avoids forcibly disabling the wasm GC proposal in Wizer since that has
type system effects a

wizer: Disallow GC mutations (#11877)

Explicitly disallow mutation-related GC instructions in Wizer. This
avoids forcibly disabling the wasm GC proposal in Wizer since that has
type system effects as well. Instead this disallows the subset of the
proposal which is incompatible with Wizer, specifically mutating GC
references which can't be snapshot.

show more ...

wizer: Reject mutable reference-typed globals (#11866)

These can't currently be snapshotted, so they're rejected. This involved
building out support for readonly reference-typed globals where the
sn

wizer: Reject mutable reference-typed globals (#11866)

These can't currently be snapshotted, so they're rejected. This involved
building out support for readonly reference-typed globals where the
snapshot now only contains mutable globals, not all globals.

show more ...

wizer: Make `wasmtime` an optional dependency (#11863)

* wizer: Make `wasmtime` an optional dependency

This commit refactors Wizer to have `wasmtime` as an optional dependency
of the crate rather t

wizer: Make `wasmtime` an optional dependency (#11863)

* wizer: Make `wasmtime` an optional dependency

This commit refactors Wizer to have `wasmtime` as an optional dependency
of the crate rather than being a hard dependency. This, in theory,
enables Wizer to be used with other runtimes (such as a JS-based
runtime), but otherwise for now provides a cleaner separation between
Wizer and Wasmtime and clearly indicates the interface between the two.

* Fix standalone wizer build

* Fix feature'd compile

* Fix compile of the `wizer` subcommand

show more ...

wizer: Refactor snapshotting of globals/memories (#11857)

This commit updates the loop-over-globals approach of snapshotting to
instead use a list of names produced from the instrumentation pass. Th

wizer: Refactor snapshotting of globals/memories (#11857)

This commit updates the loop-over-globals approach of snapshotting to
instead use a list of names produced from the instrumentation pass. This
removes the hardcoding of the `__wizer_*` names in the snapshotting pass
and instead centralizes the naming in one location.

show more ...

Remove more module linking support from Wizer (#11855)

This commit simplifies some stages of Wizer by removing more support for
the now-ancient "module linking" proposal which was the precursor to t

Remove more module linking support from Wizer (#11855)

This commit simplifies some stages of Wizer by removing more support for
the now-ancient "module linking" proposal which was the precursor to the
component model. Much of this will come back, but in a very different
form, for component model support as component-init is merged into this
repository. For now though to prepare for this refactoring this commit
removes all this historical support to keep core module wizening focused
on just core modules.

show more ...

wizer: Use utilties from wasm-encoder for conversion (#11853)

Simplify some code here and there by using the translations provided by
the `wasm-encoder` crate to avoid needing to duplicate them insi

wizer: Use utilties from wasm-encoder for conversion (#11853)

Simplify some code here and there by using the translations provided by
the `wasm-encoder` crate to avoid needing to duplicate them inside of
Wizer.

show more ...

Disallow fewer instructions in Wizer (#11851)

Specifically allow `table.get`, `ref.null`, `ref.is_null`, `select`
(typed), `ref.func`, and `table.size`. These are fine to execute and the
incompatibi

Disallow fewer instructions in Wizer (#11851)

Specifically allow `table.get`, `ref.null`, `ref.is_null`, `select`
(typed), `ref.func`, and `table.size`. These are fine to execute and the
incompatibility with Wizer instead comes when a table or a reference
type global is mutated. That mutation is disallowed elsewhere, however,
so there's no need to reject these instructions.

show more ...

Integrate wizer into this repository (#11805)

* Remove misc wizer-related files

* Integrate the Wizer manifest with this repo's workspace

* Enable some more wasmtime features

* Get wizer tests pa

Integrate wizer into this repository (#11805)

* Remove misc wizer-related files

* Integrate the Wizer manifest with this repo's workspace

* Enable some more wasmtime features

* Get wizer tests passing in-repo

* Remove duplicate dummy wizer module

* Integer `wasmtime wizer` subcommand into the CLI

* Fully integrate wizer into `wasmtime` CLI

* Split `wasmtime run` into helper functions
* Split `Wizer::run` into helper functions
* Weave the two together in `wasmtime wizer`

The end goal is to have all CLI options in `wasmtime run` applicable for
`wasmtime wizer` as well with some light edits between the two. Overall
though we shouldn't have to proactively support commands in one or the
other and everything ideally should "just work".

* Fix clippy warnings and bench compiles

* Fix benchmarks

* Create a store-per-iteration
* Use the right wasms in the regex benchmark

* Get wizer fuzzer building again

* Get CLI working again

* Run rustfmt

* Remove precompiled wasms from the tree

35M for some wasms is a bit heavy so instead build them from source.

* Update vet configuration for fuzzers/tests

* Update publish script with wasmtime-wizer

* Fix clippy lint

* Some docs and more clippy lints

prtest:full

* Relax version requirement

* Try to fix asan build

* Remove rustflags too

* Un-exclude wizer

* Adjust publish script

* Update lock file after rebase

* Integrate bytecodealliance/wizer#139

Use deterministic results for relaxed simd operations by default.

* Handle preloads in wizer

* Appease clippy

* Use deterministic relaxed simd in wizer tests

show more ...

chore(deps): update wasmtime & upstream deps (#136)

* chore(deps): update wasmtime & upstream deps

This commit updates wasmtime along with other deps:

- wasmtime: v31 -> v36.0.2
- wasmparser et al

chore(deps): update wasmtime & upstream deps (#136)

* chore(deps): update wasmtime & upstream deps

This commit updates wasmtime along with other deps:

- wasmtime: v31 -> v36.0.2
- wasmparser et al: *.228 -> *.238

* fix(deps): use major version of wasmtime in dep

Co-authored-by: Till Schneidereit <[email protected]>

---------

Co-authored-by: Till Schneidereit <[email protected]>

show more ...

chore: bump deps (#132)

enable reference types and bulk memory support by default (#129)

* enable reference types and bulk memory support by default

* update tests

---------

Co-authored-by: Gentle <[email protected]>

add option for reference-types (#124)

* add option for reference_types

only enables the wasmtime feature, which allows initializing Modules
that were compiled with newer llvm versions but don't act

add option for reference-types (#124)

* add option for reference_types

only enables the wasmtime feature, which allows initializing Modules
that were compiled with newer llvm versions but don't actually use
reference_types

Disabled by default

* Update src/lib.rs

Co-authored-by: Nick Fitzgerald <[email protected]>

* reject reference-types related instructions

* add setter for wasm_reference_types

* bail instead of unreachable

* tests for reference-types still rejecting tyble modifying instructions

* better error messages

* test indirect call with reference_types enabled

* same docstring for library and cli

* (hopefully) actually use the new call_indirect

---------

Co-authored-by: Gentle <[email protected]>
Co-authored-by: Nick Fitzgerald <[email protected]>

show more ...

Update Wasmtime and the WASI implementation to v23 (#109)

* Update wasmtime and the wasi implementation to v23

* Fix benchmark compilation

Update dependencies, wasmtime-19

updates, replace wasi-cap-std-sync with wasi-common

Update wasmtime

Update to Wasmtime 16

Update wasmtime/wasm-tools dependencies

This updates all the various deps to their latest versions.