Fix a panic with a massive `max_wasm_stack` configured (#12869)

* Fix a panic with a massive `max_wasm_stack` configured

This commit fixes a panic through a `checked_add(...).unwrap()` which
can ha

Fix a panic with a massive `max_wasm_stack` configured (#12869)

* Fix a panic with a massive `max_wasm_stack` configured

This commit fixes a panic through a `checked_add(...).unwrap()` which
can happen when `Config::max_wasm_stack` is configured to be a very
large value. This is a mostly benign panic as it's unlikely this is
configured much in the wild, but nevertheless seems like a good issues
to fix regardless.

* Fix an overflow/OOM panic in pulley

prtest:full

* Fix CI

* Another CI fix

* Fix test on 32-bit

* Fix miri test

show more ...

Avoid `copy_from_slice` in the p1 adapter (#12088)

* Avoid `copy_from_slice` in the p1 adapter

This is not fully optimized on nightly where a possible panic point
remains which breaks the build of

Avoid `copy_from_slice` in the p1 adapter (#12088)

* Avoid `copy_from_slice` in the p1 adapter

This is not fully optimized on nightly where a possible panic point
remains which breaks the build of the adapter, so resort to
`ptr::copy_nonverlapping` to avoid this dead panic code.

* Fix macos

* More macos fixes

prtest:full

show more ...

Manually implement libcalls in cranelift-filetests (#11824)

* Manually implement libcalls in cranelift-filetests

Instead of relying on the system `libm` to provide the implementation of
these libca

Manually implement libcalls in cranelift-filetests (#11824)

* Manually implement libcalls in cranelift-filetests

Instead of relying on the system `libm` to provide the implementation of
these libcalls through `dlopen` this commit instead changes
cranelift-filetests to use handwritten implementations of various
libcalls. This has the primary benefit of ensuring that the same
implementation is used in both the interpreter and compiled code when
libcalls are made. This ensure that during differential fuzzing explicit
invocations of libcalls will produce the same result, even with respect
to NaN bits, since the implementation is the same (whatever the Rust
standard library does).

* Fix mul_add on mingw

* Fix build on Windows

* Really fix the build this time...

show more ...

Clarify docs around exceptions in Cranelift (#11601)

* Clarify docs around exceptions in Cranelift

This is a result of today's Cranelift meeting with some of my questions
around the ABI bits here a

Clarify docs around exceptions in Cranelift (#11601)

* Clarify docs around exceptions in Cranelift

This is a result of today's Cranelift meeting with some of my questions
around the ABI bits here and there. Notably:

* Cranelift is audited and now documented to always consider the callee
calling convention in `try_call`, disregarding the caller calling convention.

* Wasmtime's exception throw now explicitly names the `tailcc` in the
name to indicate that it's only compatible with the tail calling convention.

* The verifier test for exceptions is expanded with a few more cases
here and there that I could think of.

* Backends now assert that the size of the calling-convention list of
payload types is the same as the size of the list of registers the
backend places results into.

* Fix a typo

show more ...

Package up exception handler state (#11577)

* Package up exception handler state

This commit is a minor refactoring of the `wasmtime-unwinder` crate to
use a `Handler` structure as a "package" for

Package up exception handler state (#11577)

* Package up exception handler state

This commit is a minor refactoring of the `wasmtime-unwinder` crate to
use a `Handler` structure as a "package" for the pc/fp/sp triple that's
required to resume to an exception handler. Freestanding functions are now
associated methods/functions of `Handler` such as finding a frame on the
stack and resuming to a handler.

This doesn't actually change any behavior, just moving some things
around to prepare for a future refactoring.

* Fix some CI issues

* Fix conditional build

show more ...

Wasmtime/Cranelift: carry "FP to SP offset" in exception data, and use it in stackwalk. (#11500)

* Wasmtime/Cranelift: carry "FP to SP offset" in exception data, and use it in stackwalk.

Currently

Wasmtime/Cranelift: carry "FP to SP offset" in exception data, and use it in stackwalk. (#11500)

* Wasmtime/Cranelift: carry "FP to SP offset" in exception data, and use it in stackwalk.

Currently Wasmtime unwinds stack frames to look for exception handlers
by walking frames one-by-one, following the FP chain as usual, and
assuming that *these frames are contiguous*: that is, that the SP in
any given frame (bottom of that frame) is immediately above the FP of
the next lower frame, plus the FP/return address pair (e.g. 16 bytes).
This allows us to get the SP for any given frame in addition to FP. We
need SP for two reasons:

- To look up dynamic context, to match Wasm tag instances for handlers
against the thrown tag;
- To actually set SP when we resume, if we do resume to a handler in
this frame.

This logic *almost but not quite* worked: I had forgotten that in our
tail-call ABI, we need to clean up incoming stack args in the
callee (because only the final callee in a parade of tail-calling
functions that reuse the same stack frame location knows how many args
it has, not the original caller). This implies that there is an
"incoming args area" *above* the FP/return address pair. Thus, frames
are not necessarily contiguous by the above definition.

In #11489 we see a case where a function of signature `(func)`
tail-calls one of `(func (param i32 i32 i32 i32 i32))`, which on
x86-64 (with four arg registers left for Wasm) is sufficient to create
incoming stack args, which then trips up the unwinder, reading a bogus
vmctx and segfaulting.

The most reasonable solution seems to be to embed the SP-to-FP offset
in the exception metadata itself, so from only the FP (which is
totally robust -- we rely on the FP chain for multiple kinds of
stack-walking) we can get the SP, allowing us to read dynamic context
and to reset SP during resume.

This PR does just that. Technically, in our ABI, the SP-to-FP offset
is constant for an entire function, but it was simpler in the
exception metadata to encode this per callsite instead (there is no
other notion of "per-function" data, only "per-callsite", so it would
be a separate binary search).

Fixes #11489.

prtest:full

* Review feedback.

show more ...

WebAssembly exception-handling support. (#11326)

* WebAssembly exception-handling support.

This PR introduces support for the [Wasm exception-handling proposal],
which introduces a conventional try

WebAssembly exception-handling support. (#11326)

* WebAssembly exception-handling support.

This PR introduces support for the [Wasm exception-handling proposal],
which introduces a conventional try/catch mechanism to WebAssembly. The
PR supports modules that use `try_table` to register handlers for a
lexical scope; and provides `throw` and `throw_ref` that allocate (in
the first case) and throw exception objects.

This PR builds on top of the work in #10510 for Cranelift-level
exception support, #10919 for an unwinder, and #11230 for exception
objects built on top of GC, in addition a bunch of smaller fix and
enabling PRs around those.

[Wasm exception-handling proposal]: https://github.com/WebAssembly/exception-handling/

prtest:full

* Permit UnwindToWasm to have unused fields in Pulley builds (for now).

* Resolve miri-caught reborrowing issue.

* Ignore exceptions tests in miri for now (Pulley not supported).

* Use wasmtime_test on exceptions tests.

* Get tests passing on pulley platforms

* Add a check to `supports_host` for the generated test and assert
failure also when that is false.
* Remove `pulley_unsupported` test as it falls out of `#[wasmtime_test]`
* Remove `exceptions_store` helper as it falls out of `#[wasmtime_test]`
* Remove miri annotations as they fall out of `#[wasmtime_test]`

* Remove dead import

* Skip some unsupported tests entirely in `#[wasmtime_test]`

If the selected compiler doesn't support the host at all then there's no
need to run it. Actually running it could misinterpret `CraneliftNative`
as "run with pulley" otherwise, so avoid such false negatives.

* Cranelift: dynamic contexts: account for outgoing-args area.

---------

Co-authored-by: Alex Crichton <[email protected]>

show more ...

Bump MSRV to Rust 1.87.0 (#11396)

* Bump MSRV to Rust 1.87.0

Coincides with today's release of Rust 1.89

* Fix some lint warnings

* Fix tests with Pulley

* Fix lldb tests

* Fix a doc test on 32

Bump MSRV to Rust 1.87.0 (#11396)

* Bump MSRV to Rust 1.87.0

Coincides with today's release of Rust 1.89

* Fix some lint warnings

* Fix tests with Pulley

* Fix lldb tests

* Fix a doc test on 32-bit

show more ...

Update nightly used in CI (#10957)

A new lint was added to rustc so this updates the nightly used in CI and
then additionally fixes the lints that are firing.

Cranelift: implement an "unwinder" crate and exception throws in filetests. (#10919)

This commit introduces the next major piece of machinery (after the
previously-landed `try_call` support) that we

Cranelift: implement an "unwinder" crate and exception throws in filetests. (#10919)

This commit introduces the next major piece of machinery (after the
previously-landed `try_call` support) that we will eventually use to
implement Wasm exceptions in Wasmtime. In particular, it implements a
generic unwinder as a new crate that supports (i) walking a stack
produced by Cranelift code, (ii) serializing Cranelift exception
metadata to compact tables (in a way very similar to address maps in
Wasmtime, so they will be mappable directly from disk), (iii) using
these serialized tables to find handlers during a stack-walk, and (iv)
jumping to handlers (i.e., actually unwinding). This crate is currently
used in the filetests runner, and will next be used in Wasmtime.

The commit first performs code-motion: it moves stack-walking code from
Wasmtime to `cranelift-unwinder`. This itself has no functional effect,
but isolates the code that understands contiguous sequences of Cranelift
frames ("activations") from that which is specific to Wasmtime's
activation delimiters and metadata.

It then implements a compact exception-table format. This format uses
the `object` crate's mechanisms for directly referencing in-memory
arrays of little-endian `u32`s in a way that will allow us to find
handlers when mapping exception metadata directly from an ELF section in
a `.cwasm` (for example). The format consists of four sorted `u32`
arrays in a way that allows us to look up a callsite first, then search
its sorted array of handler offsets by tags.

It next implements the actual unwind control flow: it contains an
assembly stub for each supported architecture that transfers control to
a PC, SP, and FP value "up the stack", with payload values placed in the
payload registers we have defined per our exception ABI in Cranelift.

Finally, it puts these pieces together in the filetest runner. Note that
the runtest does a lot "by hand": we don't have entry and exit
trampolines as we do in Wasmtime, so the filetest contains three
functions, with the middle one invoking the "throw hostcall" and entry
and exit trampolines around it grabbing the appropriate entry/exit FPs
and exit PC. The dance to call back to host code is also somewhat
delicate, as we haven't done this before. The `JITModule`'s linking +
relocation support does not seem sufficient to properly define a symbol,
so instead we scan for `func_addr` instructions referencing a well-known
name (`__cranelift_throw`) and replace them with `iconst`s with the
function address at runtime, baking it in. This is somewhat ugly, but it
works. All of these filetest-specific details will be handled much more
nicely in the Wasmtime version of this functionality, as we have proper
abstractions for entry/exit trampolines and hostcalls.

show more ...

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtim

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtime to the 2024 Edition to keep
up-to-date with Rust idioms and access many of the edition features
exclusive to the 2024 edition.

prtest:full

* Reformat with the 2024 edition

show more ...

Update MSRV to 1.85.0 (#10785)

* Update MSRV to 1.85.0

This makes us eligible to update to the 2024 Rust edition but I'm
planning on deferring that to a separate follow-up change.

prtest:full

* F

Update MSRV to 1.85.0 (#10785)

* Update MSRV to 1.85.0

This makes us eligible to update to the 2024 Rust edition but I'm
planning on deferring that to a separate follow-up change.

prtest:full

* Fix unused mut

* Try harder to link `cosf` and thus `libm`

show more ...

Update linkage of `libm` for nightly Rust (#10587)

Upstream rustc has changed meaning that the `ceil` function is now
defined in bcompiler-builtins, so link a different symbol to force
pulling in `l

Update linkage of `libm` for nightly Rust (#10587)

Upstream rustc has changed meaning that the `ceil` function is now
defined in bcompiler-builtins, so link a different symbol to force
pulling in `libm` to get `dlsym` working for its symbols in Cranelift
tests.

show more ...

Enable the `unsafe-op-in-unsafe-fn` lint (#10559)

* Enable the `unsafe-op-in-unsafe-fn` lint

This commit enables the `unsafe-op-in-unsafe-fn` lint in rustc for the
entire workspace. This lint will

Enable the `unsafe-op-in-unsafe-fn` lint (#10559)

* Enable the `unsafe-op-in-unsafe-fn` lint

This commit enables the `unsafe-op-in-unsafe-fn` lint in rustc for the
entire workspace. This lint will be warn-by-default in the 2024 edition
so this is intended to smooth the future migration to the new edition.

Many `unsafe` blocks were added in places the lint warned about, with
two major exceptions. The `wasmtime` and `wasmtime-c-api` crates simply
expect this lint to fire and effectively disable the lint. They're too
big at this time to do through this PR. My hope is that one day in the
future they'll be migrated, but more realistically that probably won't
happen so these crates just won't benefit from this lint.

* Fix nostd fiber build

prtest:full

* Fix build on Windows

* Fix asan build

show more ...

Run the full test suite on 32-bit platforms (#9837)

* Run the full test suite on 32-bit platforms

This commit switches to running the full test suite in its entirety
(`./ci/run-tests.sh`) on 32-bit

Run the full test suite on 32-bit platforms (#9837)

* Run the full test suite on 32-bit platforms

This commit switches to running the full test suite in its entirety
(`./ci/run-tests.sh`) on 32-bit platforms in CI in addition to 64-bit
platforms. This notably adds i686 and armv7 as architectures that are
tested in CI.

Lots of little fixes here and there were applied to a number of tests.
Many tests just don't run on 32-bit platforms or a platform without
Cranelift support, and they've been annotated as such where necessary.
Other tests were adjusted to run on all platforms a few minor bug fixes
are here as well.

prtest:full

* Fix clippy warning

* Get wasm code working by default on 32-bit

Don't require the `pulley` feature opt-in on 32-bit platforms to get
wasm code running.

* Fix dead code warning

* Fix build on armv7

* Fix test assertion on armv7

* Review comments

* Update how tests are skipped

* Change how Pulley is defaulted

Default to pulley in `build.rs` rather than in `Cargo.toml` to make it
easier to write down the condition and comment what's happening. This
means that the `pulley-interpreter` crate and pulley support in
Cranelift is always compiled in now and cannot be removed. This should
hopefully be ok though as the `pulley-interpreter` crate is still
conditionally used (meaning it can get GC'd) and the code-size of
Cranelift is not as important as the runtime itself.

* pulley: Save/restore callee-save state on traps

* Fewer clippy warnings about casts

* Use wrapping_add in `g32_addr`, fixing arm test

show more ...

Enable `missing-unsafe-on-extern` lint (#9963)

* Enable `missing-unsafe-on-extern` lint

This'll be a hard error in the 2024 edition so go ahead and opt-in to it
now to ease our future transition.

Enable `missing-unsafe-on-extern` lint (#9963)

* Enable `missing-unsafe-on-extern` lint

This'll be a hard error in the 2024 edition so go ahead and opt-in to it
now to ease our future transition.

* Fix adapter build

* Fix custom c-api build

* Fix fuzzer build

* Fix some Windows `extern` blocks

show more ...

pulley: Support `runtests` in Cranelift filetests (#9795)

This'll help when adding unit tests for Pulley and/or might be useful
when debugging various lowerings and such in the future. I hope to
ena

pulley: Support `runtests` in Cranelift filetests (#9795)

This'll help when adding unit tests for Pulley and/or might be useful
when debugging various lowerings and such in the future. I hope to
enable more tests in the future once more pulley lowerings are
available.

show more ...

Burn down the `allow_attributes_without_reason` backlog (#9712)

* Burn down the `allow_attributes_without_reason` backlog

Just a bit, not everything fixed.

* Fix wasi-nn annotations

* Tweak `#[cf

Burn down the `allow_attributes_without_reason` backlog (#9712)

* Burn down the `allow_attributes_without_reason` backlog

Just a bit, not everything fixed.

* Fix wasi-nn annotations

* Tweak `#[cfg]`

show more ...

Enforce `uninlined_format_args` for the workspace (#9065)

* Enforce `uninlined_format_args` for the workspace

* fix: failing `Monolith Checks` job

* fix: formatting

Remove some more cranelift-filetest wasm testing bits (#8143)

* Delete now-dead code from the wasm runner in cranelift-filetest

No longer needed with tests having moved out to `tests/disas`

* Move

Remove some more cranelift-filetest wasm testing bits (#8143)

* Delete now-dead code from the wasm runner in cranelift-filetest

No longer needed with tests having moved out to `tests/disas`

* Move wasm<->clif testing to `tests/disas.rs`

No need for `test_wasm.rs` to stick around in cranelift-filetest, so
move the bits up a layer.

* Remove wasm crate dependencies

* Change bless env var name

* Force link to `libm`

show more ...

Fix some warnings on nightly Rust (#6702)

* Fix some warnings on nightly Rust

* One is to avoid using `drop(&mut T)`
* Another is to avoid using `()` in FFI functions since apparently
that's not

Fix some warnings on nightly Rust (#6702)

* Fix some warnings on nightly Rust

* One is to avoid using `drop(&mut T)`
* Another is to avoid using `()` in FFI functions since apparently
that's not recommended. Working around this in the macro was pretty
tricky, however.

* Fix a typo

* Review feedback

show more ...

x64: Add non-SSSE3 lowerings of `pshufb` (#6606)

* x64: Add non-SSSE3 lowerings of `pshufb`

Or, more accurately, add lowerings which don't use `pshufb`'s
functionality at all where possible or othe

x64: Add non-SSSE3 lowerings of `pshufb` (#6606)

* x64: Add non-SSSE3 lowerings of `pshufb`

Or, more accurately, add lowerings which don't use `pshufb`'s
functionality at all where possible or otherwise fall back to a new
libcall. This particular instruction seemed uniquely difficult to
implement in the backend so I decided to "cop out" and use libcall
instead. The libcall will be used for `popcnt`, `shuffle`, and
`swizzle` instructions when SSSE3 isn't available.

* Implemente SSE2 popcnt with Hacker's Delight

* x64: Implement passing vector arguments in the fastcall convention

Windows says that vector arguments are passed indirectly so handle that
here through the `ABIArg::ImplicitPtrArg` variant. Some additional
handling is added to the general machinst backend.

* Update `gen_load_base_offset` for x64

* Fill out remaining bits of fastcall and vector parameters

* Remove now-unnecessary `Clone` bound

show more ...

Rename define_function to define_function_with_control_plane (#6165)

And add a define_function convenience function which uses a default
control plane.

Chaos mode MVP: Skip branch optimization in MachBuffer (#6039)

* fuzz: Add chaos mode control plane

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Wa

Chaos mode MVP: Skip branch optimization in MachBuffer (#6039)

* fuzz: Add chaos mode control plane

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>

* fuzz: Skip branch optimization with chaos mode

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>

* fuzz: Rename chaos engine -> control plane

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>

* chaos mode: refactoring ControlPlane to be passed through the call stack by reference

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Remo Senekowitsch <[email protected]>

* fuzz: annotate chaos todos

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>

* fuzz: cleanup control plane

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>

* fuzz: remove control plane from compiler context

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>

* fuzz: move control plane into emit state

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>

* fuzz: fix remaining compiler errors

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>

* fix tests

* refactor emission state ctrl plane accessors

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>

* centralize conditional compilation of chaos mode

Also cleanup a few straggling dependencies on cranelift-control
that aren't needed anymore.

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>

* add cranelift-control to published crates

prtest:full

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>

* add cranelift-control to public crates

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>

---------

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>
Co-authored-by: Remo Senekowitsch <[email protected]>

show more ...

fuzzgen: Generate multiple functions per testcase (#5765)

* fuzzgen: Generate multiple functions per testcase

* fuzzgen: Fix typo

Co-authored-by: Jamey Sharp <[email protected]>

---------

Co-aut

fuzzgen: Generate multiple functions per testcase (#5765)

* fuzzgen: Generate multiple functions per testcase

* fuzzgen: Fix typo

Co-authored-by: Jamey Sharp <[email protected]>

---------

Co-authored-by: Jamey Sharp <[email protected]>

show more ...