cranelift-codegen: rename most uses of std to core and alloc (#12237)

* rename most std uses to core and alloc

* cargo fmt

Remove `expect(clippy::allow_attributes_without_reason)` from cranelift-codegen (#11182)

* Remove `expect(clippy::allow_attributes_without_reason)` from cranelift-codegen

This commit gets around to

Remove `expect(clippy::allow_attributes_without_reason)` from cranelift-codegen (#11182)

* Remove `expect(clippy::allow_attributes_without_reason)` from cranelift-codegen

This commit gets around to migrating the `cranelift-codegen` crate to
require a reason on lint directives and additionally switch to
`#[expect]` where possible.

prtest:full

* Move x64-only item to x64 backend

show more ...

Cranelift: Revive division-by-constant magic sequences (#11129)

* Revive the old module for generation division-by-constant magic numbers

* Add proptests for divide-by-constant magic number generat

Cranelift: Revive division-by-constant magic sequences (#11129)

* Revive the old module for generation division-by-constant magic numbers

* Add proptests for divide-by-constant magic number generators

* Use division-by-constant magic sequences in the ISLE mid-end

The application of these sequences have been ported to ISLE now to better
integrate with our existing mid-end optimizations and to allow for recursive
rule application in the optimized right-hand sides.

Fixes https://github.com/bytecodealliance/wasmtime/issues/6049

* Fix transcription to ISLE and failing tests

* address review feedback

* Fix ci test failure

show more ...

Adjust lint warning reason (#10910)

After digging into this more I was confused by compiler diagnostics (or
lack thereof). This is indeed dead code but it's required due to how we
use macros/generat

Adjust lint warning reason (#10910)

After digging into this more I was confused by compiler diagnostics (or
lack thereof). This is indeed dead code but it's required due to how we
use macros/generated code/etc, so just suppress the warning for now.

show more ...

Update pinned nightly used in CI (#10893)

* Update pinned nightly used in CI

Resolves some various warnings cropping up from rustc/clippy. This does
trigger what for us is a false positive (I belie

Update pinned nightly used in CI (#10893)

* Update pinned nightly used in CI

Resolves some various warnings cropping up from rustc/clippy. This does
trigger what for us is a false positive (I believe) dead code warning
which I've annotated as `#[allow]` for now (alas this can't be
`#[expect]` because it's a nighly-only warning).

* Remove dead methods when component-model is disabled

show more ...

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtim

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtime to the 2024 Edition to keep
up-to-date with Rust idioms and access many of the edition features
exclusive to the 2024 edition.

prtest:full

* Reformat with the 2024 edition

show more ...

Refactor call ABI implementation (#10722)

This refactors implementation of call ABI handling across architectures
with the goal of bringing s390x in line with other platforms.

The main idea is to
-

Refactor call ABI implementation (#10722)

This refactors implementation of call ABI handling across architectures
with the goal of bringing s390x in line with other platforms.

The main idea is to
- handle main call instruction selection and generation in ISLE
(like s390x but unlike other platforms today)
- handle argument setup mostly outside of ISLE
(like other platforms but unlike s390x today)
- handle return value processing as part of the call instructio
(like all platforms today)

All platforms now emit the main call instruction directly from ISLE,
which e.g. handles selection of the correct ISA instruction depending
on the call destination. This ISLE code calls out to helper routines
to handle argument and return value processing. These helpers are
mostly common code and provided by the Callee and/or Lower layers,
with some platform-specific additions via ISLE Context routines.

The old CallSite abstraction is no longer needed; most of the
differences between call and return_call handling disappear.
(There is still a common-code CallInfo vs. a platform-specifc
ReturnCallInfo. At this point, it should be relatively straight-
forward to make CallInfo platform-specific as well if desired,
but this is not done here.)

Some ISLE infrastructure for iterators / loops, which was only
ever used by the s390x argument processing code, has been removed.

s390x now closely matches all other platforms, with only a few
special cases (slightly different tail-call ABI requires some
differences in stack offset computations; we still need to
handle vector lane swaps for cross-ABI calls), which should
simplify future maintenance.

show more ...

Cranelift: simplify some side-effectful instructions in ISLE (#10524)

* Cranelift: simplify some side-effectful instructions in ISLE

This commit adds a new top-level ISLE entrypoint specifically fo

Cranelift: simplify some side-effectful instructions in ISLE (#10524)

* Cranelift: simplify some side-effectful instructions in ISLE

This commit adds a new top-level ISLE entrypoint specifically for instructions
in the side-effectful skeleton: `simplify_skeleton`. While these rewrites are
processed during the egraph pass, values from skeleton instructions still do not
get inserted into the egraph. Indeed, `simplify_skeleton` operates
on *instructions* rather than *values* because we do not represent side effects
as values; values do not have side effects in CLIF, instructions do. Therefore,
rather than doing a whole dynamic-programming style extraction of the best
candidate simplification like we do with the egraph, we take an eager and greedy
approach.

Furthermore, `simplify_skeleton` is limited only to skeleton instructions that
do not involve control-flow or terminators right now. This is because changing
the control-flow graph can change whether a use is dominated by a def or not,
and we do not currently have the machinery to track and fix up invalidated
uses. Addressing this is left for future commits.

* fix `MIN / -1` cprop and add negative tests for things simplify_skeleton cannot handle yet

show more ...

Simplify aegraphs by removing union-find and canonical eclass IDs. (#10471)

I was recently re-thinking through some of the core data structure
design in our aegraph implementation, and wondered: do

Simplify aegraphs by removing union-find and canonical eclass IDs. (#10471)

I was recently re-thinking through some of the core data structure
design in our aegraph implementation, and wondered: do we really need
the union-find data structure, the notion of "canonical" ID for an
eclass separate from its latest ID (root of union-node tree), and the
hashcons-key canonicalization using all of this? It's an awful lot of
complexity and has led to some fairly subtle bugs (e.g., #6126), and is
generally unsatisfying.

I had the realization: the only case where the distinction between
canonical and latest ID matters is when we expand an eclass after its
initial (eager) rewriting, which happens before we see its uses. If we
hypothesize that this happens rarely, then it should be fine to
canonicalize based only on latest ID -- we shouldn't lose much (and we
can measure this loss empirically).

The chief case where this kind of "late eclass expansion" still happens
is: if we have some expression E1 that eventually rewrites to E2 via
some simplification, and E2 already exists earlier in the program, then
E1 will join the eclass. If we then have some `E3 := E1 + 1`, and later
`E4 := E2 + 1`, we need the union-find canonicalization for E4 to GVN to
E3. Otherwise, the latest ID for the eclass that eventually contains E1
and E2 is different at the time that E3 uses it (and is GVN'd and
rewritten) and when E4 does. Put another way: E3 captures a snapshot of
its operand's eclass before a new node joins it, and is never
reprocessed when that happens, so E3 remains distinct.

But if the `E2 -> E1` rewrite is truly "directional" toward a better
representation that we will always want to choose -- say, `x + 0 -> x`,
or any constant-propagation in general -- then if the eager rewriting
for E2 produces E1's eclass ID directly *without* adding E2 to its
nodes, then all users will still canonicalize as before. This "only
return the rewrite target, don't union with it" before is exactly our
`subsume` operator.

Put another way: subsumption prevents growing eclasses later, so
snapshots in time remain the latest, and everyone canonicalizes with the
same ID. We move to a true immutable data structure, with simple
hashconsing with no magic.

The rewrite semantics are then much simpler too: if any value is
marked "subsume", we pick it (pick an arbitrary one if multiple) as our
rewrite result; otherwise, create an eclass with the original rewrite
input and all rewrite outputs (by creating union-nodes in the DFG). No
auto-subsume or factoring in availability -- that's it. "Subsume" means:
always pick the rewritten value, don't keep the original, and we use it
whenever that's unambiguously true for better canonicalization.

Given that, it turns out that we can remove the union-find mechanism
entirely. It also turns out that we can unify the scoped-hashmap for
"effectful/idempotent ops" with the ordinary hashmap for pure ops;
everything can be scoped. To get working LICM we do need to retain our
"available block" mechanism, but only to insert values at a higher scope
level in the scoped hashmap -- it is now heuristic, not load-bearing for
correctness.

I suspect that the fixpoint loop computing analysis results can go away
too, now that we truly don't update arguments -- we have a simple
immutable data structure with everything snapshotted at creation -- but
I haven't made that change yet.

This change appears to be "in the noise" in both runtime and compile
time -- a Sightglass run on the default suite shows only

```
compilation :: cycles :: benchmarks/pulldown-cmark/benchmark.wasm

Δ = 551234.50 ± 514580.62 (confidence = 99%)

new.so is 1.00x to 1.01x faster than old.so!

[61669181 72513567.85 98139932] new.so
[60991071 73064802.35 120044089] old.so

execution :: cycles :: benchmarks/bz2/benchmark.wasm

Δ = 232827.80 ± 204621.12 (confidence = 99%)

old.so is 1.00x to 1.01x faster than new.so!

[67208140 72812782.32 89996076] new.so
[69531172 72579954.52 80530142] old.so
```

which seem like suitably small swings that are fine. Spot-checking the
aegraph stats on the same function before-and-after shows the same
optimizations happening in all functions I examined, and we see the
compile-tests showing no movement except for a value renumbering in one
case. So: no effect objectively, but deletes code and significantly
simplifies the core algorithm.

show more ...

Enforce `clippy::clone_on_copy` for the workspace (#9025)

* Derive `Copy` for `Val`

* Fix `clippy::clone_on_copy` for the whole repo

* Enforce `clippy::clone_on_copy` for the workspace

* fix some

Enforce `clippy::clone_on_copy` for the workspace (#9025)

* Derive `Copy` for `Val`

* Fix `clippy::clone_on_copy` for the whole repo

* Enforce `clippy::clone_on_copy` for the workspace

* fix some more clippy::clone_on_copy that got missed

show more ...

cranelift: Add `f16const` and `f128const` instructions (#8893)

* cranelift: Add `f16const` and `f128const` instructions

* cranelift: Add constant propagation for `f16` and `f128`

egraph: Resolve all aliases at once (#8240)

* egraph: Resolve all aliases at once

This way we can use the linear-time alias rewriting pass, and then avoid
having to think about value aliases ever a

egraph: Resolve all aliases at once (#8240)

* egraph: Resolve all aliases at once

This way we can use the linear-time alias rewriting pass, and then avoid
having to think about value aliases ever again.

* Resolve aliases in facts and values_labels

When resolving aliases in values_labels, this discards debug info on
values which are replaced by aliases. However, that is equivalent to the
existing behavior in `Lower::get_value_labels`, which resolves value
aliases first and only then looks for attached debug info.

show more ...

cranelift: Constant propagate `bswap` (#8220)

cranelift: Remove support for WebAssembly tables (#8124)

Wasmtime no longer needs any of this infrastructure and neither should
anybody else.

This diff is nearly identical to @bjorn3's version of t

cranelift: Remove support for WebAssembly tables (#8124)

Wasmtime no longer needs any of this infrastructure and neither should
anybody else.

This diff is nearly identical to @bjorn3's version of the same change,
except I didn't remove Uimm64, which has started being used in other
places. I forgot bjorn3 had already tackled this part until after I was
already done, but it's reassuring that we both made the same changes.

https://github.com/bjorn3/wasmtime/commit/fb82ccb3948e949641a6d9581aa84472f68f97b8

Fixes #5532

show more ...

Update some CI dependencies (#7983)

* Update some CI dependencies

* Update to the latest nightly toolchain
* Update mdbook
* Update QEMU for cross-compiled testing
* Update `cargo nextest` for usag

Update some CI dependencies (#7983)

* Update some CI dependencies

* Update to the latest nightly toolchain
* Update mdbook
* Update QEMU for cross-compiled testing
* Update `cargo nextest` for usage with MIRI

prtest:full

* Remove lots of unnecessary imports

* Downgrade qemu as 8.2.1 seems to segfault

* Remove more imports

* Remove unused winch trait method

* Fix warnings about unused trait methods

* More unused imports

* More unused imports

show more ...

Constant propagate int-to-float conversions (#7915)

This commit is born out of a fuzz bug on x64 that was discovered recently.
Today, on `main`, and in the 17.0.1 release Wasmtime will panic when co

Constant propagate int-to-float conversions (#7915)

This commit is born out of a fuzz bug on x64 that was discovered recently.
Today, on `main`, and in the 17.0.1 release Wasmtime will panic when compiling
this wasm module for x64:

(module
(func (result v128)
i32.const 0
i32x4.splat
f64x2.convert_low_i32x4_u))

panicking with:

thread '<unnamed>' panicked at /home/alex/.cargo/registry/src/index.crates.io-6f17d22bba15001f/cranelift-codegen-0.104.1/src/machinst/lower.rs:766:21:
should be implemented in ISLE: inst = `v6 = fcvt_from_uint.f64x2 v13 ; v13 = const0`, type = `Some(types::F64X2)`
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Bisections points to the "cause" of this regression as #7859 which
more-or-less means that this has always been an issue and that PR just
happened to expose the issue. What's happening here is that egraph
optimizations are turning the IR into a form that the x64 backend can't
codegen. Namely there's no general purpose lowering of i64x2 being
converted to f64x2. The Wasm frontend never produces this but the
optimizations internally end up producing this.

Notably here the result of this function is constant and what's
happening is that a convert-of-a-splat is happening. In lieu of adding
the full general lowering to x64 (which is perhaps overdue since this is
the second or third time this panic has been triggered) I've opted to
add constant propagation optimizations for int-to-float conversions.
These are all based on the Rust `as` operator which has the same
semantics as Cranelift. This is enough to fix the issue here for the
time being.

show more ...

Bump MSRV to 1.73.0, use 1.75.0 in CI (#7739)

* Bump MSRV to 1.73.0, use 1.75.0 in CI

Pulling in the Rust update released over the winter holidays.

* Fix more warnings

Add `sextend_maybe` and `uextend_maybe` to opt ISLE (#7710)

* Add a test demonstrating a missed optimization

* Add `[su]extend_maybe` to opt ISLE

Cranelift: Add iconst shorthand to simplify ISLE opts (#7670)

* Demote `simm32` and `uimm8` to lowering ISLE only

There seems to be nothing in opt ISLE that actually wanted them, just something tha

Cranelift: Add iconst shorthand to simplify ISLE opts (#7670)

* Demote `simm32` and `uimm8` to lowering ISLE only

There seems to be nothing in opt ISLE that actually wanted them, just something that's more consistently done with using a 64-bit type to read from an Imm64.

And `simm32` feels like it's probably wrong to me -- `simm32` can't actually match `-1_i32` -- but I'm not confident enough in my analysis to actually change it.

* Cranelift: Add iconst shorthand to simplify ISLE opts

* Do a manually un-currying to avoid duplicating loading the `InstructionData`

* rustfmt is my nemesis

show more ...

ISLE: Build in maximum limit for multi constructor returns (#7526)

Allows users to avoid runaway rules that match too much stuff.

No statistically significant speed up to compilation on sightglass,

ISLE: Build in maximum limit for multi constructor returns (#7526)

Allows users to avoid runaway rules that match too much stuff.

No statistically significant speed up to compilation on sightglass, but good to
have as a safeguard anyways.

Fixes #7500

show more ...

ISLE: Allow callers to pass in storage for multi returns (#7503)

* ISLE: Allow callers to pass in storage for multi returns

This allows callers to reuse storage and avoid allocations.

Internal con

ISLE: Allow callers to pass in storage for multi returns (#7503)

* ISLE: Allow callers to pass in storage for multi returns

This allows callers to reuse storage and avoid allocations.

Internal constructors calling internal constructors won't reuse storage (at
least for now) but it is an option for the user code that is calling into ISLE.

Part of #7500

* Cranelift: Reuse storage for optimized values coming out of ISLE in the midend

show more ...

Cranelift: Clean up imports to avoid warnings (#7508)

Depending on which backends were enabled or not at compile time we might or
might not use some of these imports, which results in warnings. Move

Cranelift: Clean up imports to avoid warnings (#7508)

Depending on which backends were enabled or not at compile time we might or
might not use some of these imports, which results in warnings. Moved imports
and re-exports around a little to avoid these warnings.

No functional changes.

show more ...

Add egraph cprop optimizations for `splat` (#6148)

This commit adds constant-propagation optimizations for
`splat`-of-constant to produce a `vconst` node. This should help later
hoisting these const

Add egraph cprop optimizations for `splat` (#6148)

This commit adds constant-propagation optimizations for
`splat`-of-constant to produce a `vconst` node. This should help later
hoisting these constants out of loops if it shows up in wasm.

show more ...

cranelift: Rework block instructions to use BlockCall (#5464)

Add a new type BlockCall that represents the pair of a block name with arguments to be passed to it. (The mnemonic here is that it looks

cranelift: Rework block instructions to use BlockCall (#5464)

Add a new type BlockCall that represents the pair of a block name with arguments to be passed to it. (The mnemonic here is that it looks a bit like a function call.) Rework the implementation of jump, brz, and brnz to use BlockCall instead of storing the block arguments as varargs in the instruction's ValueList.

To ensure that we're processing block arguments from BlockCall values in instructions, three new functions have been introduced on DataFlowGraph that both sets of arguments:

inst_values - returns an iterator that traverses values in the instruction and block arguments
map_inst_values - applies a function to each value in the instruction and block arguments
overwrite_inst_values - overwrite all values in an instruction and block arguments with values from the iterator

Co-authored-by: Jamey Sharp <[email protected]>

show more ...

Make DataFlowGraph::insts public, but restricted (#5450)

We have some operations defined on DataFlowGraph purely to work around borrow-checker issues with InstructionData and other data on DataFlowG

Make DataFlowGraph::insts public, but restricted (#5450)

We have some operations defined on DataFlowGraph purely to work around borrow-checker issues with InstructionData and other data on DataFlowGraph. Part of the problem is that indexing the DFG directly hides the fact that we're only indexing the insts field of the DFG.

This PR makes the insts field of the DFG public, but wraps it in a newtype that only allows indexing. This means that the borrow checker is better able to tell when operations on memory held by the DFG won't conflict, which comes up frequently when mutating ValueLists held by InstructionData.

show more ...