Use `if cfg!(...)` instead of `#[cfg]` (#12889)

Minor style follow-up from #12841

Cranelift: perform aggressive splitting of generated ISLE function bodies in debug builds. (#12841)

As investigated in #12821, ISLE-generated Rust code with many locals
in one large function body re

Cranelift: perform aggressive splitting of generated ISLE function bodies in debug builds. (#12841)

As investigated in #12821, ISLE-generated Rust code with many locals
in one large function body results in unreasonably-large stack frames
when rustc's optimizations (and therefore LLVM's mem2reg) are
disabled. In `constructor_simplify`, a single function body generated
from all mid-end rewrite rules, I was seeing a stack-frame size of
`0x44000` bytes (272 KiB) on x86-64 (while in an opt build it is
`0x1000` bytes (4 KiB)). With a recursion depth of 5 (the default
limit) for rewrites of RHS-created nodes, this has the potential to
overrun a 1 MiB stack; and is generally quite wasteful.

In [another branch] I attempted to make `islec` do manual regalloc of
a sort just for multi-extractor iterators; but soon realized that the
problem has to do with *all* locals (which become `alloca`s in LLVM
IR), not just the iterators. We could do the equivalent thing to share
all locals but this would become grossly un-idiomatic for e.g. LHSes
of destructuring in Rust.

Instead, this PR leverages previous work in #12303 (thanks Bongjun!)
that splits function bodies in the generated Rust code from ISLE with
a configurable threshold. This PR enables that feature and makes the
threshold quite low in debug builds. This has the natural effect of
splitting locals among many smaller stack frames, which are
dynamically pushed only when matching enters a particular subtree; so
stack usage more closely mirrors the actual live-set of values bound
during matching.

With this change, the stack frame for the top-level
`constructor_simplify` is < 4 KiB on x86-64 in a debug build.

Fixes #12821.

[another branch]: https://github.com/cfallin/wasmtime/tree/isle-iterator-reuse

show more ...

Optimize ISLE compilation (#12303)

* optimize isle codegen for rustc

* opt-in for optimizing isle codegen

* have a controller threshold

Add logging support for debugging ISLE rules (#12233)

* Add logging support for debugging ISLE rules

* remove unnecessary feature guard

* use Debug print for escaping slashes

Strip prefixes in file names (#10650)

Allow codegen to remove certain prefixes from source file paths when
printing the paths in generated files. This currently is only applied to
the `OUT_DIR` and

Strip prefixes in file names (#10650)

Allow codegen to remove certain prefixes from source file paths when
printing the paths in generated files. This currently is only applied to
the `OUT_DIR` and allows generated code to be deterministic.

show more ...

Cranelift: tweak the ISLE-source-dir build script functionality. (#10466)

- Add a `cargo:rerun-if-env-changed` clause for the `ISLE_SOURCE_DIR`
environment variable to properly rebuild if the conf

Cranelift: tweak the ISLE-source-dir build script functionality. (#10466)

- Add a `cargo:rerun-if-env-changed` clause for the `ISLE_SOURCE_DIR`
environment variable to properly rebuild if the config changes.
- Update docs to point to the new env-var-based approach rather than the
old Cargo feature.

As per this Zulip conversation [1] and this GitHub comment [2].

[1]: https://bytecodealliance.zulipchat.com/#narrow/channel/217117-cranelift/topic/How.20to.20enable.20the.20isle-in-source-tree.20feature.3F/near/508028830
[2]: https://github.com/bytecodealliance/wasmtime/pull/9633#discussion_r2012230673

show more ...

Run the full test suite on 32-bit platforms (#9837)

* Run the full test suite on 32-bit platforms

This commit switches to running the full test suite in its entirety
(`./ci/run-tests.sh`) on 32-bit

Run the full test suite on 32-bit platforms (#9837)

* Run the full test suite on 32-bit platforms

This commit switches to running the full test suite in its entirety
(`./ci/run-tests.sh`) on 32-bit platforms in CI in addition to 64-bit
platforms. This notably adds i686 and armv7 as architectures that are
tested in CI.

Lots of little fixes here and there were applied to a number of tests.
Many tests just don't run on 32-bit platforms or a platform without
Cranelift support, and they've been annotated as such where necessary.
Other tests were adjusted to run on all platforms a few minor bug fixes
are here as well.

prtest:full

* Fix clippy warning

* Get wasm code working by default on 32-bit

Don't require the `pulley` feature opt-in on 32-bit platforms to get
wasm code running.

* Fix dead code warning

* Fix build on armv7

* Fix test assertion on armv7

* Review comments

* Update how tests are skipped

* Change how Pulley is defaulted

Default to pulley in `build.rs` rather than in `Cargo.toml` to make it
easier to write down the condition and comment what's happening. This
means that the `pulley-interpreter` crate and pulley support in
Cranelift is always compiled in now and cannot be removed. This should
hopefully be ok though as the `pulley-interpreter` crate is still
conditionally used (meaning it can get GC'd) and the code-size of
Cranelift is not as important as the runtime itself.

* pulley: Save/restore callee-save state on traps

* Fewer clippy warnings about casts

* Use wrapping_add in `g32_addr`, fixing arm test

show more ...

Stop formatting ISLE generated code (#9901)

The tool stack overflows in default configurations (Windows/CI).

Cranelift: rework isle-in-source-tree functionality. (#9633)

Per #9625 and #9588, a downstream user of Cranelift was misusing the
`isle-in-source-tree` feature, enabling it indiscriminately even in

Cranelift: rework isle-in-source-tree functionality. (#9633)

Per #9625 and #9588, a downstream user of Cranelift was misusing the
`isle-in-source-tree` feature, enabling it indiscriminately even in
published crates (wasmerio/wasmer#5202). This went against the intent of
the feature, to be a way for local developers to observe the generated
source. As such, it ran afoul of some safety checks for that purpose,
and also polluted users' Cargo caches in a way that we cannot now fix
except in future releases.

The best we can do is probably to take away features that are liable to
be misused. Following discussion in today's Cranelift weekly meeting, we
decided to provide this functionality under an environment variable
instead. This allows folks who know what they're doing to get the same
behavior, but does not expose a feature to crate users.

Fixes #9625.

show more ...

cranelift: fix typo (#9503)

Cranelift: Add a cargo feature to enable support for all native ISAs (#9237)

That is, all architectures other than Pulley.

Fixes #9229

Fix `cranelift-codegen` build script for Pulley backends (#9172)

* Fix `cranelift-codegen` build script for Pulley backends

The cargo feature is called "pulley" for both the pulley32 and pulley64
b

Fix `cranelift-codegen` build script for Pulley backends (#9172)

* Fix `cranelift-codegen` build script for Pulley backends

The cargo feature is called "pulley" for both the pulley32 and pulley64
backends, since they are the same code, but the build script was checking for
cargo features named "pulley32" and "pulley64" instead.

* Fix warnings in build

show more ...

Enforce `uninlined_format_args` for the workspace (#9065)

* Enforce `uninlined_format_args` for the workspace

* fix: failing `Monolith Checks` job

* fix: formatting

cranelift/codegen/meta: provide entry point to generate ISLE code only (#8462)

* extract isle code generation

* revert some customizations

* revert changes to meta/src/error.rs

move get_isle_compilations to codegen::meta (#8423)

cranelift: Optimize select_spectre_guard, carefully (#8139)

* cranelift: Optimize select_spectre_guard, carefully

This commit makes two changes to our treatment of
`select_spectre_guard`.

First, s

cranelift: Optimize select_spectre_guard, carefully (#8139)

* cranelift: Optimize select_spectre_guard, carefully

This commit makes two changes to our treatment of
`select_spectre_guard`.

First, stop annotating this instruction as having any side effects. We
only care that if its value result is used, then it's computed without
branching on the condition input. We don't otherwise care when the value
is computed, or if it's computed at all.

Second, introduce some carefully selected ISLE egraph rewrites for this
instruction. These particular rewrites are those where we can statically
determine which SSA value will be the result of the instruction. Since
there is no actual choice involved, there's no way to accidentally
introduce speculation on the condition input.

* Add filetests

show more ...

Support compilation-only build by adding a `runtime` feature (#7766)

* Add `runtime` feature to `wasmtime` crate

This feature can be disabled to build `wasmtime` only for compilation.
This can be u

Support compilation-only build by adding a `runtime` feature (#7766)

* Add `runtime` feature to `wasmtime` crate

This feature can be disabled to build `wasmtime` only for compilation.
This can be useful when cross-compiling, especially on a target that
can't run wasmtime itself (e.g. `wasm32`).

* prtest:full

* don't round pages without runtime feature

* fix async assertions

* move profiling into runtime

* enable runtime for wasmtime-wasi

* enable runtime for c-api

* fix build_artifacts in non-cache case

* fix miri extensions

* enable runtime for wast

* enable runtime for explorer

* support cranelift all-arch on wasm32

* add doc links for `WeakEngine`

* simplify lib runtime cfgs

* move limits and resources to runtime

* move stack to runtime

* move coredump and debug to runtime

* add runtime to coredump and async features

* add wasm32 build job

* combine engine modules

* single compile mod

* remove allow for macro paths

* add comments

show more ...

Add opt patterns for 3-way comparison (`Ord::cmp` or `<=>`) (#7636)

* Add opt patterns for 3-way comparison (`Ord::cmp` or `<=>`)

Compare clang: <https://cpp.godbolt.org/z/bTbe1556W>

* Add `spaces

Add opt patterns for 3-way comparison (`Ord::cmp` or `<=>`) (#7636)

* Add opt patterns for 3-way comparison (`Ord::cmp` or `<=>`)

Compare clang: <https://cpp.godbolt.org/z/bTbe1556W>

* Add `spaceship_[su]` extractors and constructors

show more ...

cranelift-codegen: add feature to explicitly enable the host ISA (#6551)

It is needed because cargo features are additive. Given a situation where you have two dependencies:

* One depends on cranel

cranelift-codegen: add feature to explicitly enable the host ISA (#6551)

It is needed because cargo features are additive. Given a situation where you have two dependencies:

* One depends on cranelift aarch64 codegen, so it enables the `aarch64` feature.
* The other depends on cranelift native codegen, so it does not enable any arch feature.

Given the additive property of features, cranelift-codegen will be built with the `aarch64` feature for both dependencies (assuming they use the same cranelift version), so the native ISA will not be included (unless it is aarch64).

With the `host-arch` feature added here, the native host ISA can now be explicitly requested without risk of another crate of the dependency tree disabling it.

show more ...

egraphs: Lift `splat` outside of int-to-float conversions (#6563)

This commit adds a targeted optimization aimed at fixing #6562 as a
temporary measure for now. The "real" fix for #6562 is to add a

egraphs: Lift `splat` outside of int-to-float conversions (#6563)

This commit adds a targeted optimization aimed at fixing #6562 as a
temporary measure for now. The "real" fix for #6562 is to add a full
lowering of `fcvt_from_uint` to the x64 backend, but for now adding this
rule should fix the specific issue cropping up.

Closes #6562

show more ...

riscv64: Initial SIMD Vector Implementation (#6240)

* riscv64: Remove unused code

* riscv64: Add vector types

* riscv64: Initial Vector ABI Load/Stores

* riscv64: Vector Loads/Stores

* riscv64:

riscv64: Initial SIMD Vector Implementation (#6240)

* riscv64: Remove unused code

* riscv64: Add vector types

* riscv64: Initial Vector ABI Load/Stores

* riscv64: Vector Loads/Stores

* riscv64: Fix `vsetvli` encoding error

* riscv64: Add SIMD `iadd` runtests

* riscv64: Rename `VecSew`

The SEW name is correct, but only for VType. We also use this type
in loads/stores as the Efective Element Width, so the name isn't
quite correct in that case.

* ci: Add V extension to RISC-V QEMU

* riscv64: Misc Cleanups

* riscv64: Check V extension in `load`/`store` for SIMD

* riscv64: Fix `sumop` doc comment

* cranelift: Fix comment typo

* riscv64: Add convert for VType and VecElementWidth

* riscv64: Remove VecElementWidth converter

show more ...

ISLE: split algebraic.isle into several files (#6140)

* ISLE: split algebraic.isle into several files

* delete `algebraic.clif`

* Add `README.md`

* Remove old `algebraic.clif` tests

---------

C

ISLE: split algebraic.isle into several files (#6140)

* ISLE: split algebraic.isle into several files

* delete `algebraic.clif`

* Add `README.md`

* Remove old `algebraic.clif` tests

---------

Co-authored-by: Jamey Sharp <[email protected]>

show more ...

ISLE: move `icmp` rewrites to separate file. (#6120)

* ISLE: move `icmp` rewrites to separate file.

Move `icmp`-related rewrite rules from `algebraic.isle` to `icmp.isle`.
Also move `icmp`-related

ISLE: move `icmp` rewrites to separate file. (#6120)

* ISLE: move `icmp` rewrites to separate file.

Move `icmp`-related rewrite rules from `algebraic.isle` to `icmp.isle`.
Also move `icmp`-related tests from `algebraic.clif` to `icmp.clif`.

* Put parameterized and unparameterized `icmp` tests in separate files

* Undo refactoring of (ir)reflexivity rewrites

* Fix `icmp-parameterised.clif`

* Undo formatting/comment changes

show more ...

cranelift-isle: Rewrite error reporting (#5318)

There were several issues with ISLE's existing error reporting
implementation.

- When using Miette for more readable error reports, it would panic

cranelift-isle: Rewrite error reporting (#5318)

There were several issues with ISLE's existing error reporting
implementation.

- When using Miette for more readable error reports, it would panic if
errors were reported from multiple files in the same run.
- Miette is pretty heavy-weight for what we're doing, with a lot of
dependencies.
- The `Error::Errors` enum variant led to normalization steps in many
places, to avoid using that variant to represent a single error.

This commit:
- replaces Miette with codespan-reporting
- gets rid of a bunch of cargo-vet exemptions
- replaces the `Error::Errors` variant with a new `Errors` type
- removes source info from `Error` variants so they're easy to construct
- adds source info only when formatting `Errors`
- formats `Errors` with a custom `Debug` impl
- shares common code between ISLE's callers, islec and cranelift-codegen
- includes a source snippet even with fancy-errors disabled

I tried to make this a series of smaller commits but I couldn't find any
good split points; everything was too entangled with everything else.

show more ...

egraph-based midend: draw the rest of the owl (productionized). (#4953)

* egraph-based midend: draw the rest of the owl.

* Rename `egg` submodule of cranelift-codegen to `egraph`.

* Apply some

egraph-based midend: draw the rest of the owl (productionized). (#4953)

* egraph-based midend: draw the rest of the owl.

* Rename `egg` submodule of cranelift-codegen to `egraph`.

* Apply some feedback from @jsharp during code walkthrough.

* Remove recursion from find_best_node by doing a single pass.

Rather than recursively computing the lowest-cost node for a given
eclass and memoizing the answer at each eclass node, we can do a single
forward pass; because every eclass node refers only to earlier nodes,
this is sufficient. The behavior may slightly differ from the earlier
behavior because we cannot short-circuit costs to zero once a node is
elaborated; but in practice this should not matter.

* Make elaboration non-recursive.

Use an explicit stack instead (with `ElabStackEntry` entries,
alongside a result stack).

* Make elaboration traversal of the domtree non-recursive/stack-safe.

* Work analysis logic in Cranelift-side egraph glue into a general analysis framework in cranelift-egraph.

* Apply static recursion limit to rule application.

* Fix aarch64 wrt dynamic-vector support -- broken rebase.

* Topo-sort cranelift-egraph before cranelift-codegen in publish script, like the comment instructs me to!

* Fix multi-result call testcase.

* Include `cranelift-egraph` in `PUBLISHED_CRATES`.

* Fix atomic_rmw: not really a load.

* Remove now-unnecessary PartialOrd/Ord derivations.

* Address some code-review comments.

* Review feedback.

* Review feedback.

* No overlap in mid-end rules, because we are defining a multi-constructor.

* rustfmt

* Review feedback.

* Review feedback.

* Review feedback.

* Review feedback.

* Remove redundant `mut`.

* Add comment noting what rules can do.

* Review feedback.

* Clarify comment wording.

* Update `has_memory_fence_semantics`.

* Apply @jameysharp's improved loop-level computation.

Co-authored-by: Jamey Sharp <[email protected]>

* Fix suggestion commit.

* Fix off-by-one in new loop-nest analysis.

* Review feedback.

* Review feedback.

* Review feedback.

* Use `Default`, not `std::default::Default`, as per @fitzgen

Co-authored-by: Nick Fitzgerald <[email protected]>

* Apply @fitzgen's comment elaboration to a doc-comment.

Co-authored-by: Nick Fitzgerald <[email protected]>

* Add stat for hitting the rewrite-depth limit.

* Some code motion in split prelude to make the diff a little clearer wrt `main`.

* Take @jameysharp's suggested `try_into()` usage for blockparam indices.

Co-authored-by: Jamey Sharp <[email protected]>

* Take @jameysharp's suggestion to avoid double-match on load op.

Co-authored-by: Jamey Sharp <[email protected]>

* Fix suggestion (add import).

* Review feedback.

* Fix stack_load handling.

* Remove redundant can_store case.

* Take @jameysharp's suggested improvement to FuncEGraph::build() logic

Co-authored-by: Jamey Sharp <[email protected]>

* Tweaks to FuncEGraph::build() on top of suggestion.

* Take @jameysharp's suggested clarified condition

Co-authored-by: Jamey Sharp <[email protected]>

* Clean up after suggestion (unused variable).

* Fix loop analysis.

* loop level asserts

* Revert constant-space loop analysis -- edge cases were incorrect, so let's go with the simple thing for now.

* Take @jameysharp's suggestion re: result_tys

Co-authored-by: Jamey Sharp <[email protected]>

* Fix up after suggestion

* Take @jameysharp's suggestion to use fold rather than reduce

Co-authored-by: Jamey Sharp <[email protected]>

* Fixup after suggestion

* Take @jameysharp's suggestion to remove elaborate_eclass_use's return value.

* Clarifying comment in terminator insts.

Co-authored-by: Jamey Sharp <[email protected]>
Co-authored-by: Nick Fitzgerald <[email protected]>

show more ...