Cranelift: Generate integer numeric ops and conversions for ISLE in the meta crate (#11065)

* Cranelift: Generate integer numeric ops and conversions for ISLE in the meta crate

This automatically g

Cranelift: Generate integer numeric ops and conversions for ISLE in the meta crate (#11065)

* Cranelift: Generate integer numeric ops and conversions for ISLE in the meta crate

This automatically generates operations and conversions for integer types for
use in ISLE.

Supported types are: `{i,u}{8,16,32,64,128}`

We generate

* Comparisons (eq, ne, lt, lt_eq, gt, gt_eq)
* Arithmetic operations (add, sub, mul, div, neg)
* These each have checked, wrapping, and unwrapping variants
* Bitwise operations (and, or, xor, shifts, counting leading/trailing zeros/ones)
* A variety of predicates (is_zero, is_power_of_two, is_odd, etc...)
* These generate both partial constructors and a handful of extractors
* Conversions
* These come in a variety of flavors: fallible, infallible, truncating,
unwrapping, sign-reinterpretation
* Fallible conversions are also available as an extractor

* Fix copy paste

* Rename `x_reinterpret_as_y` to `x_cast_[un]signed`

* Collapse some fallible conversions in pulley lowering

* Clean up pulley iconst lowering, make sure narrowest `xconst*` instruction is always used

* Avoid an unnecessary truncation in riscv64 lowering

* Use extractor instead of partial constructor in x64 `imm` rule

* Clean up `op mem, imm` x64 lowering rules

* Use `(i64_eq a b)` instead of `(u64_eq (i64_cast_unsigned a) (i64_cast_unsigned b))`

* Rename `<ty>_unwrapping_<op>` to `<ty>_<op>`

show more ...

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtim

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtime to the 2024 Edition to keep
up-to-date with Rust idioms and access many of the edition features
exclusive to the 2024 edition.

prtest:full

* Reformat with the 2024 edition

show more ...

meta: move all ISLE-related generated code to `cranelift-codegen-meta` (#10352)

* meta: move all ISLE-related generated code to `cranelift-codegen-meta`

Previously, the new `cranelift-assembler-x64

meta: move all ISLE-related generated code to `cranelift-codegen-meta` (#10352)

* meta: move all ISLE-related generated code to `cranelift-codegen-meta`

Previously, the new `cranelift-assembler-x64` crate would generate the
assembler Rust code, the ISLE integration code, and the ISLE's
accompanying Rust macro all in the same crate. Because these last two
files were generated in the assembler's `OUT_DIR`, they caused issues
for deterministic builds that expect those files in
`cranelift-codegen`'s `OUT_DIR` (see #10348). This change fixes that by
moving all of the assembler's generated integration code, the ISLE
instruction definitions and their accompanying Rust macro, to the
`cranelift-codegen-meta` crate.

This has the added advantage that `cranelift-assembler-x64` now can
focus exclusively on being an assembler and not being concerned with the
ISLE integration, which is substantial.

* Move lint `allow` to problem site

Apparently allowing unused variables for unimplemented parts of fixed
register instructions will not work at the macro scope; this moves the
`allow` to the offending line.

* Remove `rustfmt` of generated assembler code

show more ...

meta: deduplicate source generation infrastructure (#10348)

* Move `Formatter` and `Error` to new crate: `cranelift-srcgen`

The `cranelift-codegen-meta` crate emits both Rust and ISLE source code
u

meta: deduplicate source generation infrastructure (#10348)

* Move `Formatter` and `Error` to new crate: `cranelift-srcgen`

The `cranelift-codegen-meta` crate emits both Rust and ISLE source code
using a `Formatter` which may fail with an `Error`. The
`cranelift-assembler-x64-meta` crate uses a subset of this
functionality. To deduplicate efforts, this creates a new crate,
`cranelift-srcgen`, and moves only the `cranelift-codgen-meta`
functionality there (for now).

* Add notion of a formatted `Language`; append file locations

While developing `cranelift-assembler-x64`, it became clear that knowing
the source code location that _generated_ some code was extremely
useful. This change alters the `fmtln!` macro to append file locations
as comments to the generated code; this doesn't change all the
`fmt.line()` invocations but it's a start. To do this, we need to know
which language we are generating to emit the correct comment kind. This
led to adding a new `enum Language` to identify that.

* Rename `update_file` to `write`

* Use shared `Formatter` in `cranelift-assembler-x64-meta`

This change removes the duplicate implementation of `Formatter` in
`cranelift-assembler-x64-meta` and replaces it with the shared
`Formatter` in `cranelift-srcgen`.

* Add `Formatter::add_block`

While implementing all these changes, I noticed an opportunity: by
adding `Formatter::add_block` we can automatically adds braces and
indentation. This should be more safe (i.e., harder to forget to append
the ending `}`) and result in less verbose generator code (no need for
`fmtln!(...)` then `f.indent(|f| ...)`). This refactoring does touch a
large amount of code, though.

Now with `Formatter::add_block`, we can allow the `fmtln!` macro to
auto-format no-arg format strings (e.g., `fmtln!(f, "{x}")`), which
should be a bit more ergonomic. This does mean that a few locations that
weren't covered by `Formatter::add_block` need some extra escaping
(e.g., `{` -> `{{`).

* Avoid appending a file location comment to a Rust comment

* Make `cranelift-srcgen` publishable

This ensures the crate has a version and is tracked by
`cranelift-srcgen`.

show more ...

Procedurally generate Pulley Cranelift boilerplate (#9760)

* Procedurally generate Pulley Cranelift boilerplate

This commit is an integration of the `for_each_op!` macro (and extended
ops) for Cran

Procedurally generate Pulley Cranelift boilerplate (#9760)

* Procedurally generate Pulley Cranelift boilerplate

This commit is an integration of the `for_each_op!` macro (and extended
ops) for Cranelift. This procedurally generates a few new items for
Cranelift's Pulley backend to use:

* `RawInst` - a raw enumeration of instructions as-is.
* ISLE constructors (e.g. `pulley_*` ctors) - generated for all of the
`RawInst` variants.
* Register allocation methods for `RawInst`
* Pretty printing methods for `RawInst`
* Emission methods for `RawInst`

The `Inst` enum now has a `Raw` variant which contains a `RawInst`. In
this manner the main `Inst` enum can still have pseudo-insts like
`Call`, polymorphic instructions like loads/stores (probably gonna get
refactored in the future though), and slightly different representations
such as `Inst::Trap` having a `TrapCode` and `RawInst::Trap` wouldn't.

The goal of this commit is to lower the amount of effort to quickly add
and experiment with new instructions in Pulley. Ideally it's now just
(a) define them in the pulley macro, (b) implement a direct lowering
rule, and (c) implement it in the interpreter. Ideally no need to
implement anything else inside of Cranelift as everything should be
auto-generated.

Many existing `Inst` variants have been deleted in favor of their
equivalents in `RawInst` now. This undeniably increases the complexity
of the Pulley backend but at least for me I find it well worth it to
have all this boilerplate generated automatically.

* Fill out TODO

* Fix conditional generation of pulley ISLE

show more ...

Enforce `uninlined_format_args` for the workspace (#9065)

* Enforce `uninlined_format_args` for the workspace

* fix: failing `Monolith Checks` job

* fix: formatting

cranelift/codegen/meta: provide entry point to generate ISLE code only (#8462)

* extract isle code generation

* revert some customizations

* revert changes to meta/src/error.rs

move get_isle_compilations to codegen::meta (#8423)

A bunch of cleanups for cranelift-codegen-meta (#6772)

* Remove shared_defs args from all per-isa definitions

They don't and shouldn't define any isa-independent things

* Inline all define_setting

A bunch of cleanups for cranelift-codegen-meta (#6772)

* Remove shared_defs args from all per-isa definitions

They don't and shouldn't define any isa-independent things

* Inline all define_settings functions

* Merge verify_instruction_formats into shared::define()

It isn't modified in between anyway now.

* Enable document private items for cranelift-codegen-meta

The mentioned cargo bug has been fixed

* Remove unused testing_hooks cargo feature

It has been unused ever since we removed the old backend framework.

show more ...

Add clippy suggestions (#6203)

* add clippy suggestions

* revert &/ref change

* Update cranelift/isle/isle/src/parser.rs

Co-authored-by: Jamey Sharp <[email protected]>

---------

Co-authored-by

Add clippy suggestions (#6203)

* add clippy suggestions

* revert &/ref change

* Update cranelift/isle/isle/src/parser.rs

Co-authored-by: Jamey Sharp <[email protected]>

---------

Co-authored-by: Jamey Sharp <[email protected]>

show more ...

egraph-based midend: draw the rest of the owl (productionized). (#4953)

* egraph-based midend: draw the rest of the owl.

* Rename `egg` submodule of cranelift-codegen to `egraph`.

* Apply some

egraph-based midend: draw the rest of the owl (productionized). (#4953)

* egraph-based midend: draw the rest of the owl.

* Rename `egg` submodule of cranelift-codegen to `egraph`.

* Apply some feedback from @jsharp during code walkthrough.

* Remove recursion from find_best_node by doing a single pass.

Rather than recursively computing the lowest-cost node for a given
eclass and memoizing the answer at each eclass node, we can do a single
forward pass; because every eclass node refers only to earlier nodes,
this is sufficient. The behavior may slightly differ from the earlier
behavior because we cannot short-circuit costs to zero once a node is
elaborated; but in practice this should not matter.

* Make elaboration non-recursive.

Use an explicit stack instead (with `ElabStackEntry` entries,
alongside a result stack).

* Make elaboration traversal of the domtree non-recursive/stack-safe.

* Work analysis logic in Cranelift-side egraph glue into a general analysis framework in cranelift-egraph.

* Apply static recursion limit to rule application.

* Fix aarch64 wrt dynamic-vector support -- broken rebase.

* Topo-sort cranelift-egraph before cranelift-codegen in publish script, like the comment instructs me to!

* Fix multi-result call testcase.

* Include `cranelift-egraph` in `PUBLISHED_CRATES`.

* Fix atomic_rmw: not really a load.

* Remove now-unnecessary PartialOrd/Ord derivations.

* Address some code-review comments.

* Review feedback.

* Review feedback.

* No overlap in mid-end rules, because we are defining a multi-constructor.

* rustfmt

* Review feedback.

* Review feedback.

* Review feedback.

* Review feedback.

* Remove redundant `mut`.

* Add comment noting what rules can do.

* Review feedback.

* Clarify comment wording.

* Update `has_memory_fence_semantics`.

* Apply @jameysharp's improved loop-level computation.

Co-authored-by: Jamey Sharp <[email protected]>

* Fix suggestion commit.

* Fix off-by-one in new loop-nest analysis.

* Review feedback.

* Review feedback.

* Review feedback.

* Use `Default`, not `std::default::Default`, as per @fitzgen

Co-authored-by: Nick Fitzgerald <[email protected]>

* Apply @fitzgen's comment elaboration to a doc-comment.

Co-authored-by: Nick Fitzgerald <[email protected]>

* Add stat for hitting the rewrite-depth limit.

* Some code motion in split prelude to make the diff a little clearer wrt `main`.

* Take @jameysharp's suggested `try_into()` usage for blockparam indices.

Co-authored-by: Jamey Sharp <[email protected]>

* Take @jameysharp's suggestion to avoid double-match on load op.

Co-authored-by: Jamey Sharp <[email protected]>

* Fix suggestion (add import).

* Review feedback.

* Fix stack_load handling.

* Remove redundant can_store case.

* Take @jameysharp's suggested improvement to FuncEGraph::build() logic

Co-authored-by: Jamey Sharp <[email protected]>

* Tweaks to FuncEGraph::build() on top of suggestion.

* Take @jameysharp's suggested clarified condition

Co-authored-by: Jamey Sharp <[email protected]>

* Clean up after suggestion (unused variable).

* Fix loop analysis.

* loop level asserts

* Revert constant-space loop analysis -- edge cases were incorrect, so let's go with the simple thing for now.

* Take @jameysharp's suggestion re: result_tys

Co-authored-by: Jamey Sharp <[email protected]>

* Fix up after suggestion

* Take @jameysharp's suggestion to use fold rather than reduce

Co-authored-by: Jamey Sharp <[email protected]>

* Fixup after suggestion

* Take @jameysharp's suggestion to remove elaborate_eclass_use's return value.

* Clarifying comment in terminator insts.

Co-authored-by: Jamey Sharp <[email protected]>
Co-authored-by: Nick Fitzgerald <[email protected]>

show more ...

Cranelift: do not check in generated ISLE code; regenerate on every compile. (#4143)

This PR fixes #4066: it modifies the Cranelift `build.rs` workflow to
invoke the ISLE DSL compiler on every comp

Cranelift: do not check in generated ISLE code; regenerate on every compile. (#4143)

This PR fixes #4066: it modifies the Cranelift `build.rs` workflow to
invoke the ISLE DSL compiler on every compilation, rather than only
when the user specifies a special "rebuild ISLE" feature.

The main benefit of this change is that it vastly simplifies the mental
model required of developers, and removes a bunch of failure modes
we have tried to work around in other ways. There is now just one
"source of truth", the ISLE source itself, in the repository, and so there
is no need to understand a special "rebuild" step and how to handle
merge errors. There is no special process needed to develop the compiler
when modifying the DSL. And there is no "noise" in the git history produced
by constantly-regenerated files.

The two main downsides we discussed in #4066 are:
- Compile time could increase, by adding more to the "meta" step before the main build;
- It becomes less obvious where the source definitions are (everything becomes
more "magic"), which makes exploration and debugging harder.

This PR addresses each of these concerns:

1. To maintain reasonable compile time, it includes work to cut down the
dependencies of the `cranelift-isle` crate to *nothing* (only the Rust stdlib),
in the default build. It does this by putting the error-reporting bits
(`miette` crate) under an optional feature, and the logging (`log` crate) under
a feature-controlled macro, and manually writing an `Error` impl rather than
using `thiserror`. This completely avoids proc macros and the `syn` build slowness.

The user can still get nice errors out of `miette`: this is enabled by specifying
a Cargo feature `--features isle-errors`.

2. To allow the user to optionally inspect the generated source, which nominally
lives in a hard-to-find path inside `target/` now, this PR adds a feature `isle-in-source-tree`
that, as implied by the name, moves the target for ISLE generated source into
the source tree, at `cranelift/codegen/isle_generated_source/`. It seems reasonable
to do this when an explicit feature (opt-in) is specified because this is how ISLE regeneration
currently works as well. To prevent surprises, if the feature is *not* specified, the
build fails if this directory exists.

show more ...

Move generate_table from cranelift-codegen-shared to cranelift-codegen-meta

Initial ISLE integration with the x64 backend

On the build side, this commit introduces two things:

1. The automatic generation of various ISLE definitions for working with
CLIF. Specifically, it g

Initial ISLE integration with the x64 backend

On the build side, this commit introduces two things:

1. The automatic generation of various ISLE definitions for working with
CLIF. Specifically, it generates extern type definitions for clif opcodes and
the clif instruction data `enum`, as well as extractors for matching each clif
instructions. This happens inside the `cranelift-codegen-meta` crate.

2. The compilation of ISLE DSL sources to Rust code, that can be included in the
main `cranelift-codegen` compilation.

Next, this commit introduces the integration glue code required to get
ISLE-generated Rust code hooked up in clif-to-x64 lowering. When lowering a clif
instruction, we first try to use the ISLE code path. If it succeeds, then we are
done lowering this instruction. If it fails, then we proceed along the existing
hand-written code path for lowering.

Finally, this commit ports many lowering rules over from hand-written,
open-coded Rust to ISLE.

In the process of supporting ISLE, this commit also makes the x64 `Inst` capable
of expressing SSA by supporting 3-operand forms for all of the existing
instructions that only have a 2-operand form encoding:

dst = src1 op src2

Rather than only the typical x86-64 2-operand form:

dst = dst op src

This allows `MachInst` to be in SSA form, since `dst` and `src1` are
disentangled.

("3-operand" and "2-operand" are a little bit of a misnomer since not all
operations are binary operations, but we do the same thing for, e.g., unary
operations by disentangling the sole operand from the result.)

There are two motivations for this change:

1. To allow ISLE lowering code to have value-equivalence semantics. We want ISLE
lowering to translate a CLIF expression that evaluates to some value into a
`MachInst` expression that evaluates to the same value. We want both the
lowering itself and the resulting `MachInst` to be pure and referentially
transparent. This is both a nice paradigm for compiler writers that are
authoring and maintaining lowering rules and is a prerequisite to any sort of
formal verification of our lowering rules in the future.

2. Better align `MachInst` with `regalloc2`'s API, which requires that the input
be in SSA form.

show more ...

Remove default_map

Remove x86 old-backend special case from cranelift-codegen-meta

Flatten directory structure for cranelift_codegen_meta::isa

Remove registers from cranelift-codegen-meta

Remove legalizer support from cranelift-codegen-meta

Remove encoding generation from cranelift-codegen-meta

Remove the old riscv backend

Support IBM z/Architecture

This adds support for the IBM z/Architecture (s390x-ibm-linux).

The status of the s390x backend in its current form is:
- Wasmtime is fully functional and passes all test

Support IBM z/Architecture

This adds support for the IBM z/Architecture (s390x-ibm-linux).

The status of the s390x backend in its current form is:
- Wasmtime is fully functional and passes all tests on s390x.
- All back-end features supported, with the exception of SIMD.
- There is still a lot of potential for performance improvements.
- Currently the only supported processor type is z15.

show more ...

machinst x64: Only use the feature flag to enable the x64 new backend;

Before this patch, running the x64 new backend would require both
compiling with --features experimental_x64 and running with
`

machinst x64: Only use the feature flag to enable the x64 new backend;

Before this patch, running the x64 new backend would require both
compiling with --features experimental_x64 and running with
`use_new_backend`.

This patches changes this behavior so that the runtime flag is not
needed anymore: using the feature flag will enforce usage of the new
backend everywhere, making using and testing it much simpler:

cargo run --features experimental_x64 ;; other CLI options/flags

This also gives a hint at what the meta language generation would look
like after switching to the new backend.

Compiling only with the x64 codegen flag gives a nice compile time speedup.

show more ...

Add documentation for top-level items in cranelift-codegen/meta

Fix clippy warnings.

This commit fixes the current set of (stable) clippy warnings in the repo.