PCC: remove proof-carrying code (for now?). (#12800)

In late 2023, we built out an experimental feature called
Proof-Carrying Code (PCC), where we attached "facts" to values in the
CLIF IR and built

PCC: remove proof-carrying code (for now?). (#12800)

In late 2023, we built out an experimental feature called
Proof-Carrying Code (PCC), where we attached "facts" to values in the
CLIF IR and built verification of these facts after lowering to
machine instructions. We also added "memory types" describing layout
of memory and a "checked" flag on memory operations such that we could
verify that any checked memory operation accessed valid memory (as
defined by memory types attached to pointer values via
facts). Wasmtime's Cranelift backend then put appropriate memory types
and facts in its IR such that all accesses to memory (aspirationally)
could be checked, taking the whole mid-end and lowering backend of
Cranelift out of the trusted core that enforces SFI.

This basically worked, at the time, for static memories; but never for
dynamic memories, and then work on the feature lost
prioritization (aka I had to work on other things) and I wasn't able
to complete it and put it in fuzzing/enable it as a production option.

Unfortunately since then it has bit-rotted significantly -- as we add
new backend optimizations and instruction lowerings we haven't kept
the PCC framework up to date.

Inspired by the discussion in #12497 I think it's time to delete
it (hopefully just "for now"?) unless/until we can build it again. And
when we do that, we should probably get it to the point of validating
robust operation on all combinations of memory configurations before
merging. (That implies a big experiment branch rather than a bunch of
eager PRs in-tree, but so it goes.) I still believe it is possible to
build this (and I have ideas on how to do it!) but not right now.

show more ...

Partial support for no_std in cranelift_codegen (#12222)

* Move most things from std to core and alloc

* Port assembler_x64 to no_std

* before adding prelude to each file

* Most of the files now

Partial support for no_std in cranelift_codegen (#12222)

* Move most things from std to core and alloc

* Port assembler_x64 to no_std

* before adding prelude to each file

* Most of the files now work with no_std

* update isle to use alloc and core

* some instances shouldn't have been renamed, fixes cargo test

* add cranelift-assembler-x64 (no_std) to CI

* fix codegen_meta, missed one spot with std::slice

* automatically remove prelude with cargo fix

* update isle changes

* update assembler changes

* update assembler changes

* use latest codegen changes + fix FxHash problem

* add imports

* fix floating issues with libm

* remove unused import

* temporarily remove OnceLock

* add no_std arm support and add it into CI

* Move most things from std to core and alloc

* Port assembler_x64 to no_std

* before adding prelude to each file

* Most of the files now work with no_std

* update isle to use alloc and core

* some instances shouldn't have been renamed, fixes cargo test

* add cranelift-assembler-x64 (no_std) to CI

* automatically remove prelude with cargo fix

* update isle changes

* update assembler changes

* update assembler changes

* use latest codegen changes + fix FxHash problem

* add imports

* fix floating issues with libm

* remove unused import

* temporarily remove OnceLock

* add no_std arm support and add it into CI

* Move most things from std to core and alloc

* Port assembler_x64 to no_std

* before adding prelude to each file

* Most of the files now work with no_std

* update isle to use alloc and core

* add cranelift-assembler-x64 (no_std) to CI

* automatically remove prelude with cargo fix

* update isle changes

* update assembler changes

* use latest codegen changes + fix FxHash problem

* add imports

* fix floating issues with libm

* temporarily remove OnceLock

* add no_std arm support and add it into CI

* revert Cargo.toml formating

* remove prelude and fix cargo.toml

* cargo fmt

* remove empty lines

* bad renames

* macro_use only on no_std

* revert OnceLock change

* only use stable libm features

* update regalloc2

* update comment

* use continue instead

* Update vets

---------

Co-authored-by: Alex Crichton <[email protected]>

show more ...

Pull in new regalloc2 with fastalloc fixes for exceptions, and re-enable and add to testing. (#11533)

* Revert "Cranelift/Wasmtime: disable fastalloc (single-pass) allocator for now. (#10554)"

This

Pull in new regalloc2 with fastalloc fixes for exceptions, and re-enable and add to testing. (#11533)

* Revert "Cranelift/Wasmtime: disable fastalloc (single-pass) allocator for now. (#10554)"

This reverts commit d52e23b09191185996792b8ef18e5fca2865ca43.

* Upgrade to regalloc2 0.13.1.

Pulls in bytecodealliance/regalloc2#233 to update fastalloc to support
the looser constraints needed by exception-related changes.

* cargo-vet update.

show more ...

Update the `log` dependency (#11197)

* Update the `log` dependency

This enables getting warnings about formatting strings in the `log`
crate directives which are then additionally fixed here as wel

Update the `log` dependency (#11197)

* Update the `log` dependency

This enables getting warnings about formatting strings in the `log`
crate directives which are then additionally fixed here as well.

* Update dependency directive in `Cargo.toml`

show more ...

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtim

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtime to the 2024 Edition to keep
up-to-date with Rust idioms and access many of the edition features
exclusive to the 2024 edition.

prtest:full

* Reformat with the 2024 edition

show more ...

Refactor call ABI implementation (#10722)

This refactors implementation of call ABI handling across architectures
with the goal of bringing s390x in line with other platforms.

The main idea is to
-

Refactor call ABI implementation (#10722)

This refactors implementation of call ABI handling across architectures
with the goal of bringing s390x in line with other platforms.

The main idea is to
- handle main call instruction selection and generation in ISLE
(like s390x but unlike other platforms today)
- handle argument setup mostly outside of ISLE
(like other platforms but unlike s390x today)
- handle return value processing as part of the call instructio
(like all platforms today)

All platforms now emit the main call instruction directly from ISLE,
which e.g. handles selection of the correct ISA instruction depending
on the call destination. This ISLE code calls out to helper routines
to handle argument and return value processing. These helpers are
mostly common code and provided by the Callee and/or Lower layers,
with some platform-specific additions via ISLE Context routines.

The old CallSite abstraction is no longer needed; most of the
differences between call and return_call handling disappear.
(There is still a common-code CallInfo vs. a platform-specifc
ReturnCallInfo. At this point, it should be relatively straight-
forward to make CallInfo platform-specific as well if desired,
but this is not done here.)

Some ISLE infrastructure for iterators / loops, which was only
ever used by the s390x argument processing code, has been removed.

s390x now closely matches all other platforms, with only a few
special cases (slightly different tail-call ABI requires some
differences in stack offset computations; we still need to
handle vector lane swaps for cross-ABI calls), which should
simplify future maintenance.

show more ...

Cranelift/Wasmtime: disable fastalloc (single-pass) allocator for now. (#10554)

Unfortunately, as discovered by a recent fuzzbug [1], the single-pass
register allocator is not compatible with the ap

Cranelift/Wasmtime: disable fastalloc (single-pass) allocator for now. (#10554)

Unfortunately, as discovered by a recent fuzzbug [1], the single-pass
register allocator is not compatible with the approach to callsite
defs that exception-handling support has forced us to take. In
particular, we needed to move all call return-value defs onto the call
instruction itself, so calls could be terminators; this unbounded
number of defs is made to be a solvable allocation problem by using
`any` constraints, which allow allocation directly into spillslots;
but fastalloc appears to error out if it runs out of registers,
regardless of this constraint.

Long-term, we should fix this, but unfortunately I don't have cycles
to dive into fastalloc's internals at the moment, and it's (I think) a
tier-3 feature. As such, this PR disables its use for now. I've
filed a tracking issue in RA2 [2], and referenced this in the
Cranelift configuration option docs at least.

To keep from shifting all fuzzbugs / fuzzing corpii by altering the
`arbitrary` interpretation, I opted to keep the enum the same in the
fuzzing crate, and remap `SinglePass` to `Backtracking` there. I'm
happy to take the other approach and remove the option (thus
invalidating all fuzzbugs) if we'd prefer that instead.

[1]: https://oss-fuzz.com/testcase-detail/5433312476987392
[2]: https://github.com/bytecodealliance/regalloc2/issues/217

show more ...

Cranelift: add option to use new single-pass register allocator. (#9611)

* Cranelift: add option to use new single-pass register allocator.

In bytecodealliance/regalloc2#181, @d-sonuga added a fast

Cranelift: add option to use new single-pass register allocator. (#9611)

* Cranelift: add option to use new single-pass register allocator.

In bytecodealliance/regalloc2#181, @d-sonuga added a fast single-pass
algorithm option to regalloc2, in addition to its existing backtracking
allocator. This produces code much more quickly, at the expense of code
quality. Sometimes this tradeoff is desirable (e.g. when performing a
debug build in a fast-iteration development situation, or in an initial
JIT tier).

This PR adds a Cranelift option to select the RA2 algorithm, plumbs it
through to a Wasmtime option, and adds the option to Wasmtime fuzzing as
well.

An initial compile-time measurement in Wasmtime: `spidermonkey.wasm`
builds in 1.383s with backtracking (existing algorithm), and 1.065s with
single-pass. The resulting binary runs a simple Fibonacci benchmark in
2.060s with backtracking vs. 3.455s with single-pass.

Hence, the single-pass algorithm yields a 23% compile-time reduction, at
the cost of a 67% runtime increase.

* cargo-vet audit for allocator-api2 0.2.18 -> 0.2.20.

show more ...

PCC: support x86-64. (#7352)

* PCC: support x86-64.

This PR extends the proof-carrying-code infrastructure to support x86-64
as well as aarch64. In the process, many of the mechanisms had to be
mad

PCC: support x86-64. (#7352)

* PCC: support x86-64.

This PR extends the proof-carrying-code infrastructure to support x86-64
as well as aarch64. In the process, many of the mechanisms had to be
made a little more general.

One important change is that the PCC leaves more "breadcrumbs" on the
frontend now, avoiding the need for magic handling of facts on constant
values, etc., in the backend. For the first time a lowering rule also
gains the ability to add a fact to a vreg to preserve the chain as well.

With these changes, we can validate compilation of SpiderMonkey.wasm
with Wasm static memories on x86-64 and aarch64:

```
cfallin@fastly2:~/work/wasmtime% target/release/wasmtime compile -C pcc=yes --target x86_64 ../wasm-tests/spidermonkey.wasm
cfallin@fastly2:~/work/wasmtime% target/release/wasmtime compile -C pcc=yes --target aarch64 ../wasm-tests/spidermonkey.wasm
cfallin@fastly2:~/work/wasmtime%
```

* Don't run regalloc checker if not requested in addition to PCC; it's fairly expensive.

* Refactor x64 PCC code to avoid deep pattern matches on Gpr/Xmm types; explicitly match every instruction kind.

show more ...

PCC: draw the rest of the owl: fully-working PCC on hello-world Wasm on aarch64 (#7281)

* PCC: draw the rest of the owl: fully-working PCC on hello-world Wasm on aarch64

This needed a bit more infe

PCC: draw the rest of the owl: fully-working PCC on hello-world Wasm on aarch64 (#7281)

* PCC: draw the rest of the owl: fully-working PCC on hello-world Wasm on aarch64

This needed a bit more inference / magic than I was hoping for at first,
specifically around constants and adds. Some instructions can now
generate facts on their output registers, even if not stated. This
breaks away from the "breadcrumbs" idea, but seems to be the most
practical solution to a general problem that we have mini-lowering steps
in various places without careful preservation of PCC facts. Two
particular aspects:

- Constants: amodes on aarch64 can decompose into new
constant-generation instructions, and we need precise ranges on those
to properly check them. To avoid making the ISLE rules nightmarish,
it's best to reuse the existing semantics definitions of the Add* ALU
insts, and add a few rules for MovK/MovZ/MovN.

- Adds: similarly, amodes decompose into de-novo add instructions with
no facts. To handle this, there's now a notion of "propagating" facts
that cause an instruction with a propagating fact on the input to
generate a fact on the output.

Together, these heuristics mean that we'll eagerly generate a fact for
`mem(mt0, 0, 0) + 8 -> mem(mt0, 8, 8)`, but we won't, for example,
generate ranges on every single integer operation.

With these changes and a few other misc fixes, this PR can now check a
nontrivial "hello world" Wasm on aarch64 down to the machine-code level:

```
$ target/release/wasmtime compile -C enable-pcc=y ../wasm-tests/helloworld-rs.wasm
```

* Review feedback.

show more ...

PCC: add semantics for core add/shift/extend/amode ops on AArch64. (#7180)

* PCC: add semantics for core add/shift/extend/amode ops on AArch64.

This PR adds verification of facts on values produced

PCC: add semantics for core add/shift/extend/amode ops on AArch64. (#7180)

* PCC: add semantics for core add/shift/extend/amode ops on AArch64.

This PR adds verification of facts on values produced by adds, shifts,
and extends on AArch64, handling the various combination instructions
(adds with builtin extends or shifts, for example), and also adds
verification of all addressing modes, including those with builtin
extends and shifts.

It also splits the test suite into"succeed" and "fail" sets, and
provides cases that PCC should catch.

* Review feedback.

show more ...

Skeleton and initial support for proof-carrying code. (#7165)

* WIP veriwasm 2.0

Co-Authored-By: Chris Fallin <[email protected]>

* PCC: successfully parse some simple facts.

Co-authored-by: Nic

Skeleton and initial support for proof-carrying code. (#7165)

* WIP veriwasm 2.0

Co-Authored-By: Chris Fallin <[email protected]>

* PCC: successfully parse some simple facts.

Co-authored-by: Nick Fitzgerald <[email protected]>

* PCC: plumb facts through VCode and add framework on LowerBackend to check them.

Co-authored-by: Nick Fitzgerald <[email protected]>

* PCC: code is carrying some proofs! Very simple test-case.

Co-authored-by: Nick Fitzgerald <[email protected]>

* PCC: add a `safe` flag for checked memory accesses.

* PCC: add pretty-printing of facts to CLIF output.

* PCC: misc. cleanups.

* PCC: lots of cleanup.

* Post-rebase fixups and some misc. fixes.

* Add serde traits to facts.

* PCC: add succeed and fail tests.

* Review feedback: rename `safe` memflag to `checked`.

* Review feedback.

---------

Co-authored-by: Nick Fitzgerald <[email protected]>

show more ...

Make MachineEnv a per-ABI property (#6957)

The MachineEnv structure contains the allocatable and preferred
register sets. This is currently fixed per TargetIsa - however,
conceptually these registe

Make MachineEnv a per-ABI property (#6957)

The MachineEnv structure contains the allocatable and preferred
register sets. This is currently fixed per TargetIsa - however,
conceptually these register sets can differ between ABIs on the
same ISA.

To allow for this, replace the TargetIsa machine_env routine with
an ABIMachineSpec get_machine_env routine. To ensure the structure
is still only allocated once, cache it via static OnceLock variables.

No functional change intended.

show more ...

fuzz: Insert random instructions (#6407)

* Fix fuel consumption of ControlPlane::shuffle

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <mzrw.d

fuzz: Insert random instructions (#6407)

* Fix fuel consumption of ControlPlane::shuffle

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>

* Insert random instructions during lowering

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>

* add documentation for get_arbitrary

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>

* Fix zero-sized version of get_arbitrary

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>

* Insert ints and floats

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>

* fix inserting of floats

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>

* improve abstraction of MachInst::gen_imm_f64

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>

---------

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>

show more ...

fuzz: randomize block lowering order (#6254)

* fuzz: randomize block lowering order

Co-authored-by: Moritz Waser <[email protected]>
Co-authored-by: Remo Senekowitsch <[email protected]>

* fix block

fuzz: randomize block lowering order (#6254)

* fuzz: randomize block lowering order

Co-authored-by: Moritz Waser <[email protected]>
Co-authored-by: Remo Senekowitsch <[email protected]>

* fix block lowering order randomization

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>

* simplify control plane internals

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>

* avoid unnecessary allocations

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>

* remove unused change_order function

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>

* add arbitrary 1.3.0 to cargo vet imports lock

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>

* optimize ControlPlane::shuffle

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>

* clarify shuffle being a noop without chaos mode

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>

* reorder only direct successors of a block

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>

* rename get_permutation -> shuffled

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>

---------

Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>

show more ...

Reuse the DominatorTree postorder travesal in BlockLoweringOrder (#5843)

* Rework the blockorder module to reuse the dom tree's cfg postorder

* Update domtree tests

* Treat br_table with an empty

Reuse the DominatorTree postorder travesal in BlockLoweringOrder (#5843)

* Rework the blockorder module to reuse the dom tree's cfg postorder

* Update domtree tests

* Treat br_table with an empty jump table as multiple block exits

* Bless tests

* Change branch_idx to succ_idx and fix the comment

show more ...

Remove MachInst::gen_constant (#5427)

* aarch64: constant generation cleanup

Add support for MOVZ and MOVN generation via ISLE.
Handle f32const, f64const, and nop instructions via ISLE.
No long

Remove MachInst::gen_constant (#5427)

* aarch64: constant generation cleanup

Add support for MOVZ and MOVN generation via ISLE.
Handle f32const, f64const, and nop instructions via ISLE.
No longer call Inst::gen_constant from lower.rs.

* riscv64: constant generation cleanup

Handle f32const, f64const, and nop instructions via ISLE.

* s390x: constant generation cleanup

Fix rule priorities for "imm" term.
Only handle 32-bit stack offsets; no longer use load_constant64.

* x64: constant generation cleanup

No longer call Inst::gen_constant from lower.rs or abi.rs.

* Refactor LowerBackend::lower to return InstOutput

No longer write to the per-insn output registers; instead, return
an InstOutput vector of temp registers holding the outputs.

This will allow calling LowerBackend::lower multiple times for
the same instruction, e.g. to rematerialize constants.

When emitting the primary copy of the instruction during lowering,
writing to the per-insn registers is now done in lower_clif_block.

As a result, the ISLE lower_common routine is no longer needed.
In addition, the InsnOutput type and all code related to it
can be removed as well.

* Refactor IsleContext to hold a LowerBackend reference

Remove the "triple", "flags", and "isa_flags" fields that are
copied from LowerBackend to each IsleContext, and instead just
hold a reference to LowerBackend in IsleContext.

This will allow calling LowerBackend::lower from within callbacks
in src/machinst/isle.rs, e.g. to rematerialize constants.

To avoid having to pass LowerBackend references through multiple
functions, eliminate the lower_insn_to_regs subroutines in those
targets that still have them, and just inline into the main
lower routine. This also eliminates lower_inst.rs on aarch64
and riscv64.

Replace all accesses to the removed IsleContext fields by going
through the LowerBackend reference.

* Remove MachInst::gen_constant

This addresses the problem described in issue
https://github.com/bytecodealliance/wasmtime/issues/4426
that targets currently have to duplicate code to emit
constants between the ISLE logic and the gen_constant
callback.

After the various cleanups in earlier patches in this series,
the only remaining user of get_constant is put_value_in_regs
in Lower. This can now be removed, and instead constant
rematerialization can be performed in the put_in_regs ISLE
callback by simply directly calling LowerBackend::lower
on the instruction defining the constant (using a different
output register).

Since the check for egraph mode is now no longer performed in
put_value_in_regs, the Lower::flags member becomes obsolete.

Care needs to be taken that other calls directly to the
Lower::put_value_in_regs routine now handle the fact that
no more rematerialization is performed. All such calls in
target code already historically handle constants themselves.
The remaining call site in the ISLE gen_call_common helper
can be redirected to the ISLE put_in_regs callback.

The existing target implementations of gen_constant are then
unused and can be removed. (In some target there may still
be further opportunities to remove duplication between ISLE
and some local Rust code - this can be left to future patches.)

show more ...

Enable the ssa verifier in debug builds (#5354)

Enable regalloc2's SSA verifier in debug builds to check for any outstanding reuse of virtual registers in def constraints. As fuzzing enables debug_a

Enable the ssa verifier in debug builds (#5354)

Enable regalloc2's SSA verifier in debug builds to check for any outstanding reuse of virtual registers in def constraints. As fuzzing enables debug_assertions, this will enable the SSA verifier when fuzzing as well.

show more ...

Cranelift: log number of CLIF insts/blocks to optimize/lower (#5333)

Cranelift: add debug logs counting how many vcode instructions and blocks we lower to (#5332)

Add fixed_nonallocatable constraints when appropriate (#5253)

Plumb the set of allocatable registers through the OperandCollector and use it validate uses of fixed-nonallocatable registers, like %rs

Add fixed_nonallocatable constraints when appropriate (#5253)

Plumb the set of allocatable registers through the OperandCollector and use it validate uses of fixed-nonallocatable registers, like %rsp on x86_64.

show more ...

egraph-based midend: draw the rest of the owl (productionized). (#4953)

* egraph-based midend: draw the rest of the owl.

* Rename `egg` submodule of cranelift-codegen to `egraph`.

* Apply some

egraph-based midend: draw the rest of the owl (productionized). (#4953)

* egraph-based midend: draw the rest of the owl.

* Rename `egg` submodule of cranelift-codegen to `egraph`.

* Apply some feedback from @jsharp during code walkthrough.

* Remove recursion from find_best_node by doing a single pass.

Rather than recursively computing the lowest-cost node for a given
eclass and memoizing the answer at each eclass node, we can do a single
forward pass; because every eclass node refers only to earlier nodes,
this is sufficient. The behavior may slightly differ from the earlier
behavior because we cannot short-circuit costs to zero once a node is
elaborated; but in practice this should not matter.

* Make elaboration non-recursive.

Use an explicit stack instead (with `ElabStackEntry` entries,
alongside a result stack).

* Make elaboration traversal of the domtree non-recursive/stack-safe.

* Work analysis logic in Cranelift-side egraph glue into a general analysis framework in cranelift-egraph.

* Apply static recursion limit to rule application.

* Fix aarch64 wrt dynamic-vector support -- broken rebase.

* Topo-sort cranelift-egraph before cranelift-codegen in publish script, like the comment instructs me to!

* Fix multi-result call testcase.

* Include `cranelift-egraph` in `PUBLISHED_CRATES`.

* Fix atomic_rmw: not really a load.

* Remove now-unnecessary PartialOrd/Ord derivations.

* Address some code-review comments.

* Review feedback.

* Review feedback.

* No overlap in mid-end rules, because we are defining a multi-constructor.

* rustfmt

* Review feedback.

* Review feedback.

* Review feedback.

* Review feedback.

* Remove redundant `mut`.

* Add comment noting what rules can do.

* Review feedback.

* Clarify comment wording.

* Update `has_memory_fence_semantics`.

* Apply @jameysharp's improved loop-level computation.

Co-authored-by: Jamey Sharp <[email protected]>

* Fix suggestion commit.

* Fix off-by-one in new loop-nest analysis.

* Review feedback.

* Review feedback.

* Review feedback.

* Use `Default`, not `std::default::Default`, as per @fitzgen

Co-authored-by: Nick Fitzgerald <[email protected]>

* Apply @fitzgen's comment elaboration to a doc-comment.

Co-authored-by: Nick Fitzgerald <[email protected]>

* Add stat for hitting the rewrite-depth limit.

* Some code motion in split prelude to make the diff a little clearer wrt `main`.

* Take @jameysharp's suggested `try_into()` usage for blockparam indices.

Co-authored-by: Jamey Sharp <[email protected]>

* Take @jameysharp's suggestion to avoid double-match on load op.

Co-authored-by: Jamey Sharp <[email protected]>

* Fix suggestion (add import).

* Review feedback.

* Fix stack_load handling.

* Remove redundant can_store case.

* Take @jameysharp's suggested improvement to FuncEGraph::build() logic

Co-authored-by: Jamey Sharp <[email protected]>

* Tweaks to FuncEGraph::build() on top of suggestion.

* Take @jameysharp's suggested clarified condition

Co-authored-by: Jamey Sharp <[email protected]>

* Clean up after suggestion (unused variable).

* Fix loop analysis.

* loop level asserts

* Revert constant-space loop analysis -- edge cases were incorrect, so let's go with the simple thing for now.

* Take @jameysharp's suggestion re: result_tys

Co-authored-by: Jamey Sharp <[email protected]>

* Fix up after suggestion

* Take @jameysharp's suggestion to use fold rather than reduce

Co-authored-by: Jamey Sharp <[email protected]>

* Fixup after suggestion

* Take @jameysharp's suggestion to remove elaborate_eclass_use's return value.

* Clarifying comment in terminator insts.

Co-authored-by: Jamey Sharp <[email protected]>
Co-authored-by: Nick Fitzgerald <[email protected]>

show more ...

Cranelift: Deduplicate ABI signatures during lowering (#4829)

* Cranelift: Deduplicate ABI signatures during lowering

This commit creates the `SigSet` type which interns and deduplicates the ABI

Cranelift: Deduplicate ABI signatures during lowering (#4829)

* Cranelift: Deduplicate ABI signatures during lowering

This commit creates the `SigSet` type which interns and deduplicates the ABI
signatures that we create from `ir::Signature`s. The ABI signatures are now
referred to indirectly via a `Sig` (which is a `cranelift_entity` ID), and we
pass around a `SigSet` to anything that needs to access the actual underlying
`SigData` (which is what `ABISig` used to be).

I had to change a couple methods to return a `SmallInstVec` instead of emitting
directly to work around what would otherwise be shared and exclusive borrows of
the lowering context overlapping. I don't expect any of these to heap allocate
in practice.

This does not remove the often-unnecessary allocations caused by
`ensure_struct_return_ptr_is_returned`. That is left for follow up work.

This also opens the door for further shuffling of signature data into more
efficient representations in the future, now that we have `SigSet` to store it
all in one place and it is threaded through all the code. We could potentially
move each signature's parameter and return vectors into one big vector shared
between all signatures, for example, which could cut down on allocations and
shrink the size of `SigData` since those `SmallVec`s have pretty large inline
capacity.

Overall, this refactoring gives a 1-7% speedup for compilation on
`pulldown-cmark`:

```
compilation :: cycles :: benchmarks/pulldown-cmark/benchmark.wasm

Δ = 8754213.66 ± 7526266.23 (confidence = 99%)

dedupe.so is 1.01x to 1.07x faster than main.so!

[191003295 234620642.20 280597986] dedupe.so
[197626699 243374855.86 321816763] main.so

compilation :: cycles :: benchmarks/bz2/benchmark.wasm

No difference in performance.

[170406200 194299792.68 253001201] dedupe.so
[172071888 193230743.11 223608329] main.so

compilation :: cycles :: benchmarks/spidermonkey/benchmark.wasm

No difference in performance.

[3870997347 4437735062.59 5216007266] dedupe.so
[4019924063 4424595349.24 4965088931] main.so
```

* Use full path instead of import to avoid warnings in some build configurations

Warnings will then cause CI to fail.

* Move `SigSet` into `VCode`

show more ...

Cranelift: Remove `ABICallee` trait (#4701)

* Cranelift: Remove `ABICallee` trait

It has only one implementation: the `ABICalleeImpl` struct. By using that
directly we can avoid unnecessary laye

Cranelift: Remove `ABICallee` trait (#4701)

* Cranelift: Remove `ABICallee` trait

It has only one implementation: the `ABICalleeImpl` struct. By using that
directly we can avoid unnecessary layers of generics and abstractions as well as
a couple `Box`es that were previously putting the single implementation into a
`Box<dyn>`.

* Cranelift: Rename `ABICalleeImpl` to `AbiCallee`

* Fix comments as per review

* Rename `AbiCallee` to `Callee`

show more ...

cranelift: Introduce a feature to enable `trace` logs (#4484)

* Don't use `log::trace` directly but a feature-enabled `trace` macro
* Don't emit disassembly based on the log level