cranelift-codegen: rename most uses of std to core and alloc (#12237)

* rename most std uses to core and alloc

* cargo fmt

Cranelift: add get_exception_handler_address. (#11629)

* Cranelift: add get_exception_handler_address.

This is designed to enable applications such as #11592 that use
alternative unwinding mechanis

Cranelift: add get_exception_handler_address. (#11629)

* Cranelift: add get_exception_handler_address.

This is designed to enable applications such as #11592 that use
alternative unwinding mechanisms that may not necessarily want to walk a
stack and look up exception tables. The idea is that whenever it would
be valid to resume to an exception handler that is active on the stack,
we can provide the same PC as a first-class runtime value that would be
found in the exception table for the given handler edge. A "custom"
resume step can then use this PC as a resume-point as long as it follows
the relevant exception ABI (i.e.: restore SP, FP, any other saved
registers that the exception ABI specifies, and provide appropriate
payload value(s)).

Handlers are associated with edges out of `try_call`s (or
`try_call_indirect`s); and edges specifically, not blocks, because there
could be multiple out-edges to one block. The instruction thus takes the
block that contains the try-call and an immediate that indexes its
exceptional edges.

This CLIF instruction required a bit of infrastructure to (i) allow
naming raw blocks, not just block calls, as instruction arguments, and
(ii) allow getting the MachLabel for any other lowered block during
lowering. But given that, the lowerings themselves are straightforward
uses of MachBuffer labels to fix-up PC-relative address-loading
instructions (e.g., `LEA` or `ADR` or `AUIPC`+`ADDI`).

* Review feedback.

* Review feedback: more tests.

show more ...

Cranelift: support dynamic contexts in exception-handler lists. (#11321)

In #11285, we realized that Wasm semantics require us to match on
dynamic instances of exception tags, rather than static tag

Cranelift: support dynamic contexts in exception-handler lists. (#11321)

In #11285, we realized that Wasm semantics require us to match on
dynamic instances of exception tags, rather than static tag types. This
fundamentally requires the unwinder to be able to resolve the current
Wasm instance for each Wasm frame on the stack that has any handlers,
and our frame format does not provide this today.

We discussed many options, some of which solve the more general problem
(Wasm vmctx for any frame), but ultimately landed on a notion of
"dynamic context for evaluating tags", specific to Cranelift's
exception-catch metadata; and storing that context and carrying it
through to a place that is named in the unwind metadata. The reasoning
is fairly straightforward: we cannot afford a more general approach that
stores vmctx in every frame (I measured this at 20% overhead for a
recursive-Fibonacci benchmark that is call-intensive); and inlining
means that we may have *multiple* contexts at any given program point,
each associated with a different slice of the handler tags; so we need a
mechanism that, *just for a try-call*, intersperses contexts with tags
(or puts a context on each tag) and stores these somewhere that the
exception-unwind ABI doesn't clobber (e.g., on the stack).

This PR implements "option 4" from that issue, namely, *dynamic
exception contexts*. The idea is that this is the dual to exception
payload: while payload lets the unwinder communicate state *to* the
catching code, context lets the unwinder take state *from* the catching
code that lets it decide whether the tag is a match. Because of
inlining, we need to either associate (optional) context with every tag,
or intersperse context-updates with handler tags. I've opted for the
latter for efficiency at the CLIF level (in most cases there will be
multiple tags per context), though they are isomorphic.

The new tag-matching semantics are: when walking up the stack, upon
reaching a `try_call`, evaluate catch-clauses in listed order. A
`context` clause sets the current context. A `tagN: block(...)` clause
attempts to match the throwing exception against `tagN`, *evaluated in
the current context*, and branches to the named block if it matches. A
`default: block(...)` always branches to the named block.

Note that this lets us assume less about tags than before, and this
particularly manifests in the changes to the inliner. Whereas before,
`tagN` is `tagN` and an inner handler for that tag shadows an outer
handler (that is, tags always alias if identical indices); and whereas
before, `tagN` is not `tagM` and so we can order the tags arbitrarily
(that is, tags never alias if non-identical indices); now any two static
tag indices may or may not alias depending on the dynamic context of
each. Or, even in the same context, two may alias, because we leave the
match-predicate as an unspecified (user-chosen) algorithm during
unwinding. (This mirrors the reality that, for example, a Wasm instance
may import two tags, and dynamically these tags may be equal or
different at runtime, even instantiation-to-instantiation.) Cranelift's
only job is to faithfully carry the list of contexts and tags through to
the compiled-code metadata; and to ensure that they remain in the order
they were specified in the CLIF.

This PR introduces the Cranelift-level feature, and it will be used in
a subsequent PR that introduces Wasm exception handling. Because of
that, I've opted not to update the clif-utils runtest "runtime" to read
out contexts and do something with them -- we will have plenty of test
coverage via a bunch of Wasm tests for corner cases such as the above.
This PR does include filetests that show that contexts are carried
through to spillslots and those appear in the metadata.

Fixes #11285.

show more ...

Fix `uadd_overflow_trap` in interpreter (#11258)

Cranelift: Make `ir::{Constant,Immediate}` considered entities (#11207)

* Cranelift: Make `ir::{Constant,Immediate}` considered entities

They reference data in out-of-line pools rather than storing

Cranelift: Make `ir::{Constant,Immediate}` considered entities (#11207)

* Cranelift: Make `ir::{Constant,Immediate}` considered entities

They reference data in out-of-line pools rather than storing their data inline
in the instruction, and when an instruction containing them is moved from one
`ir::Function` to another, they need their indices updated
accordingly. Therefore, they really are entities rather than immediates.

This recategorization means that they will now be properly mapped in
`ir::InstructionData::map` calls.

* fix tests

show more ...

Cranelift: Generate an `InstructionData::map` method (#11176)

* Cranelift: Generate an `InstructionData::map` method

This allows you to map some functions, described by the given
`InstructionMapper

Cranelift: Generate an `InstructionData::map` method (#11176)

* Cranelift: Generate an `InstructionData::map` method

This allows you to map some functions, described by the given
`InstructionMapper`, over each of the entitities in an instruction, producing a
new `InstructionData`.

I intend to use this as part of an inliner API for Cranelift that I am
developing as part of prototyping [Wasmtime's compile-time
builtins](https://github.com/bytecodealliance/rfcs/pull/43).

* cargo fmt

* fix clippy

show more ...

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtim

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtime to the 2024 Edition to keep
up-to-date with Rust idioms and access many of the edition features
exclusive to the 2024 edition.

prtest:full

* Reformat with the 2024 edition

show more ...

Cranelift: initial try_call / try_call_indirect (exception) support. (#10510)

* Cranelift: initial try_call / try_call_indirect (exception) support.

This PR adds `try_call` and `try_call_indirect`

Cranelift: initial try_call / try_call_indirect (exception) support. (#10510)

* Cranelift: initial try_call / try_call_indirect (exception) support.

This PR adds `try_call` and `try_call_indirect` instructions, and
lowerings on four of five ISAs (x86-64, aarch64, riscv64, pulley; s390x
has its own non-shared ABI code that will need separate work).

It extends CLIF to support these instructions as new kinds of branches,
and extends block-calls to accept `retN` and `exnN` block-call args that
carry the normal return values or exception payloads (respectively) into
the appropriate successor blocks.

It wires up the "normal return path" so that it continues to work.
It updates the ABI so that unwinding is possible without an initial
register state at throw: specifically, as per our RFC, all registers are
clobbered. It also includes metadata in the `MachBuffer` that describes
exception-catch destinations. However, no unwinder exists to interpret
these catch-destinations yet, so they are untested.

* Add try_call_indirect lowering as well.

show more ...

Add some more context to the CLIF verifier (#10410)

Helps when debugging verifier failures.

Enable some more 2024 migration lints (#9962)

Most don't produce many warnings except for `rust-2024-incompatible-pat`
which required removal of a number of `ref` and `ref mut` keywords
throughout t

Enable some more 2024 migration lints (#9962)

Most don't produce many warnings except for `rust-2024-incompatible-pat`
which required removal of a number of `ref` and `ref mut` keywords
throughout the workspace.

show more ...

Update nightly in CI to latest (#9195)

Additionally address some new warnings that are cropping up throughout
the codebase.

Remove reference types (`r32` and `r64`) from Cranelift (#9164)

* Remove reference types (`r32` and `r64`) from Cranelift

* restore fuzz regression test

Cranelift: add stack_switch CLIF instruction (#9078)

* stack_switch instruction

* Update cranelift/codegen/src/isa/x64/pcc.rs

Co-authored-by: Nick Fitzgerald <[email protected]>

* cargo fmt

* on

Cranelift: add stack_switch CLIF instruction (#9078)

* stack_switch instruction

* Update cranelift/codegen/src/isa/x64/pcc.rs

Co-authored-by: Nick Fitzgerald <[email protected]>

* cargo fmt

* only lower on linux

* give stack_switch the call() side effect

* add function in filetest doing switching only

* Extend documentation of new instruction

* better comments on how we handle the payloads

* Revert "only lower on linux"

This reverts commit 2af10f944186629de1615aa0ed999b7f49d13132.

* Add StackSwitchModel, use to compile stack_switch

* turn stack_switch_model into partial constructor

---------

Co-authored-by: Nick Fitzgerald <[email protected]>

show more ...

Enforce `uninlined_format_args` for the workspace (#9065)

* Enforce `uninlined_format_args` for the workspace

* fix: failing `Monolith Checks` job

* fix: formatting

Cranelift: Add an `is_safepoint` predicate to `Opcode` (#9066)

Cranelift: Stop sign-extending `Imm64` immediates in builder methods (#9046)

* Cranelift: Stop sign-extending `Imm64` immediates in builder methods

Originally, we sign-extended `Imm64` immediates f

Cranelift: Stop sign-extending `Imm64` immediates in builder methods (#9046)

* Cranelift: Stop sign-extending `Imm64` immediates in builder methods

Originally, we sign-extended `Imm64` immediates from their controlling type
var's width to `i64` in the builder methods for `BinaryImm64` and a couple other
instruction formats: https://github.com/bytecodealliance/wasmtime/pull/1687

At some point, we decided that the unused bits in the `Imm64` when the
controlling type var's width is less than 64 should be zero. And we started
asserting that in various places, for example:
https://github.com/bytecodealliance/wasmtime/blob/87817f38a128caa76eaa6a3c3c8ceac81a329a3e/cranelift/codegen/src/machinst/lower.rs#L1237-L1243

However, we never removed or updated the sign-extension code in the builder
methods. This commit switches the builder methods over to masking the `Imm64` to
just the valid bits, effectively doing a zero extend from the controlling type
var's width instead of a sign extend.

To avoid too much churn in tests, I made `display_inst` print the
sign-extended-from-the-controlling-type-variable's-width value of `Imm64`
immediates. I also made the `Display` implementation for `Imm64` always print in
decimal form with a `-` for negative numbers, rather than the hex it would
previously print for large negative numbers, so that `iconst.i32 0x8000_0000`
doesn't display as `iconst.i32 0xffff_ffff_8000_0000`, which is otherwise pretty
confusing.

* Rename condition

* return new immediates instead of mutating

* Update some test expectations

show more ...

Initial `f16` and `f128` support (#8860)

Introduce the `cranelift-bitset` crate; use it for stack maps in both Cranelift and Wasmtime (#8826)

* Introduce the `cranelift-bitset` crate

The eventual goal is to deduplicate bitset types betwee

Introduce the `cranelift-bitset` crate; use it for stack maps in both Cranelift and Wasmtime (#8826)

* Introduce the `cranelift-bitset` crate

The eventual goal is to deduplicate bitset types between Cranelift and Wasmtime,
especially their use in stack maps.

* Use the `cranelift-bitset` crate inside both Cranelift and Wasmtime

Mostly for stack maps, also for a variety of other random things where
`cranelift_codegen::bitset::BitSet` was previously used.

* Fix stack maps unit test in cranelift-codegen

* Uncomment `no_std` declaration

* Fix `CompountBitSet::reserve` method

* Fix `CompoundBitSet::insert` method

* Keep track of the max in a `CompoundBitSet`

Makes a bunch of other stuff easier, and will be needed for replacing
`cranelift_entity::EntitySet`'s bitset with this thing anyways.

* Add missing parens

* Fix a bug around insert and reserve

* Implement `with_capacity` in terms of `new` and `reserve`

* Rename `reserve` to `ensure_capacity`

show more ...

Cranelift: Remove resumable traps (#8809)

These were originally a SpiderMonkey-ism and have been unused ever
since. It was introduced for GC integration, where the runtime could do
something to make

Cranelift: Remove resumable traps (#8809)

These were originally a SpiderMonkey-ism and have been unused ever
since. It was introduced for GC integration, where the runtime could do
something to make Cranelift code hit a trap and pause for a GC and then resume
execution once GC completed. But it is unclear that, as implemented, this is
actually a useful mechanism for doing that (compared to, say, loading from some
Well Known page and the GC protecting that page and catching signals to
interrupt the mutator, or simply branching and doing a libcall). And if someone
has that particular use case in the future (Wasmtime and its GC integration
doesn't need exactly this) then we can design something for what is actually
needed at that time, instead of carrying this cruft forward forever.

show more ...

Update nightly used in CI and fix warnings (#8416)

Fixes some warnings that nightly Rust has started emitting.

cranelift: New InstructionData::map_values helper (#8231)

This is like the DataFlowGraph::map_inst_values method, but that method
mutably borrows `self`, so it goes to a fair bit of trouble to drop

cranelift: New InstructionData::map_values helper (#8231)

This is like the DataFlowGraph::map_inst_values method, but that method
mutably borrows `self`, so it goes to a fair bit of trouble to drop its
borrows around callbacks so the mutable borrow can be used in the
callback too.

This new helper only actually needs to borrow two public fields from the
DFG: `value_lists` and `jump_tables`. Therefore it's possible to use
other fields in the callback as long as the compiler can see all the
fields being used in the same scope. That's good enough for everywhere
we were using this pattern, so we can simplify this way.

The case which motivated this change isn't shown here: It isn't possible
to call `dfg.map_inst_values` on every instruction in `dfg.insts`,
because the borrow on `dfg.insts` prevents taking a mutable borrow on
`dfg`. But we can call this new helper in that case.

show more ...

Decouple `serde` from its `derive` crate (#6917)

By not activating the `derive` feature on `serde`, the compilation speed
can be improved by a lot. This is because `serde` can then compile in
parall

Decouple `serde` from its `derive` crate (#6917)

By not activating the `derive` feature on `serde`, the compilation speed
can be improved by a lot. This is because `serde` can then compile in
parallel to `serde_derive`, allowing it to finish compilation possibly
even before `serde_derive`, unblocking all the crates waiting for
`serde` to start compiling much sooner.

As it turns out the main deciding factor for how long the compile time of a
project is, is primarly determined by the depth of dependencies rather
than the width. In other words, a crate's compile times aren't affected
by how many crates it depends on, but rather by the longest chain of
dependencies that it needs to wait on. In many cases `serde` is part of
that long chain, as it is part of a long chain if the `derive` feature
is active:

`proc-macro2` compile build script > `proc-macro2` run build script >
`proc-macro2` > `quote` > `syn` > `serde_derive` > `serde` >
`serde_json` (or any crate that depends on serde)

By decoupling it from `serde_derive`, the chain is shortened and compile
times get much better.

Check this issue for a deeper elaboration:
https://github.com/serde-rs/serde/issues/2584

For `wasmtime` I'm seeing a reduction from 24.75s to 22.45s when
compiling in `release` mode. This is because wasmtime through `gimli`
has a dependency on `indexmap` which can only start compiling when
`serde` is finished, which you want to happen as early as possible so
some of wasmtime's dependencies can start compiling.

To measure the full effect, the dependencies can't by themselves
activate the `derive` feature. I've upstreamed a patch for
`fxprof-processed-profile` which was the only dependency that activated
it for `wasmtime` (not yet published to crates.io). `wasmtime-cli` and
co. may need patches for their dependencies to see a similar
improvement.

show more ...

add result type assertion (#6184)

Revert "simple_gvn: recognize commutative operators (#6135)" (#6142)

This reverts commit c85bf27ff8cdaffec26a25d46b4343a06e00027a.

simple_gvn: recognize commutative operators (#6135)

* simple_gvn: recognize commutative operators

Normalize instructions with commutative opcodes by sorting the arguments. This
means instructions l

simple_gvn: recognize commutative operators (#6135)

* simple_gvn: recognize commutative operators

Normalize instructions with commutative opcodes by sorting the arguments. This
means instructions like `iadd v0, v1` and `iadd v1, v0` will be considered
identical by GVN and deduplicated.

* Remove `UsubSat` and `SsubSat` from `is_commutative`

They are not actually commutative

* Remove `TODO`s

* Move InstructionData normalization into helper fn

* Add normalization of commutative instructions in the epgrah implementation

* Handle reflexive icmp/fcmps in GVN

* Change formatting of `normalize_in_place`

* suggestions from code review

show more ...