Changes in lower.rs

763622c3 - Preserve `try_call[_indirect]` stack maps during lowering (#12934)

Wed, 01 Apr 2026 23:08:47 +0000

Preserve `try_call[_indirect]` stack maps during lowering (#12934)* Preserve `try_call[_indirect]` stack maps during loweringBranch instructions are skipped in the main lowering loop, which means the stackmap forwarding code is never reached for them. The branch lowering path didn'tforward stack maps either. This was fine because branch instructions couldn'tpreviously ever be safepoints. However, with the introduction of `try_call` and`try_call_indirect`, we now have instructions that are both safepoints andbranches.This caused GC references live across `try_call[_indirect]` instructions to notbe traced during garbage collection, leading to use-after-free within the GCheap sandbox when the collector swept those untraced-but-still-live objects.The fix adds stack map forwarding after branch lowering, mirroring theexisting logic for non-branch instructions.Fixes bytecodealliance/wasmtime#11753.* update disas test List of files: /wasmtime-44.0.1/cranelift/codegen/src/machinst/lower.rs

2f7dbd61 - PCC: remove proof-carrying code (for now?). (#12800)

Tue, 31 Mar 2026 04:36:33 +0000

PCC: remove proof-carrying code (for now?). (#12800)In late 2023, we built out an experimental feature calledProof-Carrying Code (PCC), where we attached "facts" to values in theCLIF IR and built verification of these facts after lowering tomachine instructions. We also added "memory types" describing layoutof memory and a "checked" flag on memory operations such that we couldverify that any checked memory operation accessed valid memory (asdefined by memory types attached to pointer values viafacts). Wasmtime's Cranelift backend then put appropriate memory typesand facts in its IR such that all accesses to memory (aspirationally)could be checked, taking the whole mid-end and lowering backend ofCranelift out of the trusted core that enforces SFI.This basically worked, at the time, for static memories; but never fordynamic memories, and then work on the feature lostprioritization (aka I had to work on other things) and I wasn't ableto complete it and put it in fuzzing/enable it as a production option.Unfortunately since then it has bit-rotted significantly -- as we addnew backend optimizations and instruction lowerings we haven't keptthe PCC framework up to date.Inspired by the discussion in #12497 I think it's time to deleteit (hopefully just "for now"?) unless/until we can build it again. Andwhen we do that, we should probably get it to the point of validatingrobust operation on all combinations of memory configurations beforemerging. (That implies a big experiment branch rather than a bunch ofeager PRs in-tree, but so it goes.) I still believe it is possible tobuild this (and I have ideas on how to do it!) but not right now. List of files: /wasmtime-44.0.1/cranelift/codegen/src/machinst/lower.rs

326599f3 - Only emit value label aliases for lowered instructions (#12779)

Mon, 16 Mar 2026 14:48:23 +0000

Only emit value label aliases for lowered instructions (#12779)If we emit a value label alias for an instruction that isn't lowered,then that signals that the value has been optimised out. However, sinceit is an alias we know that the value also exists in an earlier vreg, sowe should skip the alias and use that instead.This situation occurs often for memory indexes on AArch64. We translatememory stores into instructions such as: v8 = iconst.i32 42 v9 = uextend.i64 v6 v10 = load.i64 notrap aligned readonly can_move checked v0+56 v11 = iadd v10, v9 v12 = iconst.i64 20 v13 = iadd v11, v12 ; v12 = 20 store little heap v8, v13 ; v8 = 42Here, v6 is a memory index (which has a label) and v9 is anextension of the memory index (which has a label alias, added bycast_index_to_pointer_ty()). This is lowered to: 40c: 52800540 mov w0, #0x2a // #42 410: f9401c41 ldr x1, [x2, #56] 414: 91005021 add x1, x1, #0x14 418: b8384820 str w0, [x1, w24, uxtw]The uextend has been folded into the str, so v9 has been optimisedout. But v6 is still present in w24, so the debuginfo should use thatinstead.This fixes the following tests for AArch64: native_debug::lldb::dwarf_cold_block native_debug::lldb::dwarf_fib_wasm native_debug::lldb::dwarf_fib_wasm_dwarf5 native_debug::lldb::dwarf_fib_wasm_split4 native_debug::lldb::dwarf_fission native_debug::lldb::dwarf_fraction_norm native_debug::lldb::dwarf_imported_memory native_debug::lldb::dwarf_shared_memory native_debug::lldb::dwarf_simple native_debug::lldb::dwarf_spilled_frame_base List of files: /wasmtime-44.0.1/cranelift/codegen/src/machinst/lower.rs

76911c29 - Partial support for no_std in cranelift_codegen (#12222)

Wed, 07 Jan 2026 16:41:32 +0000

Partial support for no_std in cranelift_codegen (#12222)* Move most things from std to core and alloc* Port assembler_x64 to no_std* before adding prelude to each file* Most of the files now work with no_std* update isle to use alloc and core* some instances shouldn't have been renamed, fixes cargo test* add cranelift-assembler-x64 (no_std) to CI* fix codegen_meta, missed one spot with std::slice* automatically remove prelude with cargo fix* update isle changes* update assembler changes* update assembler changes* use latest codegen changes + fix FxHash problem* add imports* fix floating issues with libm* remove unused import* temporarily remove OnceLock* add no_std arm support and add it into CI* Move most things from std to core and alloc* Port assembler_x64 to no_std* before adding prelude to each file* Most of the files now work with no_std* update isle to use alloc and core* some instances shouldn't have been renamed, fixes cargo test* add cranelift-assembler-x64 (no_std) to CI* automatically remove prelude with cargo fix* update isle changes* update assembler changes* update assembler changes* use latest codegen changes + fix FxHash problem* add imports* fix floating issues with libm* remove unused import* temporarily remove OnceLock* add no_std arm support and add it into CI* Move most things from std to core and alloc* Port assembler_x64 to no_std* before adding prelude to each file* Most of the files now work with no_std* update isle to use alloc and core* add cranelift-assembler-x64 (no_std) to CI* automatically remove prelude with cargo fix* update isle changes* update assembler changes* use latest codegen changes + fix FxHash problem* add imports* fix floating issues with libm* temporarily remove OnceLock* add no_std arm support and add it into CI* revert Cargo.toml formating* remove prelude and fix cargo.toml* cargo fmt* remove empty lines* bad renames* macro_use only on no_std* revert OnceLock change* only use stable libm features* update regalloc2* update comment* use continue instead* Update vets---------Co-authored-by: Alex Crichton List of files: /wasmtime-44.0.1/cranelift/codegen/src/machinst/lower.rs

0889323a - cranelift-codegen: rename most uses of std to core and alloc (#12237)

Sat, 03 Jan 2026 00:54:48 +0000

cranelift-codegen: rename most uses of std to core and alloc (#12237)* rename most std uses to core and alloc* cargo fmt List of files: /wasmtime-44.0.1/cranelift/codegen/src/machinst/lower.rs

87ed3b60 - Cranelift: make all non-tail, non-indirect calls patchable, and rename patchable ABI to `preserve_all`. (#12160)

Mon, 15 Dec 2025 23:29:06 +0000

Cranelift: make all non-tail, non-indirect calls patchable, and rename patchable ABI to `preserve_all`. (#12160)* Cranelift: make all non-tail, non-indirect calls patchable, and rename patchable ABI to `preserve_all`.As discussed in this week's Cranelift meeting, we've discovered a needto generalize the `patchable_call` mechanism and corresponding`patchable` ABI slightly. In particular, we will need patchable`try_call` callsites as well in order to allow breakpoint handlers tothrow exceptions (desirable functionality eventually) and have this workin the presence of inlining. Also, it's just a nice generalization tosay that patchability is an orthogonal dimension to the call ABI and theother restrictions we initially imposed, and works as long as the basicrequirement (no return values) is met.This also renames the `patchable` ABI to `preserve_all`, to make itclear that its purpose is actually orthogonal, and it can be usedindependently of patchable callsites. It also deletes the `cold` ABI,which never actually did anything and is misleading in the presence ofan actual cold-ish (subzero temperature, actually) ABI like`preserve_all`.* Review feedback. List of files: /wasmtime-44.0.1/cranelift/codegen/src/machinst/lower.rs

a3d6e407 - Cranelift: add debug tag infrastructure. (#11768)

Mon, 06 Oct 2025 19:38:03 +0000

Cranelift: add debug tag infrastructure. (#11768)* Cranelift: add debug tag infrastructure.This PR adds *debug tags*, a kind of metadata that can attach to CLIFinstructions and be lowered to VCode instructions and as metadata onthe produced compiled code. It also adds opaque descriptor blobscarried with stackslots. Together, these two features allow decoratingIR with first-class debug instrumentation that is properly preservedby the compiler, including across optimizations andinlining. (Wasmtime's use of these features will come in followupPRs.)The key idea of a "debug tag" is to allow the Cranelift embedder toexpress whatever information it needs to, in a format that is opaqueto Cranelift itself, except for the parts that need translation duringlowering. In particular, the `DebugTag::StackSlot` variant getstranslated to a physical offset into the stackframe in the compiledmetadata output. So, for example, the embedder can emit a tagreferring to a stackslot, and another describing an offset in thatstackslot.The debug tags exist as a *sequence* on any given instruction; themeaning of the sequence is known only to the embedder, *except* thatduring inlining, the tags for the inlining call instruction areprepended to the tags of inlined instructions. In this way, acanonical use-case of tags as describing original source-languageframes can preserve the source-language view even when multiplefunctions are inlined into one.The descriptor on a stackslot may look a little odd at first, but itspurpose is to allow serializing some description ofstackslot-contained runtime user-program data, in a way that is firmlyattached to the stackslot. In particular, in the face of inlining,this descriptor is copied into the inlining (parent) function from theinlined function when the stackslot entity is copied; no othermetadata outside Cranelift needs to track the identity of stackslotsand know about that motion. This fits nicely with the ability of tagsto refer to stackslots; together, the embedder can annotateinstructions as having certain state in stackslots, and describe theformat of that state per stackslot.This infrastructure is tested with some compile-tests now;testing of the interpretation of the metadata output will come withend-to-end debug instrumentation tests in a followup PR.* Review feedback: add back sequence points and enforce tags only on sequence points or calls.* Use Vecs for debug metadata in MachBuffer to avoid SmallVec size penalty in not-used case.* Review feedback: switch from inlined stackslot descriptor blobs to u64 keys. List of files: /wasmtime-44.0.1/cranelift/codegen/src/machinst/lower.rs

4c01ee2f - Cranelift: add get_exception_handler_address. (#11629)

Fri, 05 Sep 2025 22:41:46 +0000

Cranelift: add get_exception_handler_address. (#11629)* Cranelift: add get_exception_handler_address.This is designed to enable applications such as #11592 that usealternative unwinding mechanisms that may not necessarily want to walk astack and look up exception tables. The idea is that whenever it wouldbe valid to resume to an exception handler that is active on the stack,we can provide the same PC as a first-class runtime value that would befound in the exception table for the given handler edge. A "custom"resume step can then use this PC as a resume-point as long as it followsthe relevant exception ABI (i.e.: restore SP, FP, any other savedregisters that the exception ABI specifies, and provide appropriatepayload value(s)).Handlers are associated with edges out of `try_call`s (or`try_call_indirect`s); and edges specifically, not blocks, because therecould be multiple out-edges to one block. The instruction thus takes theblock that contains the try-call and an immediate that indexes itsexceptional edges.This CLIF instruction required a bit of infrastructure to (i) allownaming raw blocks, not just block calls, as instruction arguments, and(ii) allow getting the MachLabel for any other lowered block duringlowering. But given that, the lowerings themselves are straightforwarduses of MachBuffer labels to fix-up PC-relative address-loadinginstructions (e.g., `LEA` or `ADR` or `AUIPC`+`ADDI`).* Review feedback.* Review feedback: more tests. List of files: /wasmtime-44.0.1/cranelift/codegen/src/machinst/lower.rs

3c3fb35b - Clarify docs around exceptions in Cranelift (#11601)

Wed, 03 Sep 2025 20:37:59 +0000

Clarify docs around exceptions in Cranelift (#11601)* Clarify docs around exceptions in CraneliftThis is a result of today's Cranelift meeting with some of my questionsaround the ABI bits here and there. Notably:* Cranelift is audited and now documented to always consider the callee calling convention in `try_call`, disregarding the caller calling convention.* Wasmtime's exception throw now explicitly names the `tailcc` in the name to indicate that it's only compatible with the tail calling convention.* The verifier test for exceptions is expanded with a few more cases here and there that I could think of.* Backends now assert that the size of the calling-convention list of payload types is the same as the size of the list of registers the backend places results into.* Fix a typo List of files: /wasmtime-44.0.1/cranelift/codegen/src/machinst/lower.rs

a4f8f8f4 - Add lowering rules for `{add,sub,or,and} mem, imm` on x64 (#11043)

Mon, 16 Jun 2025 19:50:33 +0000

Add lowering rules for `{add,sub,or,and} mem, imm` on x64 (#11043)* Add lowering rules for `{add,sub,or,and} mem, imm` on x64* Fold ` reg, imm` lowering rules into existing `x64_op_mem` helpers* Remove duplicate decl that got added elsewhere in a different commit on main List of files: /wasmtime-44.0.1/cranelift/codegen/src/machinst/lower.rs

90ac295e - Update Wasmtime to the 2024 Rust Edition (#10806)

Mon, 19 May 2025 16:40:55 +0000

Update Wasmtime to the 2024 Rust Edition (#10806)* Update Wasmtime to the 2024 Rust EditionNow that our MSRV supports the 2024 edition it's possible to make thisswitch. This commit moves Wasmtime to the 2024 Edition to keepup-to-date with Rust idioms and access many of the edition featuresexclusive to the 2024 edition.prtest:full* Reformat with the 2024 edition List of files: /wasmtime-44.0.1/cranelift/codegen/src/machinst/lower.rs

5ded0f4e - Refactor call ABI implementation (#10722)

Tue, 06 May 2025 15:53:24 +0000

Refactor call ABI implementation (#10722)This refactors implementation of call ABI handling across architectureswith the goal of bringing s390x in line with other platforms.The main idea is to- handle main call instruction selection and generation in ISLE (like s390x but unlike other platforms today)- handle argument setup mostly outside of ISLE (like other platforms but unlike s390x today)- handle return value processing as part of the call instructio (like all platforms today)All platforms now emit the main call instruction directly from ISLE,which e.g. handles selection of the correct ISA instruction dependingon the call destination. This ISLE code calls out to helper routinesto handle argument and return value processing. These helpers aremostly common code and provided by the Callee and/or Lower layers,with some platform-specific additions via ISLE Context routines.The old CallSite abstraction is no longer needed; most of thedifferences between call and return_call handling disappear.(There is still a common-code CallInfo vs. a platform-specifcReturnCallInfo. At this point, it should be relatively straight-forward to make CallInfo platform-specific as well if desired,but this is not done here.)Some ISLE infrastructure for iterators / loops, which was onlyever used by the s390x argument processing code, has been removed.s390x now closely matches all other platforms, with only a fewspecial cases (slightly different tail-call ABI requires somedifferences in stack offset computations; we still need tohandle vector lane swaps for cross-ABI calls), which shouldsimplify future maintenance. List of files: /wasmtime-44.0.1/cranelift/codegen/src/machinst/lower.rs

5b63c874 - [DI] Fix live range tracking off-by-one confusions (#10570)

Tue, 15 Apr 2025 20:15:48 +0000

[DI] Fix live range tracking off-by-one confusions (#10570)* Dump blocks in the VL table* Add a test* Work around #10572 in tests* [DI] Fix live range tracking off-by-one confusionsHow things used to work w.r.t. instruction indices (IIs):1) In lowering: - Reversed order: IIs represented "before IP"s. - Block args were defined one instruction too late, but this issue was masked due to how RA allocates, at least in simple examples. - Execution order: IIs represented "after IP"s.2) In RA: - IIs represented "before IP"s. - Notice the mismatch.3) In emit: - RA directions w.r.t. the explicit ProgPoint positions were not respected and always treated as "after".How things work after this change:1) In lowering: - Reversed order: IIs represent "after IP"s. - Execution order: IIs represent "before IP"s.2) In RA: - No change; mismatch fixed.3) In emit: - ProgPoint positions now respected.This fixes various "silent bad debug info" issues. List of files: /wasmtime-44.0.1/cranelift/codegen/src/machinst/lower.rs

7bf31723 - Cranelift: simplify some side-effectful instructions in ISLE (#10524)

Tue, 08 Apr 2025 17:57:03 +0000

Cranelift: simplify some side-effectful instructions in ISLE (#10524)* Cranelift: simplify some side-effectful instructions in ISLEThis commit adds a new top-level ISLE entrypoint specifically for instructionsin the side-effectful skeleton: `simplify_skeleton`. While these rewrites areprocessed during the egraph pass, values from skeleton instructions still do notget inserted into the egraph. Indeed, `simplify_skeleton` operateson *instructions* rather than *values* because we do not represent side effectsas values; values do not have side effects in CLIF, instructions do. Therefore,rather than doing a whole dynamic-programming style extraction of the bestcandidate simplification like we do with the egraph, we take an eager and greedyapproach.Furthermore, `simplify_skeleton` is limited only to skeleton instructions thatdo not involve control-flow or terminators right now. This is because changingthe control-flow graph can change whether a use is dominated by a def or not,and we do not currently have the machinery to track and fix up invalidateduses. Addressing this is left for future commits.* fix `MIN / -1` cprop and add negative tests for things simplify_skeleton cannot handle yet List of files: /wasmtime-44.0.1/cranelift/codegen/src/machinst/lower.rs

94ec88ea - Cranelift: initial try_call / try_call_indirect (exception) support. (#10510)

Tue, 08 Apr 2025 00:02:16 +0000

Cranelift: initial try_call / try_call_indirect (exception) support. (#10510)* Cranelift: initial try_call / try_call_indirect (exception) support.This PR adds `try_call` and `try_call_indirect` instructions, andlowerings on four of five ISAs (x86-64, aarch64, riscv64, pulley; s390xhas its own non-shared ABI code that will need separate work).It extends CLIF to support these instructions as new kinds of branches,and extends block-calls to accept `retN` and `exnN` block-call args thatcarry the normal return values or exception payloads (respectively) intothe appropriate successor blocks.It wires up the "normal return path" so that it continues to work.It updates the ABI so that unwinding is possible without an initialregister state at throw: specifically, as per our RFC, all registers areclobbered. It also includes metadata in the `MachBuffer` that describesexception-catch destinations. However, no unwinder exists to interpretthese catch-destinations yet, so they are untested.* Add try_call_indirect lowering as well. List of files: /wasmtime-44.0.1/cranelift/codegen/src/machinst/lower.rs

44e4919a - Cranelift: remove block params on critical-edge blocks. (#10485)

Mon, 31 Mar 2025 18:57:16 +0000

Cranelift: remove block params on critical-edge blocks. (#10485)When a block has a terminator branch that targets two or more otherblocks at the CLIF level, and any of these blocks have two or moreprecessors, the edge is a "critical edge" and we split it (insert anew empty block) so that the register allocator has a place to putmoves that happen only on that edge. Otherwise, there is no locationthat works: in the predecessor, code runs no matter which outgoingedge we take; and in the successor, code runs no matter which incomingedge we came from.Currently, when we generate these critical-edge blocks, we insertexactly one instruction: an unconditional branch. We wire up theblockparam dataflow by (i) adding block parameters to thecritical-edge block with the same signature as the original target,and (ii) adding all of these arguments to the unconditional branch. Inother words, we maintain the original block signature throughout.This is fine and correct, but it has two downsides. The first is aminor loss in compile-time efficiency (more SSA values andblock-params to process). The second, more interesting, is that ithinders future work with certain kinds of branches that may definevalues *on edges*.In particular, this approach prevents exception-handling support: a`try_call` instruction that acts as a terminator branch (withnormal-return and exceptional out-edges) defines normal-return valuesas block-call arguments that are usable on the normal-returnedge. Some of these normal-return values may be defined by loads froma return-value area. These loads need somewhere to go; they can't go"after the terminator" (then it wouldn't be a terminator), so they goin an edge block; as a result, the block-call for the normal-returnneeds to use its arguments only in the unconditional branch out of theedge block, not in the initial branch to the edge block.This PR alters the critical-edge blockparam handling to have noblock-call args on the branch into the edge block, and use theoriginal values (not the newly defined edge-block blockparams) in theblock-call out of the edge block. This will allow these values to bepossibly defined in the edge block rather than in the predecessor (theblock with the original terminator).This has no functional change today other than some perturbation ofregalloc decisions and a possibly slight compile-time speedup. List of files: /wasmtime-44.0.1/cranelift/codegen/src/machinst/lower.rs

2af0a1f7 - Introduce log2_min_function_alignment flag (#10391)

Thu, 13 Mar 2025 22:55:43 +0000

Introduce log2_min_function_alignment flag (#10391)* Remove function_alignment handling from cranelift-object and cranelift-jitIt is already handled by MachBuffer. The symbol_alignment could also beremoved as no current backend has a symbol alignment bigger than thefunction alignment, but keeping it around is a bit safer when newbackends are introduced.* Introduce log2_min_function_alignment flagThis is required for cg_clif to implement -Zmin-function-alignment. List of files: /wasmtime-44.0.1/cranelift/codegen/src/machinst/lower.rs

66989d9d - Fix minor formatting issues (#9748)

Thu, 05 Dec 2024 22:33:30 +0000

Fix minor formatting issues (#9748)* format: fix typo* format: wrap line length* format: re-wrap comment* format: organize crate dependencies List of files: /wasmtime-44.0.1/cranelift/codegen/src/machinst/lower.rs

708b86c2 - Don't panic when instructions aren't in ISLE (#9631)

Wed, 20 Nov 2024 17:57:04 +0000

Don't panic when instructions aren't in ISLE (#9631)Instead return a codegen error which can be more easily caught,displayed, and rendered. This is intended to help with enabling testingof Pulley in the near future where Pulley doesn't implement mostinstructions at this time and this should make the in-progress state ofthe backend a bit easier to test. List of files: /wasmtime-44.0.1/cranelift/codegen/src/machinst/lower.rs

a05baa38 - Optimize 128-bit multiplication some more in backends (#9215)

Wed, 11 Sep 2024 21:53:42 +0000

Optimize 128-bit multiplication some more in backends (#9215)* Add some tests that will get optimized nextThese tests currently reflect the output of Cranelift today and willshowcase the diff in the next commit of the optimizations implemented.* Optimize 128-bit multiplication some more in backendsThis commits adds support for more patterns of improved 128-bitmultiplication to the various backends of Cranelift. Notably:* Using `isplit` to discard the lower bits of a multiplication maps directly to instructions on aarch64 and riscv64.* Multiplying sign-extended 64-bit halves maps directly to supported instructions on s390x/riscv64/aarch64 (x86_64 already has these implemented).This relies on adding a new method to test whether a `Value` is dead andunused during lowering. While generally not useful this is applicablefor multi-result instructions such as `isplit` where one result may endup not being used. This also is required because the egraphoptimizations can't currently handle multi-result instructions like`isplit` so this can't be optimized to `umulhi` or `smulhi` in CLIF. List of files: /wasmtime-44.0.1/cranelift/codegen/src/machinst/lower.rs