Changes in inst_predicates.rs

87ed3b60 - Cranelift: make all non-tail, non-indirect calls patchable, and rename patchable ABI to `preserve_all`. (#12160)

Mon, 15 Dec 2025 23:29:06 +0000

Cranelift: make all non-tail, non-indirect calls patchable, and rename patchable ABI to `preserve_all`. (#12160)* Cranelift: make all non-tail, non-indirect calls patchable, and rename patchable ABI to `preserve_all`.As discussed in this week's Cranelift meeting, we've discovered a needto generalize the `patchable_call` mechanism and corresponding`patchable` ABI slightly. In particular, we will need patchable`try_call` callsites as well in order to allow breakpoint handlers tothrow exceptions (desirable functionality eventually) and have this workin the presence of inlining. Also, it's just a nice generalization tosay that patchability is an orthogonal dimension to the call ABI and theother restrictions we initially imposed, and works as long as the basicrequirement (no return values) is met.This also renames the `patchable` ABI to `preserve_all`, to make itclear that its purpose is actually orthogonal, and it can be usedindependently of patchable callsites. It also deletes the `cold` ABI,which never actually did anything and is misleading in the presence ofan actual cold-ish (subzero temperature, actually) ABI like`preserve_all`.* Review feedback. List of files: /wasmtime-44.0.1/cranelift/codegen/src/inst_predicates.rs

c00e9ea2 - Cranelift: add patchable call instructions. (#12101)

Tue, 02 Dec 2025 00:59:15 +0000

Cranelift: add patchable call instructions. (#12101)* Cranelift: add patchable call instructions.The new `patchable_call` CLIF instruction pairs with the `patchable`ABI, and emits a callsite with one new key property: the MachBuffercarries metadata that describes exactly which byte range to "NOP out"(overwrite with NOP instructions) to disable that callsite. Doing so issemantically valid and explicitly supported.This enables patching of code at runtime to dynamically turn on and offfeatures such as instrumentation or debugging hooks. We plan to use thisto implement breakpoints in Wasmtime's guest debugging support.As part of this change, I added a notion of "unit of NOP bytes" to theMachBuffer so that the consumer (e.g., Wasmtime's Cranelift-based codecompilation pipeline and metadata-producing logic) can handle patchablecallsites without any other special knowledge of the ISA.For the "real metal" ISAs there are perfectly well-defined NOPs to use,but for Pulley, where all opcodes are assigned at compile time by macromagic, I explicitly defined NOP as opcode byte 0 by moving `Nop`'sdefinition to the top of the list and adding a unit test asserting itsencoding.A design note: in principle it would be possible, as an alternative, totreat "patchability" as an orthogonal dimension of all callsites, andemit the metadata describing the instruction-offset range for anycallsite with the flag set. The only truly necessary semanticrestriction is that there are no return values (because if we turn thecallsite off, nothing writes to them); we could support patchability forother ABIs and for the other kinds of call instructions. The `patchable`ABI would then be better described as something like the "no clobbersABI". I opted not to generalize in this way because it creates someless-tested corners and the generalized form, at least at the MachInstlevel, is not really much simpler in the end.A testing note: I opted not to implement actual code patching in the`cranelift-tools` filetest runner and test patching callsites in/out viasome actuation (e.g. a magic hostcall, like we do for throws) because(i) that's a lot of new plumbing and (ii) we are going to test this veryshortly in Wasmtime anyway and (iii) the correctness (or not) of thelocation-and-length metadata is easy enough to verify in thedisassemblies in the compile-tests.* Review feedback: remove dependence on (and test for) NOP being the literal byte 0. List of files: /wasmtime-44.0.1/cranelift/codegen/src/inst_predicates.rs

a3d6e407 - Cranelift: add debug tag infrastructure. (#11768)

Mon, 06 Oct 2025 19:38:03 +0000

Cranelift: add debug tag infrastructure. (#11768)* Cranelift: add debug tag infrastructure.This PR adds *debug tags*, a kind of metadata that can attach to CLIFinstructions and be lowered to VCode instructions and as metadata onthe produced compiled code. It also adds opaque descriptor blobscarried with stackslots. Together, these two features allow decoratingIR with first-class debug instrumentation that is properly preservedby the compiler, including across optimizations andinlining. (Wasmtime's use of these features will come in followupPRs.)The key idea of a "debug tag" is to allow the Cranelift embedder toexpress whatever information it needs to, in a format that is opaqueto Cranelift itself, except for the parts that need translation duringlowering. In particular, the `DebugTag::StackSlot` variant getstranslated to a physical offset into the stackframe in the compiledmetadata output. So, for example, the embedder can emit a tagreferring to a stackslot, and another describing an offset in thatstackslot.The debug tags exist as a *sequence* on any given instruction; themeaning of the sequence is known only to the embedder, *except* thatduring inlining, the tags for the inlining call instruction areprepended to the tags of inlined instructions. In this way, acanonical use-case of tags as describing original source-languageframes can preserve the source-language view even when multiplefunctions are inlined into one.The descriptor on a stackslot may look a little odd at first, but itspurpose is to allow serializing some description ofstackslot-contained runtime user-program data, in a way that is firmlyattached to the stackslot. In particular, in the face of inlining,this descriptor is copied into the inlining (parent) function from theinlined function when the stackslot entity is copied; no othermetadata outside Cranelift needs to track the identity of stackslotsand know about that motion. This fits nicely with the ability of tagsto refer to stackslots; together, the embedder can annotateinstructions as having certain state in stackslots, and describe theformat of that state per stackslot.This infrastructure is tested with some compile-tests now;testing of the interpretation of the metadata output will come withend-to-end debug instrumentation tests in a followup PR.* Review feedback: add back sequence points and enforce tags only on sequence points or calls.* Use Vecs for debug metadata in MachBuffer to avoid SmallVec size penalty in not-used case.* Review feedback: switch from inlined stackslot descriptor blobs to u64 keys. List of files: /wasmtime-44.0.1/cranelift/codegen/src/inst_predicates.rs

3932e8f1 - Some fixes for try_call (#10593)

Thu, 17 Apr 2025 16:16:41 +0000

Some fixes for try_call (#10593)* Fix cranelift-frontend handling of try_call* Implement eliminate_unreachable_code for exception tables* Ensure try_call is considered a memory fence* Don't error on try_call in the verifier if no TargetIsa is passed* Don't clobber all registers for try_call unless the tail call conv is usedThis way other consumers of Cranelift don't have to pay the cost of theway Wasmtime will implement unwinding on exceptions.* Allow SystemV call conv with try_call List of files: /wasmtime-44.0.1/cranelift/codegen/src/inst_predicates.rs

94ec88ea - Cranelift: initial try_call / try_call_indirect (exception) support. (#10510)

Tue, 08 Apr 2025 00:02:16 +0000

Cranelift: initial try_call / try_call_indirect (exception) support. (#10510)* Cranelift: initial try_call / try_call_indirect (exception) support.This PR adds `try_call` and `try_call_indirect` instructions, andlowerings on four of five ISAs (x86-64, aarch64, riscv64, pulley; s390xhas its own non-shared ABI code that will need separate work).It extends CLIF to support these instructions as new kinds of branches,and extends block-calls to accept `retN` and `exnN` block-call args thatcarry the normal return values or exception payloads (respectively) intothe appropriate successor blocks.It wires up the "normal return path" so that it continues to work.It updates the ABI so that unwinding is possible without an initialregister state at throw: specifically, as per our RFC, all registers areclobbered. It also includes metadata in the `MachBuffer` that describesexception-catch destinations. However, no unwinder exists to interpretthese catch-destinations yet, so they are untested.* Add try_call_indirect lowering as well. List of files: /wasmtime-44.0.1/cranelift/codegen/src/inst_predicates.rs

4d876371 - Add "pure" flag to `ir::MemFlags` (#10340)

Thu, 06 Mar 2025 21:51:49 +0000

Add "pure" flag to `ir::MemFlags` (#10340)* Add "pure" flag to `ir::MemFlags`This flag represents whether the memory operation's safety (e.g. the validity ofits `notrap` and `readonly` claims) is purely a function of its datadependencies.If this flag is `true`, then it is okay to code motion this instruction toarbitrary locations, in the function, including across blocks and conditionalbranches, so long as data dependencies (and trap ordering, if relevant) areupheld.If this flag is `false`, then the memory operation's safety potentially reliesupon invariants that are not reflected in its data dependencies, and thereforeit is not safe to code motion this operation. For example, this operation couldbe in a block that is dominated by a control-flow bounds check that makes thisoperation safe, and that invariant is not reflected in its operands. It would beunsafe to code motion such an instruction above its associated bounds check,even if its data dependencies would still be satisfied.I've added this flag because we were doing exactly that kind of code motionwhere we moved a `readonly` and `notrap` memory operation past its associatednull-check and therefore it was no longer safe to perform and we would get asegfault. This could only be triggered when the Wasm typed-function-referencesproposal was enabled, which is not a tier-1 proposal, so it is not considered avulnerability. Nonetheless, it is a pretty scary kind of bug, and other codepaths weren't affected due to pretty subtle interactions. And this is themotivation for the new "pure" flag: without needing to explicitly opt intodata-dependency-based code motion (i.e. set the "pure" flag), it is too easy toaccidentally move loads past their control-flow-based safety guards.* fix load-hoisting test; also test that non-pure loads don't hoist* Rename `pure` flag to `can_move` List of files: /wasmtime-44.0.1/cranelift/codegen/src/inst_predicates.rs

a88eb702 - Cranelift: dedupe `trap[n]z` instructions (#10004)

Tue, 14 Jan 2025 18:28:51 +0000

Cranelift: dedupe `trap[n]z` instructions (#10004)* Cranelift: dedupe `trap[n]z` instructionsThis commit extends our existing support for merging idempotently side-effectfulinstructions that produce exactly one value to those that produce zero or onevalue, and marks the `trap[n]z` instructions as having idempotent sideeffects. This cleans up a lot test cases in our `disas` test suite, particularlythose related to explicit bounds checks and GC.As an aside, it seems like it should be easy to extend this to idempotentlyside-effectful instructions that produce multiple values as well, but I don'tbelieve we have any such instructions, so I didn't bother.* Update more disas tests* review feedback List of files: /wasmtime-44.0.1/cranelift/codegen/src/inst_predicates.rs

b81ef46c - Remove reference types (`r32` and `r64`) from Cranelift (#9164)

Thu, 22 Aug 2024 21:39:26 +0000

Remove reference types (`r32` and `r64`) from Cranelift (#9164)* Remove reference types (`r32` and `r64`) from Cranelift* restore fuzz regression test List of files: /wasmtime-44.0.1/cranelift/codegen/src/inst_predicates.rs

41eca60b - cranelift: Add `f16const` and `f128const` instructions (#8893)

Wed, 17 Jul 2024 16:39:47 +0000

cranelift: Add `f16const` and `f128const` instructions (#8893)* cranelift: Add `f16const` and `f128const` instructions* cranelift: Add constant propagation for `f16` and `f128` List of files: /wasmtime-44.0.1/cranelift/codegen/src/inst_predicates.rs

b869b66b - cranelift: Delete redundant DCE optimization pass (#8227)

Mon, 13 May 2024 16:12:48 +0000

cranelift: Delete redundant DCE optimization pass (#8227)The egraph pass and the dead-code elimination pass both removeinstructions whose results are unused. If the optimization level is"none", neither pass runs, and if it's anything else both passes run. Idon't think we should do this work twice.Note that the DCE pass is different than the "eliminate unreachablecode" pass, which removes entire blocks that are unreachable from theentry block. That pass might still be necessary. List of files: /wasmtime-44.0.1/cranelift/codegen/src/inst_predicates.rs

1721fe3f - Cranelift: Do not dedupe/GVN bitcasts from reference values (#8317)

Mon, 08 Apr 2024 22:42:25 +0000

Cranelift: Do not dedupe/GVN bitcasts from reference values (#8317)* Cranelift: Do not dedupe/GVN bitcasts from reference valuesDeduping bitcasts to integers from references can make the references no longlonger live across safepoints, and instead only the bitcasted integer resultswould be. Because the reference is no longer live after the safepoint, thesafepoint's stack map would not have an entry for the reference, which couldresult in the collector reclaiming an object too early, which is basically ause-after-free bug. Luckily, we sandbox the GC heap now, so such UAF bugs aren'tmemory unsafe, but they could potentially result in denial of serviceattacks. Either way, we don't want those bugs!On the other hand, it is technically fine to dedupe bitcasts *to* referencetypes. Doing so extends, rather than shortens, the live range of the GCreference. This potentially adds it to more stack maps than it otherwise wouldhave been in, which means it might unnecessarily survive a GC it otherwisewouldn't have. But that is fine. Shrinking live ranges of GC references, andremoving them from stack maps they otherwise should have been in, is theproblematic transformation.* Add additional logging and debug asserts for GC stuff List of files: /wasmtime-44.0.1/cranelift/codegen/src/inst_predicates.rs

f684a5fb - remove `iadd_cout` and `isub_bout` (#6198)

Tue, 11 Apr 2023 23:39:32 +0000

remove `iadd_cout` and `isub_bout` (#6198) List of files: /wasmtime-44.0.1/cranelift/codegen/src/inst_predicates.rs

7b8854f8 - egraphs: fix handling of effectful-but-idempotent ops and GVN. (#5800)

Thu, 02 Mar 2023 02:10:42 +0000

egraphs: fix handling of effectful-but-idempotent ops and GVN. (#5800)* Revert "egraphs: disable GVN of effectful idempotent ops (temporarily). (#5808)"This reverts commit c7e25718665aa3fd5f28d4a3d0c94580eb040c37.* egraphs: fix handling of effectful-but-idempotent ops and GVN.This PR addresses #5796: currently, ops that are effectful, i.e., remainin the side-effecting skeleton (which we keep in the `Layout` while theegraph exists), but are idempotent and thus mergeable by a GVN pass, arenot handled properly.GVN is still possible on effectful but idempotent ops precisely becauseour GVN does not create partial redundancies: it removes an instructiononly when it is dominated by an identical instruction. An isntructionwill not be "hoisted" to a point where it could execute in the optimizedcode but not in the original.However, there are really two parts to the egraph implementation thatproduce this effect: the deduplication on insertion into the egraph, andthe elaboration with a scoped hashmap. The deduplication lets us give asingle name (value ID) to all copies of an identical instruction, andthen elaboration will re-create duplicates if GVN should not hoist ormerge some of them.Because deduplication need not worry about dominance or scopes, we use asimple (non-scoped) hashmap to dedup/intern ops as "egraph nodes".When we added support for GVN'ing effectful but idempotent ops (#5594),we kept the use of this simple dedup'ing hashmap, but these ops do notget elaborated; instead they stay in the side-effecting skeleton. Thus,we inadvertently created potential for weird code-motion effects.The proposal in #5796 would solve this in a clean way by treating theseops as pure again, and keeping them out of the skeleton, instead putting"force" pseudo-ops in the skeleton. However, this is a little morecomplex than I would like, and I've realized that @jameysharp's earliersuggestion is much simpler: we can keep an actual scoped hashmapseparately just for the effectful-but-idempotent ops, and use it to GVNwhile we build the egraph. In effect, we're fusing a separate GVN passwith the egraph pass (but letting it interact corecursively withegraph rewrites. This is in principle similar to how we keep a separatemap for loads and fuse this pass with the egraph rewrite pass as well.Note that we can use a `ScopedHashMap` here without the "context" (asneeded by `CtxHashMap`) because, as noted by @jameysharp, in practicethe ops we want to GVN have all their args inline. Equality on the`InstructinoData` itself is conservative: two insts whose structcontents compare shallowly equal are definitely identical, but identicalinsts in a deep-equality sense may not compare shallowly equal, due tolist indirection. This is fine for GVN, because it is still sound toskip any given GVN opportunity (and keep the original instructions).Fixes #5796.* Add comments from review. List of files: /wasmtime-44.0.1/cranelift/codegen/src/inst_predicates.rs

c7e25718 - egraphs: disable GVN of effectful idempotent ops (temporarily). (#5808)

Thu, 16 Feb 2023 21:29:03 +0000

egraphs: disable GVN of effectful idempotent ops (temporarily). (#5808)This is a short-term fix to the same bug that #5800 is addressing(#5796), but with less risk: it simply turns off GVN'ing of effectfulbut idempotent ops. Because we have an upcoming release, and this is amiscompile (albeit to do with trapping behavior), we would like to makethe simplest possible fix that avoids the bug, and backport it. I willthen rebase #5800 on top of a revert of this followed by the morecomplete fix. List of files: /wasmtime-44.0.1/cranelift/codegen/src/inst_predicates.rs

80c147d9 - Rework br_table to use BlockCall (#5731)

Thu, 16 Feb 2023 17:23:27 +0000

Rework br_table to use BlockCall (#5731)Rework br_table to use BlockCall, allowing us to avoid adding new nodes during ssa construction to hold block arguments. Additionally, many places where we previously matched on InstructionData to extract branch destinations can be replaced with a use of branch_destination or branch_destination_mut. List of files: /wasmtime-44.0.1/cranelift/codegen/src/inst_predicates.rs

d99783fc - Move default blocks into jump tables (#5756)

Fri, 10 Feb 2023 16:53:30 +0000

Move default blocks into jump tables (#5756)Move the default block off of the br_table instrution, and into the JumpTable that it references. List of files: /wasmtime-44.0.1/cranelift/codegen/src/inst_predicates.rs

b0b3f67c - Move jump tables to the DataFlowGraph (#5745)

Wed, 08 Feb 2023 05:21:35 +0000

Move jump tables to the DataFlowGraph (#5745)Move the storage for jump tables off of FunctionStencil and onto DataFlowGraph. This change is in service of #5731, making it easier to access the jump table data in the context of helpers like inst_values. List of files: /wasmtime-44.0.1/cranelift/codegen/src/inst_predicates.rs

3343cf80 - Add assertions for matches that used to use analyze_branch (#5733)

Tue, 07 Feb 2023 22:51:18 +0000

Add assertions for matches that used to use analyze_branch (#5733)Following up from #5730, add debug assertions to ensure that new branch instructions don't slip through matches that used to use analyze_branch. List of files: /wasmtime-44.0.1/cranelift/codegen/src/inst_predicates.rs

2c842599 - Refactor matches that used to consume BranchInfo (#5734)

Tue, 07 Feb 2023 21:29:42 +0000

Refactor matches that used to consume BranchInfo (#5734)Explicitly borrow the instruction data, and use a mutable borrow to avoid rematch. List of files: /wasmtime-44.0.1/cranelift/codegen/src/inst_predicates.rs

c8a6adf8 - Remove analyze_branch and BranchInfo (#5730)

Tue, 07 Feb 2023 01:06:57 +0000

Remove analyze_branch and BranchInfo (#5730)We don't have overlap in behavior for branch instructions anymore, so we can remove analyze_branch and instead match on the InstructionData directly. Co-authored-by: Jamey Sharp List of files: /wasmtime-44.0.1/cranelift/codegen/src/inst_predicates.rs