en Copyright 2015 Java http://172.16.0.5:8080/history/wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs#099102d9 Remove `expect(clippy::allow_attributes_without_reason)` from cranelift-codegen (#11182)* Remove `expect(clippy::allow_attributes_without_reason)` from cranelift-codegenThis commit gets around to migrating the `cranelift-codegen` crate torequire a reason on lint directives and additionally switch to`#[expect]` where possible.prtest:full* Move x64-only item to x64 backend List of files: /wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs Mon, 07 Jul 2025 17:25:40 +0000 Alex Crichton <alex@alexcrichton.com> http://172.16.0.5:8080/history/wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs#66989d9d Fix minor formatting issues (#9748)* format: fix typo* format: wrap line length* format: re-wrap comment* format: organize crate dependencies List of files: /wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs Thu, 05 Dec 2024 22:33:30 +0000 Andrew Brown <andrew.brown@intel.com> http://172.16.0.5:8080/history/wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs#0c0153c1 Enforce `clippy::clone_on_copy` for the workspace (#9025)* Derive `Copy` for `Val`* Fix `clippy::clone_on_copy` for the whole repo* Enforce `clippy::clone_on_copy` for the workspace* fix some more clippy::clone_on_copy that got missed List of files: /wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs Sat, 27 Jul 2024 01:11:06 +0000 Nick Fitzgerald <fitzgen@gmail.com> http://172.16.0.5:8080/history/wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs#7adf3cac cranelift-codegen: Prepare cranelift codegen for usage from Winch (#5413)This commit prepares the x64 pieces from cranelift codegen to be consumed by Winch for binary emission. This change doesn't introduce or modifies functionality it makes the necessary pieces for binary emission public. This change also improves documentation where applicable. List of files: /wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs Mon, 12 Dec 2022 17:01:06 +0000 Saúl Cabrera <saulecabrera@gmail.com> http://172.16.0.5:8080/history/wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs#b652ce2f ISLE: add support for multi-extractors and multi-constructors. (#4908)* ISLE: add support for multi-extractors and multi-constructors. This support allows for rules that process multiple matching values per extractor call on the left-hand side, and as a result, can produce multiple values from the constructor whose body they define. This is useful in situations where we are matching on an input data structure that can have multiple "nodes" for a given value or ID, for example in an e-graph. * Review feedback: all multi-ctors and multi-etors return iterators; no `Vec` case. * Add additional warning suppressions to generated-code toplevels to be consistent with new islec output. List of files: /wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs Wed, 21 Sep 2022 23:36:50 +0000 Chris Fallin <chris@cfallin.org> http://172.16.0.5:8080/history/wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs#5d671952 Cranelift: do not check in generated ISLE code; regenerate on every compile. (#4143)This PR fixes #4066: it modifies the Cranelift `build.rs` workflow to invoke the ISLE DSL compiler on every compilation, rather than only when the user specifies a special "rebuild ISLE" feature. The main benefit of this change is that it vastly simplifies the mental model required of developers, and removes a bunch of failure modes we have tried to work around in other ways. There is now just one "source of truth", the ISLE source itself, in the repository, and so there is no need to understand a special "rebuild" step and how to handle merge errors. There is no special process needed to develop the compiler when modifying the DSL. And there is no "noise" in the git history produced by constantly-regenerated files. The two main downsides we discussed in #4066 are: - Compile time could increase, by adding more to the "meta" step before the main build; - It becomes less obvious where the source definitions are (everything becomes more "magic"), which makes exploration and debugging harder. This PR addresses each of these concerns: 1. To maintain reasonable compile time, it includes work to cut down the dependencies of the `cranelift-isle` crate to *nothing* (only the Rust stdlib), in the default build. It does this by putting the error-reporting bits (`miette` crate) under an optional feature, and the logging (`log` crate) under a feature-controlled macro, and manually writing an `Error` impl rather than using `thiserror`. This completely avoids proc macros and the `syn` build slowness. The user can still get nice errors out of `miette`: this is enabled by specifying a Cargo feature `--features isle-errors`. 2. To allow the user to optionally inspect the generated source, which nominally lives in a hard-to-find path inside `target/` now, this PR adds a feature `isle-in-source-tree` that, as implied by the name, moves the target for ISLE generated source into the source tree, at `cranelift/codegen/isle_generated_source/`. It seems reasonable to do this when an explicit feature (opt-in) is specified because this is how ISLE regeneration currently works as well. To prevent surprises, if the feature is *not* specified, the build fails if this directory exists. List of files: /wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs Thu, 12 May 2022 05:25:24 +0000 Chris Fallin <chris@cfallin.org> http://172.16.0.5:8080/history/wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs#eb435f30 x64: use constant pool for u64 constants rather than `movabs`. (#4088)* Allow emitting u64 constants into constant pool. * Use constant pool for constants on x64 that do not fit in a simm32 and are needed as a RegMem or RegMemImm. * Fix rip-relative addressing bug in pinsrd emission. List of files: /wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs Tue, 10 May 2022 16:21:05 +0000 Chris Fallin <chris@cfallin.org> http://172.16.0.5:8080/history/wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs#2af8d1e9 Cranelift/ISLE: re-apply prio-trie fix, this time with fixed fix. (#4117)* ISLE compiler: fix priority-trie interval bug. (#4093) This PR fixes a bug in the ISLE compiler related to rule priorities. An important note first: the bug did not affect the correctness of the Cranelift backends, either in theory (because the rules should be correct applied in any order, even contrary to the stated priorities) or in practice (because the generated code actually does not change at all with the DSL compiler fix, only with a separate minimized bug example). The issue was a simple swap of `min` for `max` (see first commit). This is the minimal fix, I think, to get a correct priority-trie with the minimized bug example in this commit. However, while debugging this, I started to convince myself that the complexity of merging multiple priority ranges using the sort of hybrid interval tree / string-matching trie data structure was unneeded. The original design was built with the assumption we might have a bunch of different priority levels, and would need the efficiency of merging where possible. But in practice we haven't used priorities this way: the vast majority of lowering rules exist at the default (priority 0), and just a few overrides are explicitly at prio 1, 2 or (rarely) 3. So, it turns out to be a lot simpler to label trie edges with (prio, symbol) rather than (prio-range, symbol), and delete the whole mess of interval-splitting logic on insertion. It's easier (IMHO) to convince oneself that the resulting insertion algorithm is correct. I was worried that this might impact the size of the generated Rust code or its runtime, but In fact, to my initial surprise (but it makes sense given the above "rarely used" factor), the generated code with this compiler fix is *exactly the same*. I rebuilt with `--features rebuild-isle,all-arch` but... there were no diffs to commit! This is to me the simplest evidence that we didn't really need that complexity. * Fix earlier commit from #4093: properly sort trie. This commit fixes an in-hindsight-obvious bug in #4093: the trie's edges must be sorted recursively, not just at the top level. With this fix, the generated code differs only in one cosmetic way (a let-binding moves) but otherwise is the same. This includes @fitzgen's fix to the CI (from the revert in #4102) that deletes manifests to actually check that the checked-in source is consistent with the checked-in compiler. The force-rebuild step is now in a shell script for convenience: anyone hacking on the ISLE compiler itself can use this script to more easily rebuild everything. * Add note to build.rs to remind to update force-rebuild-isle.sh List of files: /wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs Mon, 09 May 2022 23:36:48 +0000 Chris Fallin <chris@cfallin.org> http://172.16.0.5:8080/history/wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs#f85047b0 Rework x64 addressing-mode lowering to be slightly more flexible. (#4080)This PR refactors the x64 backend address-mode lowering to use an incremental-build approach, where it considers each node in a tree of `iadd`s that feed into a load/store address and, at each step, builds the best possible `Amode`. It will combine an arbitrary number of constant offsets (an extension beyond the current rules), and can capture a left-shifted (scaled) index in any position of the tree (another extension). This doesn't have any measurable performance improvement on our Wasm benchmarks in Sightglass, unfortunately, because the IR lowered from wasm32 will do address computation in 32 bits and then `uextend` it to add to the 64-bit heap base. We can't quite lift the 32-bit adds to 64 bits because this loses the wraparound semantics. (We could label adds as "expected not to overflow", and allow *those* to be lifted to 64 bit operations; wasm32 heap address computation should fit this. This is `add nuw` (no unsigned wrap) in LLVM IR terms. That's likely my next step.) Nevertheless, (i) this generalizes the cases we can handle, which should be a good thing, all other things being equal (and in this case, no compile time impact was measured); and (ii) might benefit non-Wasm frontends. List of files: /wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs Mon, 02 May 2022 23:20:39 +0000 Chris Fallin <chris@cfallin.org> http://172.16.0.5:8080/history/wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs#03793b71 ISLE: remove all uses of argument polarity, and remove it from the language. (#4091)This PR removes "argument polarity": the feature of ISLE extractors that lets them take inputs aside from the value to be matched. Cases that need this expressivity have been subsumed by #4072 with if-let clauses; we can now finally remove this misfeature of the language, which has caused significant confusion and has always felt like a bit of a hack. This PR (i) removes the feature from the ISLE compiler; (ii) removes it from the reference documentation; and (iii) refactors away all uses of the feature in our three existing backends written in ISLE. List of files: /wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs Mon, 02 May 2022 16:52:12 +0000 Chris Fallin <chris@cfallin.org> http://172.16.0.5:8080/history/wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs#12b4374c [AArch64] Port atomic rmw to ISLE (#4021)Also fix and extend the current implementation: - AtomicRMWOp::Clr != AtomicRmwOp::And, as the input needs to be inverted first. - Inputs to the cmp for the RMWLoop case are sign-extended when needed. - Lower Xchg to Swp. - Lower Sub to Add with a negated input. - Added more runtests. Copyright (c) 2022, Arm Limited. List of files: /wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs Wed, 27 Apr 2022 20:13:59 +0000 Sam Parker <sam.parker@arm.com> http://172.16.0.5:8080/history/wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs#dd45f445 x64 backend: add lowerings with load-op-store fusion. (#4071)x64 backend: add lowerings with load-op-store fusion. These lowerings use the `OP [mem], reg` forms (or in AT&T syntax, `OP %reg, (mem)`) -- i.e., x86 instructions that load from memory, perform an ALU operation, and store the result, all in one instruction. Using these instruction forms, we can merge three CLIF ops together: a load, an arithmetic operation, and a store. List of files: /wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs Wed, 27 Apr 2022 01:58:26 +0000 Chris Fallin <chris@cfallin.org> http://172.16.0.5:8080/history/wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs#164bfeaf x64 backend: migrate stores, and remainder of loads (I128 case), to ISLE. (#4069) List of files: /wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs Tue, 26 Apr 2022 16:50:46 +0000 Chris Fallin <chris@cfallin.org> http://172.16.0.5:8080/history/wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs#f384938a x64 backend: fix a load-op merging bug with integer min/max. (#4068)The recent work in #4061 introduced a notion of "unique uses" for CLIF values that both simplified the load-op merging rules and allowed loads to merge in some more places. Unfortunately there's one factor that PR didn't account for: a unique use at the CLIF level could become a multiple-use at the VCode level, when a lowering uses a value multiple times! Making this less error-prone in general is hard, because we don't know the lowering in VCode until it's emitted, so we can't ahead-of-time know that a value will be used multiple times and prevent its merging. But we *can* know in the lowerings themselves when we're doing this. At least we get a panic from regalloc when we get this wrong; no bad code (uninitialized register being read) should ever come from a backend bug like this. This is still a bit less than ideal, but for now the fix is: in `cmp_and_choose` in the x64 backend (which compares values, then picks one or the other with a cmove), explicitly put values in registers. Fixes #4067 (thanks @Mrmaxmeier for the report!). List of files: /wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs Mon, 25 Apr 2022 17:32:09 +0000 Chris Fallin <chris@cfallin.org> http://172.16.0.5:8080/history/wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs#e4b7c8a7 Cranelift: fix #3953: rework single/multiple-use logic in lowering. (#4061)* Cranelift: fix #3953: rework single/multiple-use logic in lowering. This PR addresses the longstanding issue with loads trying to merge into compares on x86-64, and more generally, with the lowering framework falsely recognizing "single uses" of one op by another (which would normally allow merging of side-effecting ops like loads) when there is *indirect* duplication. To fix this, we replace the direct `value_uses` count with a transitive notion of uniqueness (not unlike Rust's `&`/`&mut` and how a `&mut` downgrades to `&` when accessed through another `&`!). A value is used multiple times transitively if it has multiple direct uses, or is used by another op that is used multiple times transitively. The canonical example of badness is: ``` v1 := load v2 := ifcmp v1, ... v3 := selectif v2, ... v4 := selectif v2, ... ``` both `v3` and `v4` effectively merge the `ifcmp` (`v2`), so even though the use of `v1` is "unique", it is codegenned twice. This is why we ~~can't have nice things~~ can't merge loads into compares (#3953). There is quite a subtle and interesting design space around this problem and how we might solve it. See the long doc-comment on `ValueUseState` in this PR for more justification for the particular design here. In particular, this design deliberately simplifies a bit relative to an "optimal" solution: some uses can *become* unique depending on merging, but we don't design our data structures for such updates because that would require significant extra costly tracking (some sort of transitive refcounting). For example, in the above, if `selectif` somehow did not merge `ifcmp`, then we would only codegen the `ifcmp` once into its result register (and use that register twice); then the load *is* uniquely used, and could be merged. But that requires transitioning from "multiple use" back to "unique use" with careful tracking as we do pattern-matching, which I've chosen to make out-of-scope here for now. In practice, I don't think it will matter too much (and we can always improve later). With this PR, we can now re-enable load-op merging for compares. A subsequent commit does this. * Update x64 backend to allow load-op merging for `cmp`. * Update filetests. * Add test for cmp-mem merging on x64. * Comment fixes. * Rework ValueUseState analysis for better performance. * Update s390x filetest: iadd_ifcout cannot merge loads anymore because it has multiple outputs (ValueUseState limitation) * Address review comments. List of files: /wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs Sat, 23 Apr 2022 01:00:48 +0000 Chris Fallin <chris@cfallin.org> http://172.16.0.5:8080/history/wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs#6a36a1d1 X64: Port Sqrt to ISLE (#4065) List of files: /wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs Fri, 22 Apr 2022 07:42:22 +0000 Johnnie Birch <johnnie.l.birch.jr@intel.com> http://172.16.0.5:8080/history/wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs#a0318f36 Switch Cranelift over to regalloc2. (#3989)This PR switches Cranelift over to the new register allocator, regalloc2. See [this document](https://gist.github.com/cfallin/08553421a91f150254fe878f67301801) for a summary of the design changes. This switchover has implications for core VCode/MachInst types and the lowering pass. Overall, this change brings improvements to both compile time and speed of generated code (runtime), as reported in #3942: ``` Benchmark Compilation (wallclock) Execution (wallclock) blake3-scalar 25% faster 28% faster blake3-simd no diff no diff meshoptimizer 19% faster 17% faster pulldown-cmark 17% faster no diff bz2 15% faster no diff SpiderMonkey, 21% faster 2% faster fib(30) clang.wasm 42% faster N/A ``` List of files: /wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs Thu, 14 Apr 2022 17:28:21 +0000 Chris Fallin <chris@cfallin.org> http://172.16.0.5:8080/history/wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs#7a55779c x64: fix miscompilation of `select.i128` (#4017)Issue #3963 identified a miscompilation with select in which the second in the pair of `CMOV`s (one pair per `i128` register) used the wrong flag. This change fixes the error in the x64 ISLE helper function emitting these `CMOV` instructions. List of files: /wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs Tue, 12 Apr 2022 16:56:57 +0000 Andrew Brown <andrew.brown@intel.com> http://172.16.0.5:8080/history/wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs#f62199da x64: port `load` to ISLE (#3993)This change moves the majority of the lowerings for CLIF's `load` instruction over to ISLE. To do so, it also migrates the previous mechanism for creating an `Amode` (`lower_to_amode`) to several ISLE rules (see `to_amode`). List of files: /wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs Fri, 08 Apr 2022 01:31:22 +0000 Andrew Brown <andrew.brown@intel.com> http://172.16.0.5:8080/history/wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs#e8dd13cf x64: port the remainder of `select` to ISLE (#3973)Previous changes had ported the difficult "`select` based on an `fcmp`" patterns to ISLE; this completes porting of `select` by moving over the final two kinds of patterns: - `select` based on an `icmp` - `select` based on a value List of files: /wasmtime-44.0.1/cranelift/codegen/src/isa/x64/lower/isle/generated_code.rs Wed, 30 Mar 2022 20:32:26 +0000 Andrew Brown <andrew.brown@intel.com>