cranelift-assembler-x64: no_std support (#12235)

* cranelift-assembler-x64: no_std support

* cargo fmt

* Update cranelift/assembler-x64/src/xmm.rs

Co-authored-by: bjorn3 <[email protected]

cranelift-assembler-x64: no_std support (#12235)

* cranelift-assembler-x64: no_std support

* cargo fmt

* Update cranelift/assembler-x64/src/xmm.rs

Co-authored-by: bjorn3 <[email protected]>

* Update cranelift/assembler-x64/src/imm.rs

Co-authored-by: bjorn3 <[email protected]>

* Update cranelift/assembler-x64/src/api.rs

Co-authored-by: bjorn3 <[email protected]>

* Reorder import of alloc

* #![no_std]

* revert main.rs

* cargo fmt

* remove core feature

* Fix CI

---------

Co-authored-by: bjorn3 <[email protected]>

show more ...

x64: Refactor emission of LoadExtName (#11124)

* x64: Expand symbol-related tests

* x64: Refactor emission of `LoadExtName`

Don't use raw bytes when emitting this instruction but instead use
symb

x64: Refactor emission of LoadExtName (#11124)

* x64: Expand symbol-related tests

* x64: Refactor emission of `LoadExtName`

Don't use raw bytes when emitting this instruction but instead use
symbolic assembler directives to avoid needing to hardcode bytes. This
additionally fixes an issue where with `colocated` symbols the offset of
the symbol was not taken into account (but for Wasmtime it's always been
0 so this otherwise hasn't come up so far)

* Fix emit tests

* Update test expectation

* Fuse `KnownOffsetTable` and `CodeSink` in the new assembler

Enables a `CodeSink` to know about all offsets, not just those against
labels. Resolves an issue where the fuzzing code sink was otherwise
unaware of `DeferredTarget::None`.

* Review comments

show more ...

x64: Migrate `lea` to the new assembler (#11084)

* x64: Add a shrink-before-emit optimization

This commit adds a helper function to attempt to shrink a function just
before emission if possible ba

x64: Migrate `lea` to the new assembler (#11084)

* x64: Add a shrink-before-emit optimization

This commit adds a helper function to attempt to shrink a function just
before emission if possible based on the results of register allocation.
While many instructions can be selected ahead of time (e.g. instructions
with smaller immediate widths) some are only possible after register
allocation has happened. For example the `AND RAX, imm32` instruction is
never generated in ISLE because it doesn't make sense to constrain a
register to only `RAX`, but if register allocation happened to put
`AND r/m64, imm32` into `RAX` then it's possible to use the more compact
encoding instead.

This function updates the emission of external instructions to, just
before emission, attempt to pattern-match these instructions and shrink
to a smaller instruction. This is done for a number of instructions
which support a smaller encoding when the operands are `RAX` and an
immediate. In the future my thinking was to use this additionally for
the optimization to change `LEA` instructions into `ADD` instructions if
the src/destination are equal and the addressing mode matches.

* x64: Migrate `lea` to the new assembler

Adding `lea` is easy enough but preserving the optimization of
using `add` instead of `lea` required a bit of finesse. This hooks into
the `emit_maybe_shrink` added previously for implementing this.

* Fix test build

show more ...

x64: Migrate `mulx` to the new assembler (#10887)

* x64: Migrate `mulx` to the new assembler

This is an interesting instruction as it has a relatively unique shape
compared to many others. The VEX

x64: Migrate `mulx` to the new assembler (#10887)

* x64: Migrate `mulx` to the new assembler

This is an interesting instruction as it has a relatively unique shape
compared to many others. The VEX encoding is used to give it a 3-operand
form, although it still has an implicit 4th operand as well. The other
unique part about this instruction is that if the two write-only
operands are the same then that has a different semantic meaning than if
they are different.

Modeling the two-output form of the instruction was pretty easy, the
only changes needed were to add the `r32a` and `r32b` locations as
previously only `r32` was available. Modeling the one-output form of the
instruction led to a "hook" where these instructions specify that they
use a custom regalloc function. That skips the auto-generated regalloc
entirely and defers to a new `custom` submodule in the assembler crate.
These custom variants handle only having a single write operand on the
instruction.

* Fix fuzz build

show more ...

x64: Migrate xmm mov-family instructions to new assembler (#10834)

* x64: Migrate xmm mov-family instructions to new assembler

This commit migrates xmm movement-style instructions, such as
`mov{s{s

x64: Migrate xmm mov-family instructions to new assembler (#10834)

* x64: Migrate xmm mov-family instructions to new assembler

This commit migrates xmm movement-style instructions, such as
`mov{s{s,d},{a,u}p{s,d},dq{a,u}}` to the new assembler. This enables
deleting the `XmmMovRM` class of instructions in ISLE. Along the way a
number of notable changes were made:

* ISLE constructors for assembler instructions now take `SyntheticAmode`
instead of `Amode` since it's already supported anyway and it's a more
flexible argument to take.
* The conversion from `SyntheticAmode` to the assembler `Amode<Gpr>` was
fixed where one variant needed to use `rsp` instead of `rbp`.
* The `is_move` method was updated to ignore `movss` and `movsd`
instructions since they're not true movement-related instructions in
register-to-register situations.

* Use `gen_move` helper in Winch

show more ...

x64: Convert some lane-manipulation instructions to the new assembler (#10825)

* x64: Convert some lane-manipulation instructions to the new assembler

This was mostly straightforward but ended up r

x64: Convert some lane-manipulation instructions to the new assembler (#10825)

* x64: Convert some lane-manipulation instructions to the new assembler

This was mostly straightforward but ended up requiring handling for a
new style of instruction not previously before-seen: a write-only
`GprMem` operand. This needed some extra treatment/helpers in the ISLE
integration to ensure that this was all hooked up correctly from ISLE
constructors down to the assembler instructions.

* Remove `GprToXmm` as well

Already have mappings for `movd` and `movq`, so let's migrate this as well.

* Remove some old emit tests

show more ...

x64: Migrate div instructions to the new assembler (#10820)

* x64: Migrate div instructions to the new assembler

This mostly required adding a new kind of operand representing a
`TrapCode` and plum

x64: Migrate div instructions to the new assembler (#10820)

* x64: Migrate div instructions to the new assembler

This mostly required adding a new kind of operand representing a
`TrapCode` and plumbing around a few bits and bobs to ensure that this
compile-time-only-abstraction does not need to be accounted for in all
the encoding bits.

* Fix doc tests

* Shrink the size of `Amode`

Reduces the size of `KnownOffset` so `Option<KnownOffset>` doesn't have
such a high alignment or size, greatly shrinking the size of the
instruction and prevents the changes to `Div` from increasing the size
of the overall `Inst` enum. In fact it now shrinks!

* Don't model a trap code as an operand

* Render trap information in `Display`

show more ...

Minor optimizations to assembler-generated code (#10811)

* Minor optimizations to assembler-generated code

This is a few minor changes in the hopes of optimizing the compile-time
of the generated c

Minor optimizations to assembler-generated code (#10811)

* Minor optimizations to assembler-generated code

This is a few minor changes in the hopes of optimizing the compile-time
of the generated code itself for the x64 assembler crate, including:

* Add `derive(Copy)` to all instructions to benefit from a specialized
implementation of `derive(Clone)` when a `Copy` implementation is also
present (e.g. it's `*foo` instead of a structural `clone`-each-field).
* Don't use `write!` in `Display for Inst` and instead delegate directly
with methods to avoid formatting machinery.
* Use helper methods in `RegisterVisitor` trait to have register
allocation be a method-per-operand and shrink the methods a bit.

* Fix clippy warnings

show more ...

asm: implement write-only operands (#10759)

* asm: implement write-only operands

This change adds support to the new assembler for write-only operands.
This implementation appeared first in [#10754

asm: implement write-only operands (#10759)

* asm: implement write-only operands

This change adds support to the new assembler for write-only operands.
This implementation appeared first in [#10754] but is split out here to
unblock implementation of instructions that require it: multiplication,
conversions, moves, etc. This starts roughly the same as what was
implemented for write-only XMMs in [#10754] but includes support for
write-only GPRs as well and generates the temporary registers which are
needed.

[#10754]: https://github.com/bytecodealliance/wasmtime/pull/10754

Co-authored-by: Johnnie Birch <[email protected]>

* fix: use `to_reg()` to extract the Cranelift type from `Writable`

---------

Co-authored-by: Johnnie Birch <[email protected]>

show more ...

asm: plumb fixed registers through operand visitor (#10644)

* asm: plumb fixed registers through operand visitor

This finishes plumbing out fixed registers using the visitor pattern in
`cranelift-c

asm: plumb fixed registers through operand visitor (#10644)

* asm: plumb fixed registers through operand visitor

This finishes plumbing out fixed registers using the visitor pattern in
`cranelift-codegen`. This was skipped initially in [#10629] because
[#10632] offered a different approach; while we discuss whether and how
that should proceed, this change at least fixes up the current state of
things.

[#10629]: https://github.com/bytecodealliance/wasmtime/pull/10629
[#10632]: https://github.com/bytecodealliance/wasmtime/pull/10632

* Refactor visitor-related codegen

This simplifies and clarifies several parts related to emitting `visit`
functions for assembler instructions.

show more ...

asm: allow printing of virtual registers (#10389)

* Rename module: `reg` to `gpr`

* Override `to_string` implementation for `cranelift-codegen` types

* Add pretty-printing tests

asm: sse orpd implementation (#10273)

* sse orpd implementation

assembler integration with isle

format

add clippy reason, reorder avx priority in isle

bless tests for orpd

create separate xmm m

asm: sse orpd implementation (#10273)

* sse orpd implementation

assembler integration with isle

format

add clippy reason, reorder avx priority in isle

bless tests for orpd

create separate xmm module

validate function rewrite sse condition

add quote from manual for sse prefix

format changes

move Xmm bits under Reg

* use new isle constructors for sse

* remove unused function

* minor changes

show more ...

asm: re-allow prop-testing with `cargo test` (#10185)

In #10110, I originally intended to use `arbitrary` implementations in
two ways: for long-running fuzz testing (e.g., with OSS-Fuzz) but also
fo

asm: re-allow prop-testing with `cargo test` (#10185)

In #10110, I originally intended to use `arbitrary` implementations in
two ways: for long-running fuzz testing (e.g., with OSS-Fuzz) but also
for quick property testing with `cargo test`. This latter use case could
replace the tedious emit tests we had to write in `cranelift-codegen`
_and_ find corner cases that we otherwise might not explore. It helped
me during development: just run `cargo test` to check if anything is
obviously wrong. `arbtest` seemed to be able to run ~1000 test cases and
found mistakes well within the one second time limit I gave it.

@alexcrichton improved #10110 by avoiding `Arbitrary` implementations
everywhere and unconditionally depending on the `arbitrary` crate. This
was the right change, but it removed the ability to property test using
`cargo test`. What this change does is retain the general intent of his
change (no extra dependencies) but add `Arbitrary` implementations for
`cfg(test)` as well to run property tests during `cargo test`.

The only downside I see here is the added complexity when conditionally
compiling the fuzz-related bits: `#[cfg(any(test, feature = "fuzz"))]`.
Perhaps there is a better way to do this, but this seemed to work fine.
Let me know what you think.

show more ...

asm: introduce a new x64 assembler (#10110)

* asm: add initial infrastructure for an external assembler

This change adds some initial logic implementing an external assembler
for Cranelift's x64 ba

asm: introduce a new x64 assembler (#10110)

* asm: add initial infrastructure for an external assembler

This change adds some initial logic implementing an external assembler
for Cranelift's x64 backend, as proposed in RFC [#41].

This adds two crates:
- the `cranelift/assembler/meta` crate defines the instructions; to
print out the defined instructions use `cargo run -p
cranelift-assembler-meta`
- the `cranelift/assembler` crate exposes the generated Rust code for
those instructions; to see the path to the generated code use `cargo
run -p cranelift-assembler`

The assembler itself is straight-forward enough (modulo the code
generation, of course); its integration into `cranelift-codegen` is what
is most tricky about this change. Instructions that we will emit in the
new assembler are contained in the `Inst::External` variant. This
unfortunately increases the memory size of `Inst`, but only temporarily
if we end up removing the extra `enum` indirection by adopting the new
assembler wholesale. Another integration point is ISLE: we generate ISLE
definitions and a Rust helper macro to make the external assembler
instructions accessible to ISLE lowering.

This change introduces some duplication: the encoding logic (e.g. for
REX instructions) currently lives both in `cranelift-codegen` and the
new assembler crate. The `Formatter` logic for the assembler `meta`
crate is quite similar to the other `meta` crate. This minimal
duplication felt worth the additional safety provided by the new
assembler.

The `cranelift-assembler` crate is fuzzable (see the `README.md`). It
will generate instructions with randomized operands and compare their
encoding and pretty-printed string to a known-good disassembler,
currently `capstone`. This gives us confidence we previously didn't have
regarding emission. In the future, we may want to think through how to
fuzz (or otherwise check) the integration between `cranelift-codegen`
and this new assembler level.

[#41]: https://github.com/bytecodealliance/rfcs/pull/41

* asm: bless Cranelift file tests

Using the new assembler's pretty-printing results in slightly different
disassembly of compiled CLIF. This is because the assembler matches a
certain configuration of `capstone`, causing the following obvious
differences:

- instructions with only two operands only print two operands; the
original `MInst` instructions separate out the read-write operand into
two separate operands (SSA-like)
- the original instructions have some space padding after the
instruction mnemonic, those from the new assembler do not

This change uses the slightly new style as-is, but this is open for
debate; we can change the configuration of `capstone` that we fuzz
against. My only preferences would be to (1) retain some way to visually
distinguish the new assembler instructions in the disassembly
(temporarily, for debugging) and (2) eventually transition to
pretty-printing instructions in Intel-style (`rw, r`) instead of the
current (`r, rw`).

* ci: skip formatting when `rustfmt` not present

Though it is likely that `rustfmt` is present in a Rust environment,
some CI tasks do not have this tool installed. To handle this case
(plus the chance that other Wasmtime builds are similar), this change
skips formatting with a `stderr` warning when `rustfmt` fails.

* vet: audit `arbtest` for use as a dev-dependency

* ci: make assembler crates publishable

In order to satisfy `ci/publish.rs`, it would appear that we need to use
a version that matches the rest of the Cranelift crates.

* review: use Cargo workspace values

* review: document `Inst`, move `Inst::name`

* review: clarify 'earlier' doc comment

* review: document multi-byte opcodes

* review: document `Rex` builder methods

* review: document encoding rules

* review: clarify 'bits' -> 'width'

* review: clarify confusing legacy prefixes

* review: tweak IA-32e language

* review: expand documentation for format

* review: move feature list closer to enum

* review: add a TODO to remove AT&T operand ordering

* review: move prefix emission to separate lines

* review: add testing note

* review: fix incomplete sentence

* review: rename `MinusRsp` to `NonRspGpr`

* review: add TODO for commented out instructions

* review: add conservative down-conversion to `is_imm*`

* Fuzzing updates for cranelift-assembler-x64 (#10)

* Fuzzing updates for cranelift-assembler-x64

* Ensure fuzzers build on CI
* Move fuzz crate into the main workspace
* Move `fuzz.rs` support code directly into fuzzer
* Move `capstone` dependency into the fuzzer

* Make `arbitrary` an optional dependency

Shuffle around a few things in a few locations for this.

* vet: skip audit for `cranelift-assembler-x64-fuzz`

Co-authored-by: Alex Crichton <[email protected]>

* review: use 32-bit form for 8-bit and 16-bit reg-reg

Cranelift's existing lowering for 8-bit and 16-bit reg-reg `AND` used
the wider version of the instruction--the 32-bit reg-reg `AND`. As
pointed out by @cfallin [here], this was likely due to avoid partial
register stalls. This change keeps that lowering by distinguishing more
precisely between `GprMemImm` that are in register or memory.

[here]: https://github.com/bytecodealliance/wasmtime/pull/10110#discussion_r1939822255

* fix: skip `rustfmt` on generated code in more cases

Apparently `rustfmt` is not found on the `x86_64-unknown-illumos` build.
This change skips the action in this new case.

prtest:full

* fix: feed Cargo the meta crate version

This fixes errors with the `publish.rs` script.

prtest:full

---------

Co-authored-by: Alex Crichton <[email protected]>

show more ...