cranelift-assembler-x64: no_std support (#12235)

* cranelift-assembler-x64: no_std support

* cargo fmt

* Update cranelift/assembler-x64/src/xmm.rs

Co-authored-by: bjorn3 <[email protected]

cranelift-assembler-x64: no_std support (#12235)

* cranelift-assembler-x64: no_std support

* cargo fmt

* Update cranelift/assembler-x64/src/xmm.rs

Co-authored-by: bjorn3 <[email protected]>

* Update cranelift/assembler-x64/src/imm.rs

Co-authored-by: bjorn3 <[email protected]>

* Update cranelift/assembler-x64/src/api.rs

Co-authored-by: bjorn3 <[email protected]>

* Reorder import of alloc

* #![no_std]

* revert main.rs

* cargo fmt

* remove core feature

* Fix CI

---------

Co-authored-by: bjorn3 <[email protected]>

show more ...

x64: handle ISA features more completely (#11272)

* asm: generate boolean terms of CPU features

As discussed [here], we will soon need the ability to express more
complex combinations of CPU featur

x64: handle ISA features more completely (#11272)

* asm: generate boolean terms of CPU features

As discussed [here], we will soon need the ability to express more
complex combinations of CPU features. These are best expressed as
boolean terms: e.g., `(32-bit OR 64-bit) AND ...`, `(32-bit OR 64-bit)
AND ((AVX512VL AND AVX512F) OR AVX10.1)`. This change modifies the
generated code to have a `Inst::is_available` method which contains a
Rust-ified version of the instruction's boolean term. To do this, we now
pass in a `Features` trait, which the instruction can query to see if
its desired feature set is available.

[here]: https://bytecodealliance.zulipchat.com/#narrow/channel/217117-cranelift/topic/boolean.20terms.20for.20x64.20features

* x64: wire up `Inst::is_available` in Cranelift

This change makes us of the assembler's new generated
`Inst::is_available` methods to check an instruction's feature set in a
more succinct (and likely quicker) way. Unfortunately, this does not
allow us to print the missing ISA requirements on failure--something to
address later.

* Rename `Features` to `AvailableFeatures`

* Remove unused `InstructionSet`

* asm: fix all feature definitions

This is a mechanical transformation converting all instruction
definitions. Now, instructions should have a correct boolean term
describe the features required: e.g., `(_64b | compat) & avx`.

* x64: replace `use_*` with `has_*` when checking ISA features

In Cranelift, the `has_*` flags of `isa::x64::settings::Flags` indicate
that the CPU _has_ some capability; the `use_*` flags indicate that
Cranelift _should emit_ instructions using those capabilities. Further,
the `use_*` flags may turned on by the presence of more than one `has_*`
flags; e.g., when `has_avx` and `has_avx2` are available, `use_avx2` is
enabled.

Now that Cranelift's new x64 assembler understands boolean terms, we no
longer need the `use_*` flags for checking if an instruction can be
emitted: instead, we should use the `has_*` flags and rely on the logic
encoded in `Inst::is_available`.

* asm: materialize `Features` via `Inst::features`

For better error messages (and just for general use of CPU features, see
discussion [here]), this change adds `Inst::features`--a way to
explicitly examine the boolean term for an instruction. This function
returns a `&'static Features` that contains the `AND` and `OR` branches
defining when an instruction is available. This is all generated into
something that looks like:

```rust
pub fn features(&self) -> &'static Features {
const F1: &'static Features = &Features::Feature(Feature::_64b);
const F2: &'static Features = &Features::Feature(Feature::compat);
const F0: &'static Features = &Features::Or(F1, F2);
F0
}
```

This change makes use of `for_each_feature` more: we build up the
`AvailableFeatures` trait and the `Feature` enum from it. This should be
a bit more direct than searching through the generated code (?).

[here]: https://github.com/bytecodealliance/wasmtime/pull/11272#discussion_r2214465871

show more ...

x64: EVEX Encoding for new assembler (#11153)

* evex prefix for new assembler, vaddpd inst, prtest:full

* review: refactor `VexLength` and `EvexLength` into a single `Length`

* review: fix order o

x64: EVEX Encoding for new assembler (#11153)

* evex prefix for new assembler, vaddpd inst, prtest:full

* review: refactor `VexLength` and `EvexLength` into a single `Length`

* review: fix order of instruction definition to match manual

Also, adds some missing `.r()` helpers for consistency.

* review: remove opmask registers and mask kind for now

This will require more work to expose these as special,
runtime-available flags on the generated instruction.

* review: remove as-yet unknown operand patterns for EVEX

It's likely they will share logic with the VEX patterns but we can
refactor that in the future.

* review: fix up the `mmm` bit field, document

* fix: use new `Length` name in doc tests

* review: rename `VexW` to `WBit`

* Ensure VEX instructions cannot be defined with `Length::L512`

* Use `WBit` in REX instructions

---------

Co-authored-by: Andrew Brown <[email protected]>

show more ...

x64: Refactor emission of LoadExtName (#11124)

* x64: Expand symbol-related tests

* x64: Refactor emission of `LoadExtName`

Don't use raw bytes when emitting this instruction but instead use
symb

x64: Refactor emission of LoadExtName (#11124)

* x64: Expand symbol-related tests

* x64: Refactor emission of `LoadExtName`

Don't use raw bytes when emitting this instruction but instead use
symbolic assembler directives to avoid needing to hardcode bytes. This
additionally fixes an issue where with `colocated` symbols the offset of
the symbol was not taken into account (but for Wasmtime it's always been
0 so this otherwise hasn't come up so far)

* Fix emit tests

* Update test expectation

* Fuse `KnownOffsetTable` and `CodeSink` in the new assembler

Enables a `CodeSink` to know about all offsets, not just those against
labels. Resolves an issue where the fuzzing code sink was otherwise
unaware of `DeferredTarget::None`.

* Review comments

show more ...

x64: Migrate `mulx` to the new assembler (#10887)

* x64: Migrate `mulx` to the new assembler

This is an interesting instruction as it has a relatively unique shape
compared to many others. The VEX

x64: Migrate `mulx` to the new assembler (#10887)

* x64: Migrate `mulx` to the new assembler

This is an interesting instruction as it has a relatively unique shape
compared to many others. The VEX encoding is used to give it a 3-operand
form, although it still has an implicit 4th operand as well. The other
unique part about this instruction is that if the two write-only
operands are the same then that has a different semantic meaning than if
they are different.

Modeling the two-output form of the instruction was pretty easy, the
only changes needed were to add the `r32a` and `r32b` locations as
previously only `r32` was available. Modeling the one-output form of the
instruction led to a "hook" where these instructions specify that they
use a custom regalloc function. That skips the auto-generated regalloc
entirely and defers to a new `custom` submodule in the assembler crate.
These custom variants handle only having a single write operand on the
instruction.

* Fix fuzz build

show more ...

x64: convert `movq` (#10871)

* asm: add an `imm64` operand

* x64: add scalar moves

* asm: fix up `mov` immediates for Capstone

* x64: convert `movq` instruction

* pcc: remove ability to check `M

x64: convert `movq` (#10871)

* asm: add an `imm64` operand

* x64: add scalar moves

* asm: fix up `mov` immediates for Capstone

* x64: convert `movq` instruction

* pcc: remove ability to check `Mov64MR` moves

* x64: bless tests using `movq`

* winch: convert `movq` instruction

* pcc: disable tests for now

* review: model partial written-to locations with `w`

show more ...

Initial support for vex encoding for the new assembler. (#10754)

* Initial support for vex encoding for the new assembler.

* Addressing many PR comments: Remove ymm and zmm support, cleanup, etc

-

Initial support for vex encoding for the new assembler. (#10754)

* Initial support for vex encoding for the new assembler.

* Addressing many PR comments: Remove ymm and zmm support, cleanup, etc

---------

Co-authored-by: Andrew Brown <[email protected]>

show more ...

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtim

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtime to the 2024 Edition to keep
up-to-date with Rust idioms and access many of the edition features
exclusive to the 2024 edition.

prtest:full

* Reformat with the 2024 edition

show more ...

asm: implement write-only operands (#10759)

* asm: implement write-only operands

This change adds support to the new assembler for write-only operands.
This implementation appeared first in [#10754

asm: implement write-only operands (#10759)

* asm: implement write-only operands

This change adds support to the new assembler for write-only operands.
This implementation appeared first in [#10754] but is split out here to
unblock implementation of instructions that require it: multiplication,
conversions, moves, etc. This starts roughly the same as what was
implemented for write-only XMMs in [#10754] but includes support for
write-only GPRs as well and generates the temporary registers which are
needed.

[#10754]: https://github.com/bytecodealliance/wasmtime/pull/10754

Co-authored-by: Johnnie Birch <[email protected]>

* fix: use `to_reg()` to extract the Cranelift type from `Writable`

---------

Co-authored-by: Johnnie Birch <[email protected]>

show more ...

asm: refactor how REX prefixes are emitted (#10745)

This does not change functionality; instead, it reorganizes how REX
prefixes were encoded by the assembler to simplify code generation.
`RexFlags`

asm: refactor how REX prefixes are emitted (#10745)

This does not change functionality; instead, it reorganizes how REX
prefixes were encoded by the assembler to simplify code generation.
`RexFlags` now becomes the more-correct `RexPrefix` and constructors
like `RexPrefix::two_op` build the necessary byte for emission with
`RexPrefix::encode`.

show more ...

Inline assembler-x64 `generated_files` in `main.rs` (#10739)

The public function `generated_files` in `cranelift-assembler-x64` makes
the generated `rlib` non-deterministic because it contains the f

Inline assembler-x64 `generated_files` in `main.rs` (#10739)

The public function `generated_files` in `cranelift-assembler-x64` makes
the generated `rlib` non-deterministic because it contains the full
paths of generated files. But this function is only used in `main.rs` of
the same crate, so this change inlines it there to keep the library artifact
deterministic while maintaining the same behavior.

show more ...

asm: plumb fixed registers through (#10629)

* asm: plumb fixed registers through

This change introduces a new `Fixed` wrapper type that allows us to
plumb fixed registers through the assembler. The

asm: plumb fixed registers through (#10629)

* asm: plumb fixed registers through

This change introduces a new `Fixed` wrapper type that allows us to
plumb fixed registers through the assembler. These instructions are not
yet used by `cranelift-codegen`, but will be necessary at some point as
described in [#10238]. The end result of this is that up at the
`cranelift-codegen` level, fixed registers should look like
run-of-the-mill virtual registers using the appropriate types, but down
in `cranelift-assembler-x64` we panic if the register allocator does not
give them the correct register.

[#10238]: https://github.com/bytecodealliance/wasmtime/issues/10238

* fix: reduce `size_of::<Inst>` test

It is unclear why adding more `Fixed<R>` fields would somehow _reduce_
the size of `Inst`. What is quite clear is that I do not understand how
Rust chooses to lay out these fields.

show more ...

asm: allow printing of virtual registers (#10389)

* Rename module: `reg` to `gpr`

* Override `to_string` implementation for `cranelift-codegen` types

* Add pretty-printing tests

meta: move all ISLE-related generated code to `cranelift-codegen-meta` (#10352)

* meta: move all ISLE-related generated code to `cranelift-codegen-meta`

Previously, the new `cranelift-assembler-x64

meta: move all ISLE-related generated code to `cranelift-codegen-meta` (#10352)

* meta: move all ISLE-related generated code to `cranelift-codegen-meta`

Previously, the new `cranelift-assembler-x64` crate would generate the
assembler Rust code, the ISLE integration code, and the ISLE's
accompanying Rust macro all in the same crate. Because these last two
files were generated in the assembler's `OUT_DIR`, they caused issues
for deterministic builds that expect those files in
`cranelift-codegen`'s `OUT_DIR` (see #10348). This change fixes that by
moving all of the assembler's generated integration code, the ISLE
instruction definitions and their accompanying Rust macro, to the
`cranelift-codegen-meta` crate.

This has the added advantage that `cranelift-assembler-x64` now can
focus exclusively on being an assembler and not being concerned with the
ISLE integration, which is substantial.

* Move lint `allow` to problem site

Apparently allowing unused variables for unimplemented parts of fixed
register instructions will not work at the macro scope; this moves the
`allow` to the offending line.

* Remove `rustfmt` of generated assembler code

show more ...

asm: sse orpd implementation (#10273)

* sse orpd implementation

assembler integration with isle

format

add clippy reason, reorder avx priority in isle

bless tests for orpd

create separate xmm m

asm: sse orpd implementation (#10273)

* sse orpd implementation

assembler integration with isle

format

add clippy reason, reorder avx priority in isle

bless tests for orpd

create separate xmm module

validate function rewrite sse condition

add quote from manual for sse prefix

format changes

move Xmm bits under Reg

* use new isle constructors for sse

* remove unused function

* minor changes

show more ...

ci: generate a list of generated files (#10267)

* ci: generate a list of generated files

This fix is necessary for Windows users who may be using absolute-path
target directories: the previous solu

ci: generate a list of generated files (#10267)

* ci: generate a list of generated files

This fix is necessary for Windows users who may be using absolute-path
target directories: the previous solution, separating the paths by `:`,
runs into issues with Windows absolute paths (e.g., `C:\...`). This
change is similar to #10266 but should avoid any further OS
compatibility issues during a hypothetical cross-compilation.

prtest:full

* fix: debug string

show more ...

asm: switch to using workspace clippy settings (#10230)

* asm: switch to using workspace clippy settings

This removes all the custom clippy allowances and just adopts the
project's Clippy settings.

asm: switch to using workspace clippy settings (#10230)

* asm: switch to using workspace clippy settings

This removes all the custom clippy allowances and just adopts the
project's Clippy settings. This was suggested over in [#10216].

[#10216]: https://github.com/bytecodealliance/wasmtime/pull/10216#discussion_r1951135923

* fix: add one more allow reason

show more ...

asm: fix sign-extended immediates (#10216)

* asm: replace `emit_simm` with `Imm*::encode`

* asm: expand fuzz doc comment

* asm: pretty print sign-extended immediates

As mentioned in #10200, `caps

asm: fix sign-extended immediates (#10216)

* asm: replace `emit_simm` with `Imm*::encode`

* asm: expand fuzz doc comment

* asm: pretty print sign-extended immediates

As mentioned in #10200, `capstone` has a peculiar way of pretty-printing
immediates, especially signed immediates. It is simpler (and perhaps
more clear) for us to just print immediates in one consistent format:
`0xffff...`, e.g. This change parses capstone's pretty-printed
immediates and converts them to our simpler format if the first attempt
to match this assembler's output with `capston` fails.

* asm: add types for sign-extended immediates

As pointed out in #10200, it could be confusing for users for
`cranelift-assembler-x64` to pass unsigned integers to certain assembler
instructions and have them unexpectedly sign-extended. Well, it can't be
too surprising since these instructions have a `_sx*` suffix, but this
change implements @alexcrichton's additional suggestion to create
separate types for the immediates that may be sign-extended.

These new types (`Simm8`, `Simm16`, `Simm32`) are quite similar to their
vanilla counterparts (`Imm8`, `Imm16`, `Imm32`) but have additional
sign-extension logic when pretty-printed. This means the vanilla
versions can be simplified and the pre-existing `Simm32` is renamed to
the more appropriate `AmodeOffset`.

* clippy: fix casts, find pattern

show more ...

asm: re-allow prop-testing with `cargo test` (#10185)

In #10110, I originally intended to use `arbitrary` implementations in
two ways: for long-running fuzz testing (e.g., with OSS-Fuzz) but also
fo

asm: re-allow prop-testing with `cargo test` (#10185)

In #10110, I originally intended to use `arbitrary` implementations in
two ways: for long-running fuzz testing (e.g., with OSS-Fuzz) but also
for quick property testing with `cargo test`. This latter use case could
replace the tedious emit tests we had to write in `cranelift-codegen`
_and_ find corner cases that we otherwise might not explore. It helped
me during development: just run `cargo test` to check if anything is
obviously wrong. `arbtest` seemed to be able to run ~1000 test cases and
found mistakes well within the one second time limit I gave it.

@alexcrichton improved #10110 by avoiding `Arbitrary` implementations
everywhere and unconditionally depending on the `arbitrary` crate. This
was the right change, but it removed the ability to property test using
`cargo test`. What this change does is retain the general intent of his
change (no extra dependencies) but add `Arbitrary` implementations for
`cfg(test)` as well to run property tests during `cargo test`.

The only downside I see here is the added complexity when conditionally
compiling the fuzz-related bits: `#[cfg(any(test, feature = "fuzz"))]`.
Perhaps there is a better way to do this, but this seemed to work fine.
Let me know what you think.

show more ...

asm: prepare docs for publishing (#10178)

This change is motivated by staring at the output of `cargo doc` for the
`cranelift-assembler-x64` crate. In order to write a sensible top-level
example, I

asm: prepare docs for publishing (#10178)

This change is motivated by staring at the output of `cargo doc` for the
`cranelift-assembler-x64` crate. In order to write a sensible top-level
example, I felt it was best to refactor how we construct the
instructions: this removes the `build` module and adds conventional
`<inst_name>::new` functions to each instruction. Also, a generated
`From` implementation makes it easier to convert to an `Inst`.

This change has other doc-related refactorings and tweaks, but should
not change any functionality.

show more ...

asm: introduce a new x64 assembler (#10110)

* asm: add initial infrastructure for an external assembler

This change adds some initial logic implementing an external assembler
for Cranelift's x64 ba

asm: introduce a new x64 assembler (#10110)

* asm: add initial infrastructure for an external assembler

This change adds some initial logic implementing an external assembler
for Cranelift's x64 backend, as proposed in RFC [#41].

This adds two crates:
- the `cranelift/assembler/meta` crate defines the instructions; to
print out the defined instructions use `cargo run -p
cranelift-assembler-meta`
- the `cranelift/assembler` crate exposes the generated Rust code for
those instructions; to see the path to the generated code use `cargo
run -p cranelift-assembler`

The assembler itself is straight-forward enough (modulo the code
generation, of course); its integration into `cranelift-codegen` is what
is most tricky about this change. Instructions that we will emit in the
new assembler are contained in the `Inst::External` variant. This
unfortunately increases the memory size of `Inst`, but only temporarily
if we end up removing the extra `enum` indirection by adopting the new
assembler wholesale. Another integration point is ISLE: we generate ISLE
definitions and a Rust helper macro to make the external assembler
instructions accessible to ISLE lowering.

This change introduces some duplication: the encoding logic (e.g. for
REX instructions) currently lives both in `cranelift-codegen` and the
new assembler crate. The `Formatter` logic for the assembler `meta`
crate is quite similar to the other `meta` crate. This minimal
duplication felt worth the additional safety provided by the new
assembler.

The `cranelift-assembler` crate is fuzzable (see the `README.md`). It
will generate instructions with randomized operands and compare their
encoding and pretty-printed string to a known-good disassembler,
currently `capstone`. This gives us confidence we previously didn't have
regarding emission. In the future, we may want to think through how to
fuzz (or otherwise check) the integration between `cranelift-codegen`
and this new assembler level.

[#41]: https://github.com/bytecodealliance/rfcs/pull/41

* asm: bless Cranelift file tests

Using the new assembler's pretty-printing results in slightly different
disassembly of compiled CLIF. This is because the assembler matches a
certain configuration of `capstone`, causing the following obvious
differences:

- instructions with only two operands only print two operands; the
original `MInst` instructions separate out the read-write operand into
two separate operands (SSA-like)
- the original instructions have some space padding after the
instruction mnemonic, those from the new assembler do not

This change uses the slightly new style as-is, but this is open for
debate; we can change the configuration of `capstone` that we fuzz
against. My only preferences would be to (1) retain some way to visually
distinguish the new assembler instructions in the disassembly
(temporarily, for debugging) and (2) eventually transition to
pretty-printing instructions in Intel-style (`rw, r`) instead of the
current (`r, rw`).

* ci: skip formatting when `rustfmt` not present

Though it is likely that `rustfmt` is present in a Rust environment,
some CI tasks do not have this tool installed. To handle this case
(plus the chance that other Wasmtime builds are similar), this change
skips formatting with a `stderr` warning when `rustfmt` fails.

* vet: audit `arbtest` for use as a dev-dependency

* ci: make assembler crates publishable

In order to satisfy `ci/publish.rs`, it would appear that we need to use
a version that matches the rest of the Cranelift crates.

* review: use Cargo workspace values

* review: document `Inst`, move `Inst::name`

* review: clarify 'earlier' doc comment

* review: document multi-byte opcodes

* review: document `Rex` builder methods

* review: document encoding rules

* review: clarify 'bits' -> 'width'

* review: clarify confusing legacy prefixes

* review: tweak IA-32e language

* review: expand documentation for format

* review: move feature list closer to enum

* review: add a TODO to remove AT&T operand ordering

* review: move prefix emission to separate lines

* review: add testing note

* review: fix incomplete sentence

* review: rename `MinusRsp` to `NonRspGpr`

* review: add TODO for commented out instructions

* review: add conservative down-conversion to `is_imm*`

* Fuzzing updates for cranelift-assembler-x64 (#10)

* Fuzzing updates for cranelift-assembler-x64

* Ensure fuzzers build on CI
* Move fuzz crate into the main workspace
* Move `fuzz.rs` support code directly into fuzzer
* Move `capstone` dependency into the fuzzer

* Make `arbitrary` an optional dependency

Shuffle around a few things in a few locations for this.

* vet: skip audit for `cranelift-assembler-x64-fuzz`

Co-authored-by: Alex Crichton <[email protected]>

* review: use 32-bit form for 8-bit and 16-bit reg-reg

Cranelift's existing lowering for 8-bit and 16-bit reg-reg `AND` used
the wider version of the instruction--the 32-bit reg-reg `AND`. As
pointed out by @cfallin [here], this was likely due to avoid partial
register stalls. This change keeps that lowering by distinguishing more
precisely between `GprMemImm` that are in register or memory.

[here]: https://github.com/bytecodealliance/wasmtime/pull/10110#discussion_r1939822255

* fix: skip `rustfmt` on generated code in more cases

Apparently `rustfmt` is not found on the `x86_64-unknown-illumos` build.
This change skips the action in this new case.

prtest:full

* fix: feed Cargo the meta crate version

This fixes errors with the `publish.rs` script.

prtest:full

---------

Co-authored-by: Alex Crichton <[email protected]>

show more ...