cranelift-codegen: rename most uses of std to core and alloc (#12237)

* rename most std uses to core and alloc

* cargo fmt

Remove `expect(clippy::allow_attributes_without_reason)` from cranelift-codegen (#11182)

* Remove `expect(clippy::allow_attributes_without_reason)` from cranelift-codegen

This commit gets around to

Remove `expect(clippy::allow_attributes_without_reason)` from cranelift-codegen (#11182)

* Remove `expect(clippy::allow_attributes_without_reason)` from cranelift-codegen

This commit gets around to migrating the `cranelift-codegen` crate to
require a reason on lint directives and additionally switch to
`#[expect]` where possible.

prtest:full

* Move x64-only item to x64 backend

show more ...

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtim

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtime to the 2024 Edition to keep
up-to-date with Rust idioms and access many of the edition features
exclusive to the 2024 edition.

prtest:full

* Reformat with the 2024 edition

show more ...

Add support for loading, storing and bitcasting small vectors on x64 and aarch64 (#10693)

Enable the `unsafe-op-in-unsafe-fn` lint (#10559)

* Enable the `unsafe-op-in-unsafe-fn` lint

This commit enables the `unsafe-op-in-unsafe-fn` lint in rustc for the
entire workspace. This lint will

Enable the `unsafe-op-in-unsafe-fn` lint (#10559)

* Enable the `unsafe-op-in-unsafe-fn` lint

This commit enables the `unsafe-op-in-unsafe-fn` lint in rustc for the
entire workspace. This lint will be warn-by-default in the 2024 edition
so this is intended to smooth the future migration to the new edition.

Many `unsafe` blocks were added in places the lint warned about, with
two major exceptions. The `wasmtime` and `wasmtime-c-api` crates simply
expect this lint to fire and effectively disable the lint. They're too
big at this time to do through this PR. My hope is that one day in the
future they'll be migrated, but more realistically that probably won't
happen so these crates just won't benefit from this lint.

* Fix nostd fiber build

prtest:full

* Fix build on Windows

* Fix asan build

show more ...

Enforce `uninlined_format_args` for the workspace (#9065)

* Enforce `uninlined_format_args` for the workspace

* fix: failing `Monolith Checks` job

* fix: formatting

Initial `f16` and `f128` support (#8860)

Update some CI dependencies (#7983)

* Update some CI dependencies

* Update to the latest nightly toolchain
* Update mdbook
* Update QEMU for cross-compiled testing
* Update `cargo nextest` for usag

Update some CI dependencies (#7983)

* Update some CI dependencies

* Update to the latest nightly toolchain
* Update mdbook
* Update QEMU for cross-compiled testing
* Update `cargo nextest` for usage with MIRI

prtest:full

* Remove lots of unnecessary imports

* Downgrade qemu as 8.2.1 seems to segfault

* Remove more imports

* Remove unused winch trait method

* Fix warnings about unused trait methods

* More unused imports

* More unused imports

show more ...

Remove unsigned variants of DataValue (#6218)

* remove unsigned variants of DataValue

* make value operation names more in-line with cranelift IR

Add `{u,s}{add,sub,mul}_overflow` instructions (#5784)

* add `{u,s}{add,sub,mul}_overflow` with interpreter

* add `{u,s}{add,sub,mul}_overflow` for x64

* add `{u,s}{add,sub,mul}_overflow` for aarc

Add `{u,s}{add,sub,mul}_overflow` instructions (#5784)

* add `{u,s}{add,sub,mul}_overflow` with interpreter

* add `{u,s}{add,sub,mul}_overflow` for x64

* add `{u,s}{add,sub,mul}_overflow` for aarch64

* 128bit filetests for `{u,s}{add,sub,mul}_overflow`

* `{u,s}{add,sub,mul}_overflow` emit tests for x64

* `{u,s}{add,sub,mul}_overflow` emit tests for aarch64

* Initial review changes

* add `with_flags_extended` helper

* add `with_flags_chained` helper

show more ...

cranelift: Add big and little endian memory accesses to interpreter (#5893)

* Added `mem_flags` parameter to `State::checked_{load,store}` as the means
for determining the endianness, typically deri

cranelift: Add big and little endian memory accesses to interpreter (#5893)

* Added `mem_flags` parameter to `State::checked_{load,store}` as the means
for determining the endianness, typically derived from an instruction.

* Added `native_endianness` property to `InterpreterState` as fallback when
determining endianness, such as in cases where there are no memory flags
avaiable or set.

* Added `to_be` and `to_le` methods to `DataValue`.

* Added `AtomicCas` and `AtomicRmw` to list of instructions with retrievable
memory flags for `InstructionData::memflags`.

* Enabled `atomic-{cas,rmw}-subword-{big,little}.clif` for interpreter run
tests.

show more ...

cranelift: Remove booleans (#5031)

Remove the boolean types from cranelift, and the associated instructions breduce, bextend, bconst, and bint. Standardize on using 1/0 for the return value from ins

cranelift: Remove booleans (#5031)

Remove the boolean types from cranelift, and the associated instructions breduce, bextend, bconst, and bint. Standardize on using 1/0 for the return value from instructions that produce scalar boolean results, and -1/0 for boolean vector elements.

Fixes #3205

Co-authored-by: Afonso Bordado <[email protected]>
Co-authored-by: Ulrich Weigand <[email protected]>
Co-authored-by: Chris Fallin <[email protected]>

show more ...

Make DataValue, not Ieee32/64, respect IEEE754 (#4860)

* cranelift-codegen: Remove all uses of DataValue

This type is only used by the interpreter, cranelift-fuzzgen, and
filetests. I haven't fo

Make DataValue, not Ieee32/64, respect IEEE754 (#4860)

* cranelift-codegen: Remove all uses of DataValue

This type is only used by the interpreter, cranelift-fuzzgen, and
filetests. I haven't found another convenient crate for those to all
depend on where this type can live instead, but this small refactor at
least makes it obvious that code generation does not in any way depend
on the implementation of this type.

* Make DataValue, not Ieee32/64, respect IEEE754

This fixes #4857 by partially reverting #4849.

It turns out that Ieee32 and Ieee64 need bitwise equality semantics so
they can be used as hash-table keys.

Moving the IEEE754 semantics up a layer to DataValue makes sense in
conjunction with #4855, where we introduced a DataValue::bitwise_eq
alternative implementation of equality for those cases where users of
DataValue still want the bitwise equality semantics.

* cranelift-interpreter: Use eq/ord from DataValue

This fixes #4828, again, now that the comparison operators on DataValue
have the right IEEE754 semantics.

* Add regression test from issue #4857

show more ...

cranelift: Bitwise compare fuzzgen results (#4855)

s390x: Support both big- and little-endian vector lane order (#4682)

This implements the s390x back-end portion of the solution for
https://github.com/bytecodealliance/wasmtime/issues/4566

We no

s390x: Support both big- and little-endian vector lane order (#4682)

This implements the s390x back-end portion of the solution for
https://github.com/bytecodealliance/wasmtime/issues/4566

We now support both big- and little-endian vector lane order
in code generation. The order used for a function is determined
by the function's ABI: if it uses a Wasmtime ABI, it will use
little-endian lane order, and big-endian lane order otherwise.
(This ensures that all raw_bitcast instructions generated by
both wasmtime and other cranelift frontends can always be
implemented as a no-op.)

Lane order affects the implementation of a number of operations:
- Vector immediates
- Vector memory load / store (in big- and little-endian variants)
- Operations explicitly using lane numbers
(insertlane, extractlane, shuffle, swizzle)
- Operations implicitly using lane numbers
(iadd_pairwise, narrow/widen, promote/demote, fcvt_low, vhigh_bits)

In addition, when calling a function using a different lane order,
we need to lane-swap all vector values passed or returned in registers.

A small number of changes to common code were also needed:

- Ensure we always select a Wasmtime calling convention on s390x
in crates/cranelift (func_signature).

- Fix vector immediates for filetests/runtests. In PR #4427,
I attempted to fix this by byte-swapping the V128 value, but
with the new scheme, we'd instead need to perform a per-lane
byte swap. Since we do not know the actual type in write_to_slice
and read_from_slice, this isn't easily possible.

Revert this part of PR #4427 again, and instead just mark the
memory buffer as little-endian when emitting the trampoline;
the back-end will then emit correct code to load the constant.

- Change a runtest in simd-bitselect-to-vselect.clif to no longer
make little-endian lane order assumptions.

- Remove runtests in simd-swizzle.clif that make little-endian
lane order assumptions by relying on implicit type conversion
when using a non-i16x8 swizzle result type (this feature should
probably be removed anyway).

Tested with both wasmtime and cg_clif.

show more ...

Allow 64-bit vectors and implement for interpreter (#4509)

* Allow 64-bit vectors and implement for interpreter

The AArch64 backend already supports 64-bit vectors; this simply allows
instructio

Allow 64-bit vectors and implement for interpreter (#4509)

* Allow 64-bit vectors and implement for interpreter

The AArch64 backend already supports 64-bit vectors; this simply allows
instructions to make use of that.

Implemented support for 64-bit vectors within the interpreter to allow
interpret runtests to use them.

Copyright (c) 2022 Arm Limited

* Disable 64-bit SIMD `iaddpairwise` tests on s390x

Copyright (c) 2022 Arm Limited

show more ...

s390x: Implement full SIMD support (#4427)

This adds full support for all Cranelift SIMD instructions
to the s390x target. Everything is matched fully via ISLE.

In addition to adding support fo

s390x: Implement full SIMD support (#4427)

This adds full support for all Cranelift SIMD instructions
to the s390x target. Everything is matched fully via ISLE.

In addition to adding support for many new instructions,
and the lower.isle code to match all SIMD IR patterns,
this patch also adds ABI support for vector types.
In particular, we now need to handle the fact that
vector registers 8 .. 15 are partially callee-saved,
i.e. the high parts of those registers (which correspond
to the old floating-poing registers) are callee-saved,
but the low parts are not. This is the exact same situation
that we already have on AArch64, and so this patch uses the
same solution (the is_included_in_clobbers callback).

The bulk of the changes are platform-specific, but there are
a few exceptions:

- Added ISLE extractors for the Immediate and Constant types,
to enable matching the vconst and swizzle instructions.

- Added a missing accessor for call_conv to ABISig.

- Fixed endian conversion for vector types in data_value.rs
to enable their use in runtests on the big-endian platforms.

- Enabled (nearly) all SIMD runtests on s390x. [ Two test cases
remain disabled due to vector shift count semantics, see below. ]

- Enabled all Wasmtime SIMD tests on s390x.

There are three minor issues, called out via FIXMEs below,
which should be addressed in the future, but should not be
blockers to getting this patch merged. I've opened the
following issues to track them:

- Vector shift count semantics
https://github.com/bytecodealliance/wasmtime/issues/4424

- is_included_in_clobbers vs. link register
https://github.com/bytecodealliance/wasmtime/issues/4425

- gen_constant callback
https://github.com/bytecodealliance/wasmtime/issues/4426

All tests, including all newly enabled SIMD tests, pass
on both z14 and z15 architectures.

show more ...

cranelift: Add support for i128 immediates in parser

cranelift: Fix big-endian regression in data_value.rs

PR https://github.com/bytecodealliance/wasmtime/pull/3187 introduced a
change to the write_to_slice and read_from_slice routines in
data_value.r

cranelift: Fix big-endian regression in data_value.rs

PR https://github.com/bytecodealliance/wasmtime/pull/3187 introduced a
change to the write_to_slice and read_from_slice routines in
data_value.rs that switched byte order on big-endian systems:
the code used to use native byte order, and now hard-codes
little-endian byte order.

Fix by using native byte order again.

show more ...

cranelift: Add stack support to the interpreter with virtual addresses (#3187)

* cranelift: Add stack support to the interpreter

We also change the approach for heap loads and stores.

Previous

cranelift: Add stack support to the interpreter with virtual addresses (#3187)

* cranelift: Add stack support to the interpreter

We also change the approach for heap loads and stores.

Previously we would use the offset as the address to the heap. However,
this approach does not allow using the load/store instructions to
read/write from both the heap and the stack.

This commit changes the addressing mechanism of the interpreter. We now
return the real addresses from the addressing instructions
(stack_addr/heap_addr), and instead check if the address passed into
the load/store instructions points to an area in the heap or the stack.

* cranelift: Add virtual addresses to cranelift interpreter

Adds a Virtual Addressing scheme that was discussed as a better
alternative to returning the real addresses.

The virtual addresses are split into 4 regions (stack, heap, tables and
global values), and the address itself is composed of an `entry` field
and an `offset` field. In general the `entry` field corresponds to the
instance of the resource (e.g. table5 is entry 5) and the `offset` field
is a byte offset inside that entry.

There is one exception to this which is the stack, where due to only
having one stack, the whole address is an offset field.

The number of bits in entry vs offset fields is variable with respect to
the `region` and the address size (32bits vs 64bits). This is done
because with 32 bit addresses we would have to compromise on heap size,
or have a small number of global values / tables. With 64 bit addresses
we do not have to compromise on this, but we need to support 32 bit
addresses.

* cranelift: Remove interpreter trap codes

* cranelift: Calculate frame_offset when entering or exiting a frame

* cranelift: Add safe read/write interface to DataValue

* cranelift: DataValue write full 128bit slot for booleans

* cranelift: Use DataValue accessors for trampoline.

show more ...

cranelift: Fix trampoline args for b1 types

Our DataValues only have one size of booleans so we are always going to
have this mismatch of sizes

cranelift: Add basic i128 support in interpreter

cranelift: Add support for i128 values in DataValue

Add comment why thiserror is not used

Remove thiserror dependency from cranelift_codegen