Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtim

Update Wasmtime to the 2024 Rust Edition (#10806)

* Update Wasmtime to the 2024 Rust Edition

Now that our MSRV supports the 2024 edition it's possible to make this
switch. This commit moves Wasmtime to the 2024 Edition to keep
up-to-date with Rust idioms and access many of the edition features
exclusive to the 2024 edition.

prtest:full

* Reformat with the 2024 edition

show more ...

Typo (#10563)

Decouple `serde` from its `derive` crate (#6917)

By not activating the `derive` feature on `serde`, the compilation speed
can be improved by a lot. This is because `serde` can then compile in
parall

Decouple `serde` from its `derive` crate (#6917)

By not activating the `derive` feature on `serde`, the compilation speed
can be improved by a lot. This is because `serde` can then compile in
parallel to `serde_derive`, allowing it to finish compilation possibly
even before `serde_derive`, unblocking all the crates waiting for
`serde` to start compiling much sooner.

As it turns out the main deciding factor for how long the compile time of a
project is, is primarly determined by the depth of dependencies rather
than the width. In other words, a crate's compile times aren't affected
by how many crates it depends on, but rather by the longest chain of
dependencies that it needs to wait on. In many cases `serde` is part of
that long chain, as it is part of a long chain if the `derive` feature
is active:

`proc-macro2` compile build script > `proc-macro2` run build script >
`proc-macro2` > `quote` > `syn` > `serde_derive` > `serde` >
`serde_json` (or any crate that depends on serde)

By decoupling it from `serde_derive`, the chain is shortened and compile
times get much better.

Check this issue for a deeper elaboration:
https://github.com/serde-rs/serde/issues/2584

For `wasmtime` I'm seeing a reduction from 24.75s to 22.45s when
compiling in `release` mode. This is because wasmtime through `gimli`
has a dependency on `indexmap` which can only start compiling when
`serde` is finished, which you want to happen as early as possible so
some of wasmtime's dependencies can start compiling.

To measure the full effect, the dependencies can't by themselves
activate the `derive` feature. I've upstreamed a patch for
`fxprof-processed-profile` which was the only dependency that activated
it for `wasmtime` (not yet published to crates.io). `wasmtime-cli` and
co. may need patches for their dependencies to see a similar
improvement.

show more ...

x64: Add non-SSSE3 lowerings of `pshufb` (#6606)

* x64: Add non-SSSE3 lowerings of `pshufb`

Or, more accurately, add lowerings which don't use `pshufb`'s
functionality at all where possible or othe

x64: Add non-SSSE3 lowerings of `pshufb` (#6606)

* x64: Add non-SSSE3 lowerings of `pshufb`

Or, more accurately, add lowerings which don't use `pshufb`'s
functionality at all where possible or otherwise fall back to a new
libcall. This particular instruction seemed uniquely difficult to
implement in the backend so I decided to "cop out" and use libcall
instead. The libcall will be used for `popcnt`, `shuffle`, and
`swizzle` instructions when SSSE3 isn't available.

* Implemente SSE2 popcnt with Hacker's Delight

* x64: Implement passing vector arguments in the fastcall convention

Windows says that vector arguments are passed indirectly so handle that
here through the `ABIArg::ImplicitPtrArg` variant. Some additional
handling is added to the general machinst backend.

* Update `gen_load_base_offset` for x64

* Fill out remaining bits of fastcall and vector parameters

* Remove now-unnecessary `Clone` bound

show more ...

Handle signature() for more libcalls (#6174)

* Handle signature() for more libcalls

This is necessary to be able to call them in the interpreter. All the
remaining libcalls which signature() doesn'

Handle signature() for more libcalls (#6174)

* Handle signature() for more libcalls

This is necessary to be able to call them in the interpreter. All the
remaining libcalls which signature() doesn't handle are never used in
clif ir. Only in code compiled by a backend.

* Fix libcall declarations in cranelift-frontend

* Add function signatures

* Use correct pointer type instead of I64

show more ...

cranelift: Add LibCalls to the interpreter (#4782)

* cranelift: Add libcall handlers to interpreter

* cranelift: Fuzz IshlI64 libcall

* cranelift: Revert back to fuzzing udivi64

* cranelift

cranelift: Add LibCalls to the interpreter (#4782)

* cranelift: Add libcall handlers to interpreter

* cranelift: Fuzz IshlI64 libcall

* cranelift: Revert back to fuzzing udivi64

* cranelift: Use sdiv as a fuzz libcall

* cranelift: Register Sdiv in fuzzgen

* cranelift: Add multiple libcalls to fuzzer

* cranelift: Register a single libcall handler

* cranelift: Simplify args checking in interpreter

* cranelift: Remove unused LibCalls

* cranelift: Cleanup interpreter libcall types

* cranelift: Fix Interpreter Docs

show more ...

Implement an incremental compilation cache for Cranelift (#4551)

This is the implementation of https://github.com/bytecodealliance/wasmtime/issues/4155, using the "inverted API" approach suggested b

Implement an incremental compilation cache for Cranelift (#4551)

This is the implementation of https://github.com/bytecodealliance/wasmtime/issues/4155, using the "inverted API" approach suggested by @cfallin (thanks!) in Cranelift, and trait object to provide a backend for an all-included experience in Wasmtime.

After the suggestion of Chris, `Function` has been split into mostly two parts:

- on the one hand, `FunctionStencil` contains all the fields required during compilation, and that act as a compilation cache key: if two function stencils are the same, then the result of their compilation (`CompiledCodeBase<Stencil>`) will be the same. This makes caching trivial, as the only thing to cache is the `FunctionStencil`.
- on the other hand, `FunctionParameters` contain the... function parameters that are required to finalize the result of compilation into a `CompiledCode` (aka `CompiledCodeBase<Final>`) with proper final relocations etc., by applying fixups and so on.

Most changes are here to accomodate those requirements, in particular that `FunctionStencil` should be `Hash`able to be used as a key in the cache:

- most source locations are now relative to a base source location in the function, and as such they're encoded as `RelSourceLoc` in the `FunctionStencil`. This required changes so that there's no need to explicitly mark a `SourceLoc` as the base source location, it's automatically detected instead the first time a non-default `SourceLoc` is set.
- user-defined external names in the `FunctionStencil` (aka before this patch `ExternalName::User { namespace, index }`) are now references into an external table of `UserExternalNameRef -> UserExternalName`, present in the `FunctionParameters`, and must be explicitly declared using `Function::declare_imported_user_function`.
- some refactorings have been made for function names:
- `ExternalName` was used as the type for a `Function`'s name; while it thus allowed `ExternalName::Libcall` in this place, this would have been quite confusing to use it there. Instead, a new enum `UserFuncName` is introduced for this name, that's either a user-defined function name (the above `UserExternalName`) or a test case name.
- The future of `ExternalName` is likely to become a full reference into the `FunctionParameters`'s mapping, instead of being "either a handle for user-defined external names, or the thing itself for other variants". I'm running out of time to do this, and this is not trivial as it implies touching ISLE which I'm less familiar with.

The cache computes a sha256 hash of the `FunctionStencil`, and uses this as the cache key. No equality check (using `PartialEq`) is performed in addition to the hash being the same, as we hope that this is sufficient data to avoid collisions.

A basic fuzz target has been introduced that tries to do the bare minimum:

- check that a function successfully compiled and cached will be also successfully reloaded from the cache, and returns the exact same function.
- check that a trivial modification in the external mapping of `UserExternalNameRef -> UserExternalName` hits the cache, and that other modifications don't hit the cache.
- This last check is less efficient and less likely to happen, so probably should be rethought a bit.

Thanks to both @alexcrichton and @cfallin for your very useful feedback on Zulip.

Some numbers show that for a large wasm module we're using internally, this is a 20% compile-time speedup, because so many `FunctionStencil`s are the same, even within a single module. For a group of modules that have a lot of code in common, we get hit rates up to 70% when they're used together. When a single function changes in a wasm module, every other function is reloaded; that's still slower than I expect (between 10% and 50% of the overall compile time), so there's likely room for improvement.

Fixes #4155.

show more ...

s390x: Implement tls_value (#4616)

Implement the tls_value for s390 in the ELF general-dynamic mode.

Notable differences to the x86_64 implementation are:
- We use a __tls_get_offset libcall ins

s390x: Implement tls_value (#4616)

Implement the tls_value for s390 in the ELF general-dynamic mode.

Notable differences to the x86_64 implementation are:
- We use a __tls_get_offset libcall instead of __tls_get_addr.
- The current thread pointer (stored in a pair of access registers)
needs to be added to the result of __tls_get_offset.
- __tls_get_offset has a variant ABI that requires the address of
the GOT (global offset table) is passed in %r12.

This means we need a new libcall entries for __tls_get_offset.
In addition, we also need a way to access _GLOBAL_OFFSET_TABLE_.
The latter is a "magic" symbol with a well-known name defined
by the ABI and recognized by the linker. This patch introduces
a new ExternalName::KnownSymbol variant to support such names
(originally due to @afonso360).

We also need to emit a relocation on a symbol placed in a
constant pool, as well as an extra relocation on the call
to __tls_get_offset required for TLS linker optimization.

Needed by the cg_clif frontend.

show more ...

cranelift: Implement scalar FMA on x86 (#4460)

x86 does not have dedicated instructions for scalar FMA, lower
to a libcall which seems to be what llvm does.

Cranelift: Add `LibCall::Memcmp`

The comment says the enum is "likely to grow" and the function's been in libc since C89, so hopefully this is ok.

I'd like to use it for emitting things like array

Cranelift: Add `LibCall::Memcmp`

The comment says the enum is "likely to grow" and the function's been in libc since C89, so hopefully this is ok.

I'd like to use it for emitting things like array equality.

show more ...

Remove more old backend ISA concepts (#3402)

This also paves the way for unifying TargetIsa and MachBackend, since now they map one to one. In theory the two traits could be merged, which would be n

Remove more old backend ISA concepts (#3402)

This also paves the way for unifying TargetIsa and MachBackend, since now they map one to one. In theory the two traits could be merged, which would be nice to limit the number of total concepts. Also they have quite different responsibilities, so it might be fine to keep them separate.

Interestingly, this PR started as removing RegInfo from the TargetIsa trait since the adapter returned a dummy value there. From the fallout, noticed that all Display implementations didn't needed an ISA anymore (since these were only used to render ISA specific registers). Also the whole family of RegInfo / ValueLoc / RegUnit was exclusively used for the old backend, and these could be removed. Notably, some IR instructions needed to be removed, because they were using RegUnit too: this was the oddball of regfill / regmove / regspill / copy_special, which were IR instructions inserted by the old regalloc. Fare thee well!

show more ...

Remove ancient register allocation (#3401)

Implement GOT relocations in SimpleJIT

cranelift: add i64.{ishl,ushr,ashr} libcalls.

These libcalls are useful for 32-bit platforms.

On x86_32 in particular, commit 4ec16fa0 added support for legalizing
64-bit shifts through SIMD operat

cranelift: add i64.{ishl,ushr,ashr} libcalls.

These libcalls are useful for 32-bit platforms.

On x86_32 in particular, commit 4ec16fa0 added support for legalizing
64-bit shifts through SIMD operations. However, that legalization
requires SIMD to be enabled and SSE 4.1 to be supported, which is not
acceptable as a hard requirement.

show more ...

cranelift: add i64.[us]{div,rem} libcalls.

These libcalls are useful for 32-bit platforms.

Add TLS support for ELF and MachO (#1174)

* Add TLS support
* Add binemit and legalize tests
* Spill all caller-saved registers when necessary

Make `get_libcall_funcref` `pub(crate)` (#1291)

* Make `get_libcall_funcref` `pub(crate)`

Closes https://github.com/bytecodealliance/cranelift/issues/1273.

Since get_libcall_funcref is only us

Make `get_libcall_funcref` `pub(crate)` (#1291)

* Make `get_libcall_funcref` `pub(crate)`

Closes https://github.com/bytecodealliance/cranelift/issues/1273.

Since get_libcall_funcref is only used internally by the verifier,
it doesn't make sense to have it be public. This will encourage users to
look elsewhere for `memcpy` (they should be looking at
https://docs.rs/cranelift-frontend/0.51.0/cranelift_frontend/struct.FunctionBuilder.html#method.emit_small_memcpy)

show more ...

Renames Settings ⚠️ (fixes #976) (#1321)

This is a breaking API change: the following settings have been renamed:

- jump_tables_enabled -> enable_jump_tables
- colocated_libcalls -> use_colocat

Renames Settings ⚠️ (fixes #976) (#1321)

This is a breaking API change: the following settings have been renamed:

- jump_tables_enabled -> enable_jump_tables
- colocated_libcalls -> use_colocated_libcalls
- probestack_enabled -> enable_probestack
- allones_funcaddrs -> emit_all_ones_funcaddrs

show more ...

Fix clippy warnings.

This commit fixes the current set of (stable) clippy warnings in the repo.

Always use extern crate std in cranelift-codegen

Add cranelift-object

Implement a Windows Baldrdash calling convention;

Adds the libcall_call_conv setting and use it for libcall calls expansion;

Add support for some serde serialization (#847)

* Add support for some serde serialization

Add the dyn keyword before trait objects;