Make profiling more amenable to other in-process engines (#11865)Both the perfmap and jitdump protocols for profiling JITs require thatthere's a single file with metadata per-process. This has the
Make profiling more amenable to other in-process engines (#11865)Both the perfmap and jitdump protocols for profiling JITs require thatthere's a single file with metadata per-process. This has theunfortunate side effect of if there are multiple engines in the sameprocess there's no real way for them to coordinate when writing outrecords to this file. In an attempt to at least try to synchronize thiscommit updates these two implementations to issue a single `write`syscall with the full contents of the record to a file opened in`O_APPEND` mode.If all the stars align and a partial write doesn't happen then thatmeans we're doing our best effort to cooperate with other engines in thesame process. If a partial write happens the decision in this PR is togo ahead and retry the rest of the write (using the `write_all` helper).This is a tradeoff where it makes the single-engine use case more robust(partial writes are handled correctly) but the multi-engine use casewill produce corrupt files. It doesn't feel like a great tradeoff to saythat profiling can't be done in Wasmtime when a partial write wouldhappen for this niche use case of multiple engines, hence the directionof this tradeoff.Closes #11862
show more ...
Allow compiling debug-builtins in no_std environment to enable gdb jit (#11304)* Allow compiling debug-builtins in no_std environment to enable gdb jit- Make jit-debug crate #![no_std] when `std`
Allow compiling debug-builtins in no_std environment to enable gdb jit (#11304)* Allow compiling debug-builtins in no_std environment to enable gdb jit- Make jit-debug crate #![no_std] when `std` feature is not enabled- Make gdb_jit_int module build in no_std environment- Use std when perf_jitdebug is enabled because the perf_jitdebug module does not run in no_std environment- Change debug-builtins feature to only enable gdb_jit_int- The perf_jitdebug feature is enabled in profiling mode onlySigned-off-by: Doru Blânzeanu <[email protected]>* Remove feature listing from `default`These are otherwise present via `std`.* Shuffle around some features* Enable `wasmtime-jit-debug/std` from wasmtime's `std` feature* Only enable `perf_jitdump` from `profiling` as that's all that's explicitly needed.* Remove empty features list* Cut down on some `#[cfg]` usage* Update comments in gdb_jit_int module* Add CI checks that `debug-builtins` builds on no_std* Fix missing has_host_compiler_backend when debug-builtins enabledSigned-off-by: Doru Blânzeanu <[email protected]>---------Signed-off-by: Doru Blânzeanu <[email protected]>Co-authored-by: Alex Crichton <[email protected]>
More clearly flag internal crates as such (#10963)* More clearly flag internal crates as suchThis commit is an attempt to more clearly flag internal crates in thisproject as internal and not int
More clearly flag internal crates as such (#10963)* More clearly flag internal crates as suchThis commit is an attempt to more clearly flag internal crates in thisproject as internal and not intended for external use. Specifically:* Many crates are renamed from `wasmtime-foo` to `wasmtime-internal-foo`.* All of these crates now have `INTERNAL: ...` in their crates.io description.* All of these crates now have a warning at the top of their documentation discouraging use.This change is a result of rustsec/advisory-db#1999 where the goal is tobe crystal clear from a project perspective that usage of these cratesare highly discouraged and not supported. We'll still probably get suchadvisories but we won't be considering them CVEs from the project itselfdue to the internal nature of these crates and the discouragingwarnings.Some concrete changes used here are:* Inter-crate dependencies still use `wasmtime_foo` for naming and do so with Cargo's package-renaming features.* Crate renames are specified at the workspace level so the rename is only in one locations and all other inherit it.* Contribution documentation now has some brief guidelines about crate organization.* Update vet config* Update checks for wasmtime-fiberprtest:full* Update publish script* Another fiber rename* Fix some doc tests
Duplicate page size determination in `wasmtime-fiber` (#10803)* Duplicate page size determination in `wasmtime-fiber`Currently Wasmtime has a function `crate::runtime::vm::host_page_size`but thi
Duplicate page size determination in `wasmtime-fiber` (#10803)* Duplicate page size determination in `wasmtime-fiber`Currently Wasmtime has a function `crate::runtime::vm::host_page_size`but this isn't reachable from the `wasmtime-fiber` crate and instead thacrate uses `rustix::param::page_size` to determine the host page size.It looks like this usage of `rustix` is causing a panic in #10802.Ideally `wasmtime-fiber` would be able to use the same function but thecrate separation does not currently make that feasible. For nowduplicate the logic of `wasmtime` into `wasmtime-fiber` as it's modestenough to ensure that this does not panic.Closes #10802* Run full test suite in CIprtest:full
Enable the `unsafe-op-in-unsafe-fn` lint (#10559)* Enable the `unsafe-op-in-unsafe-fn` lintThis commit enables the `unsafe-op-in-unsafe-fn` lint in rustc for theentire workspace. This lint will
Enable the `unsafe-op-in-unsafe-fn` lint (#10559)* Enable the `unsafe-op-in-unsafe-fn` lintThis commit enables the `unsafe-op-in-unsafe-fn` lint in rustc for theentire workspace. This lint will be warn-by-default in the 2024 editionso this is intended to smooth the future migration to the new edition.Many `unsafe` blocks were added in places the lint warned about, withtwo major exceptions. The `wasmtime` and `wasmtime-c-api` crates simplyexpect this lint to fire and effectively disable the lint. They're toobig at this time to do through this PR. My hope is that one day in thefuture they'll be migrated, but more realistically that probably won'thappen so these crates just won't benefit from this lint.* Fix nostd fiber buildprtest:full* Fix build on Windows* Fix asan build
Enable `missing-unsafe-on-extern` lint (#9963)* Enable `missing-unsafe-on-extern` lintThis'll be a hard error in the 2024 edition so go ahead and opt-in to itnow to ease our future transition.
Enable `missing-unsafe-on-extern` lint (#9963)* Enable `missing-unsafe-on-extern` lintThis'll be a hard error in the 2024 edition so go ahead and opt-in to itnow to ease our future transition.* Fix adapter build* Fix custom c-api build* Fix fuzzer build* Fix some Windows `extern` blocks
Move gdbjit C helpers to a separate file (#9939)* Move gdbjit C helpers to a separate fileThis commit splits out the gdbjit-related helpers from `helpers.c` inWasmtime to a separate C file built
Move gdbjit C helpers to a separate file (#9939)* Move gdbjit C helpers to a separate fileThis commit splits out the gdbjit-related helpers from `helpers.c` inWasmtime to a separate C file built as part of the `wasmtime-jit-debug`crate. This'll help excise these helpers if gdbjit support is disabledat compile time and additionally brings them closer to the actualdefinition in the `wasmtime-jit-debug` crate.* Fix linkage issues* Fix miri tests
Leverage Rust 1.79, 1.80 (#9498)This commit is a follow-up to #9496 to leverage various APIs that theworkspace now has access to. For example most dependencies on the`once_cell` crate are now rem
Leverage Rust 1.79, 1.80 (#9498)This commit is a follow-up to #9496 to leverage various APIs that theworkspace now has access to. For example most dependencies on the`once_cell` crate are now removed in favor of the types stabilized inthe standard library: `LazyLock` and `LazyCell`. One dependency remainsin the `wasmtime` crate due to the `get_or_try_init` not being stableyet.Some additional helper methods on raw pointer slices are also availablefor removing a few minor `unsafe` blocks.
Add `rust-version.workspace = true` to all crates (#9112)Right now this is only on some crates such as `wasmtime` itself and`wasmtime-cli`, but by applying it to all crates it helps with versions
Add `rust-version.workspace = true` to all crates (#9112)Right now this is only on some crates such as `wasmtime` itself and`wasmtime-cli`, but by applying it to all crates it helps with versionselection of those using just Cranelift for example.
Refactor internal profiling APIs to be safer (#8914)Pass around `&[u8]` instead of `*const u8` and `usize` to avoid the needfor raw unsafe abstractions.Closes #8905
Fix some typos (#8641)* occurred* winch typos* tests typos* cli typos* fuzz typos* examples typos* docs typos* crates/wasmtime typos* crates/environ typos* crates/cranelift typos
Fix some typos (#8641)* occurred* winch typos* tests typos* cli typos* fuzz typos* examples typos* docs typos* crates/wasmtime typos* crates/environ typos* crates/cranelift typos* crates/test-programs typos* crates/c-api typos* crates/cache typos* crates other typos* cranelift/codegen/src/isa typos* cranelift/codegen/src other typos* cranelift/codegen other typos* cranelift other typos* ci js typo* .github workflows typo* RELEASES typo* Fix clang-format documentation line---------Co-authored-by: Andrew Brown <[email protected]>
Configure workspace lints, enable running some Clippy lints on CI (#7561)* Configure Rust lints at the workspace levelThis commit adds necessary configuration knobs to have lints configuredat th
Configure workspace lints, enable running some Clippy lints on CI (#7561)* Configure Rust lints at the workspace levelThis commit adds necessary configuration knobs to have lints configuredat the workspace level in Wasmtime rather than the crate level. Thisuses a feature of Cargo first released with 1.74.0 (last week) of the`[workspace.lints]` table. This should help create a more consistent setof lints applied across all crates in our workspace in addition topossibly running select clippy lints on CI as well.* Move `unused_extern_crates` to the workspace levelThis commit configures a `deny` lint level for the`unused_extern_crates` lint to the workspace level rather than theprevious configuration at the individual crate level.* Move `trivial_numeric_casts` to workspace level* Change workspace lint levels to `warn`CI will ensure that these don't get checked into the codebase andotherwise provide fewer speed bumps for in-process development.* Move `unstable_features` lint to workspace level* Move `unused_import_braces` lint to workspace level* Start running Clippy on CIThis commit configures our CI to run `cargo clippy --workspace` for allmerged PRs. Historically this hasn't been all the feasible due to theamount of configuration required to control the number of warnings onCI, but with Cargo's new `[lint]` table it's possible to have aone-liner to silence all lints from Clippy by default. This commit bydefault sets the `all` lint in Clippy to `allow` to by-default disablewarnings from Clippy. The goal of this PR is to enable selective accessto Clippy lints for Wasmtime on CI.* Selectively enable `clippy::cast_sign_loss`This would have fixed #7558 so try to head off future issues with thatby warning against this situation in a few crates. This lint is stillquite noisy though for Cranelift for example so it's not worthwhile atthis time to enable it for the whole workspace.* Fix CI errorprtest:full
Support multiple versions of `wasmtime` in the same crate (#6673)* Attempt versioned exports to facilitate having multiple versions in the same crate* Modify approach to use `export_name` and `li
Support multiple versions of `wasmtime` in the same crate (#6673)* Attempt versioned exports to facilitate having multiple versions in the same crate* Modify approach to use `export_name` and `link_name`* Only apply version to names in assembly and foreign item fns* Attempt to handle the s390x case* Fix alignment of backslashes in assembly file* Pretend I understand the preprocessor* Version symbols in `crates/runtime/src/helpers.c`* Stop versioning `__jit_debug_register_code` because gdb relies on it and it is uses `weak` linkage* Version symbol in `crates/fiber/src/windows.c`* Consolidate `LitStr` creation in macro* Add new crate to publish script and supply-chain config* Fix order in supply chain config* Set `audit-as-crates-io` to false* Missing `versioned_link` for Windows* Version strings used in debug* Formatting* Get rid of `versioned_str` and bring back `versioned_suffix`---------Co-authored-by: Alex Crichton <[email protected]>
Remove `maintenance` badges from the Cargo.toml files. (#6286)Several of these badges were out of date, with some crates in wide productionuse marked as "experimental". Insted of trying to keep th
Remove `maintenance` badges from the Cargo.toml files. (#6286)Several of these badges were out of date, with some crates in wide productionuse marked as "experimental". Insted of trying to keep them up to date, justremove them, since they are [no longer displayed on crates.io].[no longer displayed on crates.io]: https://doc.rust-lang.org/cargo/reference/manifest.html#the-badges-section
Add support for generating perf maps for simple perf profiling (#6030)* Add support for generating perf maps for simple perf profiling* add missing enum entry in C code* bugfix: use hexa when p
Add support for generating perf maps for simple perf profiling (#6030)* Add support for generating perf maps for simple perf profiling* add missing enum entry in C code* bugfix: use hexa when printing the code region's length too (thanks bjorn3!)* sanitize file name + use bufwriter* introduce --profile CLI flag for wasmtime* Update doc and doc comments for new --profile option* remove redundant FromStr import* Apply review feedback: make_line receives a Write impl, report errors* fix tests?* better docs
Leverage Cargo's workspace inheritance feature (#4905)* Leverage Cargo's workspace inheritance feature This commit is an attempt to reduce the complexity of the Cargo manifests in this reposito
Leverage Cargo's workspace inheritance feature (#4905)* Leverage Cargo's workspace inheritance feature This commit is an attempt to reduce the complexity of the Cargo manifests in this repository with Cargo's workspace-inheritance feature becoming stable in Rust 1.64.0. This feature allows specifying fields in the root workspace `Cargo.toml` which are then reused throughout the workspace. For example this PR shares definitions such as: * All of the Wasmtime-family of crates now use `version.workspace = true` to have a single location which defines the version number. * All crates use `edition.workspace = true` to have one default edition for the entire workspace. * Common dependencies are listed in `[workspace.dependencies]` to avoid typing the same version number in a lot of different places (e.g. the `wasmparser = "0.89.0"` is now in just one spot. Currently the workspace-inheritance feature doesn't allow having two different versions to inherit, so all of the Cranelift-family of crates still manually specify their version. The inter-crate dependencies, however, are shared amongst the root workspace. This feature can be seen as a method of "preprocessing" of sorts for Cargo manifests. This will help us develop Wasmtime but shouldn't have any actual impact on the published artifacts -- everything's dependency lists are still the same. * Fix wasi-crypto tests
Update to cap-std 0.26. (#4940)* Update to cap-std 0.26. This is primarily to pull in bytecodealliance/cap-std#271, the fix for #4936, compilation on Rust nightly on Windows. It also updates
Update to cap-std 0.26. (#4940)* Update to cap-std 0.26. This is primarily to pull in bytecodealliance/cap-std#271, the fix for #4936, compilation on Rust nightly on Windows. It also updates to rustix 0.35.10, to pull in bytecodealliance/rustix#403, the fix for bytecodealliance/rustix#402, compilation on newer versions of the libc crate, which changed a public function from `unsafe` to safe. Fixes #4936. * Update the system-interface audit for 0.23. * Update the libc supply-chain config version.
Bump Wasmtime to 2.0.0 (#4874)This commit replaces #4869 and represents the actual version bump that should have happened had I remembered to bump the in-tree version of Wasmtime to 1.0.0 prior t
Bump Wasmtime to 2.0.0 (#4874)This commit replaces #4869 and represents the actual version bump that should have happened had I remembered to bump the in-tree version of Wasmtime to 1.0.0 prior to the branch-cut date. Alas!
Bump Wasmtime to 0.41.0 (#4620)Co-authored-by: Wasmtime Publish <[email protected]>
Update some dependency versions used by Wasmtime (#4405)No major motivation here, mostly just dependency gardening.
Migrate most of wasmtime from lazy_static to once_cell (#4368)* Update tracing-core to a version which doesn't depend on lazy-static. * Update crossbeam-utils to a version that doesn't depend on
Migrate most of wasmtime from lazy_static to once_cell (#4368)* Update tracing-core to a version which doesn't depend on lazy-static. * Update crossbeam-utils to a version that doesn't depend on lazy-static. * Update crossbeam-epoch to a version that doesn't depend on lazy-static. * Update clap to a version that doesn't depend on lazy-static. * Convert Wasmtime's own use of lazy_static to once_cell. * Make `GDB_REGISTRATION`'s comment a doc comment. * Fix compilation on Windows.
Bump Wasmtime to 0.40.0 (#4378)Co-authored-by: Wasmtime Publish <[email protected]>
Update WASI to cap-std 0.25 and windows-sys. (#4302)This updates to rustix 0.35.6, and updates wasi-common to use cap-std 0.25 and windows-sys (instead of winapi). Changes include: - Better
Update WASI to cap-std 0.25 and windows-sys. (#4302)This updates to rustix 0.35.6, and updates wasi-common to use cap-std 0.25 and windows-sys (instead of winapi). Changes include: - Better error code mappings on Windows. - Fixes undefined references to `utimensat` on Darwin. - Fixes undefined references to `preadv64` and `pwritev64` on Android. - Updates to io-lifetimes 0.7, which matches the io_safety API in Rust. - y2038 bug fixes for 32-bit platforms
Bump Wasmtime to 0.39.0 (#4225)Co-authored-by: Wasmtime Publish <[email protected]>
Bump Wasmtime to 0.38.0 (#4103)Co-authored-by: Wasmtime Publish <[email protected]>
12