1# Continuous Integration (CI) 2 3The Wasmtime and Cranelift projects heavily rely on Continuous Integration (CI) 4to ensure everything keeps working and keep the final end state of the code at 5consistently high quality. The CI setup for this repository is relatively 6involved and extensive, and so it's worth covering here how it's organized and 7what's expected of contributors. 8 9All CI currently happens on GitHub Actions and is configured in the [`.github` 10directory][dir] of the repository. 11 12[dir]: https://github.com/bytecodealliance/wasmtime/tree/main/.github 13 14## PRs and CI 15 16Currently on sample of the full CI test suite is run on every Pull Request. CI 17on PRs is intended to be relatively quick and catch the majority of mistakes and 18errors. By default the test suite is run on x86\_64 Linux but this may change 19depending on what files the PR is modifying. The intention is to run "mostly 20relevant" CI on a PR by default. 21 22PR authors are expected to fix CI failures in their PR, unless the CI failure is 23systemic and unrelated to the PR. In that case other maintainers should be 24alerted to ensure that the problem can be addressed. Some reviewers may also 25wait to perform a review until CI is green on the PR as otherwise it may 26indicate changes are needed. 27 28The Wasmtime repository uses GitHub's Merge Queue feature to merge PRs which. 29Entry in to the merge queue requires green CI on the PR beforehand. Maintainers 30who have approved a PR will flag it for entry into the merge queue, and the PR 31will automatically enter the merge queue once CI is green. 32 33When entering the merge queue a PR will have the full test suite executed which 34may include tests that weren't previously run on the PR. This may surface new 35failures, and contributors are expected to fix these failures as well. 36 37To force PRs to execute the full test suite, which takes longer than the default 38test suite for PRs, then contributors can place the string "prtest:full" 39somewhere in any commit of the PR. From that point on the PR will automatically 40run the full test suite as-if it were in the merge queue. Note that when going 41through the merge queue this will rerun tests. 42 43## Tests run on CI 44 45While this may not be fully exhaustive, the general idea of all the checks we 46run on CI looks like this: 47 48* Code formatting - we run `cargo fmt -- --check` on CI to ensure that all code 49 in the repository is formatted with rustfmt. All PRs are expected to be 50 formatted with the latest stable version of rustfmt. 51 52* Book documentation tests - code snippets (Rust ones at least) in the book 53 documentation ([the `docs` 54 folder](https://github.com/bytecodealliance/wasmtime/tree/main/docs)) are 55 tested on CI to ensure they are working. 56 57* Crate tests - the moral equivalent of `cargo test --all` and `cargo test --all 58 --release` is executed on CI. This means that all workspace crates have their 59 entire test suite run, documentation tests and all, in both debug and release 60 mode. Additionally we execute all crate tests on macOS, Windows, and Linux, to 61 ensure that everything works on all the platforms. 62 63* Fuzz regression tests - we take a random sampling of the [fuzz 64 corpus](https://github.com/bytecodealliance/wasmtime-libfuzzer-corpus) and run 65 it through the fuzzers. This is mostly intended to be a pretty quick 66 regression test and testing the fuzzers still build, most of our fuzzing 67 happens on [oss-fuzz](https://oss-fuzz.com). Found issues are recorded in 68 the [oss-fuzz bug tracker](https://bugs.chromium.org/p/oss-fuzz/issues/list?q=-status%3AWontFix%2CDuplicate%20-component%3AInfra%20proj%3Awasmtime&can=1) 69 70While we do run more tests here and there, this is the general shape of what you 71can be expected to get tested on CI for all commits and all PRs. You can of 72course always feel free to expand our CI coverage by editing the CI files 73themselves, we always like to run more tests! 74 75## Artifacts produced on CI 76 77Our CI system is also responsible for producing all binary releases and 78documentation of Wasmtime and Cranelift. Currently this consists of: 79 80* Tarballs of the `wasmtime` CLI - produced for macOS, Windows, and Linux we try 81 to make these "binary compatible" wherever possible, for example producing the 82 Linux build in a really old CentOS container to have a very low glibc 83 requirement. 84 85* Tarballs of the Wasmtime C API - produced for the same set of platforms as the 86 CLI above. 87 88* Book and API documentation - the book is rendered with `mdbook` and we also 89 build all documentation with `cargo doc`. 90 91* A source code tarball which is entirely self-contained. This source tarball 92 has all dependencies vendored so the network is not needed to build it. 93 94* WebAssembly adapters for the component model to translate 95 `wasi_snapshot_preview1` to WASI Preview 2. 96 97Artifacts are produced as part of the full CI suite. This means that artifacts 98are not produced on a PR by default but can be requested via "prtest:full". All 99runs through the merge queue though, which means all merges to `main`, will 100produce a full suite of artifacts. The latest artifacts are available through 101Wasmtime's [`dev` release][dev] and downloads are also available for recent CI 102runs through the CI page in GitHub Actions. 103 104[dev]: https://github.com/bytecodealliance/wasmtime/releases/tag/dev 105