1# Testing
2
3This section describes how to run Wasmtime's tests and add new tests.
4
5Before continuing, make sure you can [build
6Wasmtime](./contributing-building.md) successfully. Can't run the tests if you
7can't build it!
8
9## Installing `wasm32` Targets
10
11To compile the tests, you'll need the `wasm32-wasi` and
12`wasm32-unknown-unknown` targets installed, which, assuming you're using
13[rustup.rs](https://rustup.rs) to manage your Rust versions, can be done as
14follows:
15
16```shell
17rustup target add wasm32-wasi wasm32-unknown-unknown
18```
19
20## Running All Tests
21
22To run all of Wasmtime's tests, execute this command:
23
24```shell
25cargo test --workspace
26```
27
28You can also exclude a particular crate from testing with `--exclude`. For
29example, if you want to avoid testing the `wastime-fuzzing` crate — which
30requires that `libclang` is installed on your system, and for some reason maybe
31you don't have it — you can run:
32
33```shell
34cargo test --workspace --exclude wasmtime-fuzzing
35```
36
37Similarly, to skip WASI integration tests, run:
38
39```shell
40cargo test --workspace --exclude test-programs
41```
42
43## Testing a Specific Crate
44
45You can test a particular Wasmtime crate with `cargo test -p
46wasmtime-whatever`. For example, to test the `wasmtime-environ` crate, execute
47this command:
48
49```shell
50cargo test -p wasmtime-environ
51```
52
53Alternatively, you can `cd` into the crate's directory, and run `cargo test`
54there, without needing to supply the `-p` flag:
55
56```shell
57cd crates/environ/
58cargo test
59```
60
61## Running the Wasm Spec Tests
62
63The spec testsuite itself is in a git submodule, so make sure you've
64checked it out and initialized its submodule:
65
66```shell
67git submodule update --init
68```
69
70When the submodule is checked out, Wasmtime runs the Wasm spec testsuite as part
71of testing the `wasmtime-cli` crate:
72
73```shell
74cargo test -p wasmtime-cli
75```
76
77## Running WASI Integration Tests Only
78
79WASI integration tests can be run separately from all other tests which
80can be useful when working on the `wasi-common` crate. This can be done by
81executing this command:
82
83```shell
84cargo test -p test-programs
85```
86
87## Adding New Tests
88
89### Adding Rust's `#[test]`-Style Tests
90
91For very "unit-y" tests, we add `test` modules in the same `.rs` file as the
92code that is being tested. These `test` modules are configured to only get
93compiled during testing with `#[cfg(test)]`.
94
95```rust
96// some code...
97
98#[cfg(test)]
99mod tests {
100    use super::*;
101
102    #[test]
103    fn some_test_for_that_code() {
104        // ...
105    }
106}
107```
108
109If you're writing a unit test and a `test` module doesn't already exist, you can
110create one.
111
112For more "integration-y" tests, we create a `tests` directory within the crate,
113and put the tests inside there. For example, there are various code
114cache-related tests at `crates/environ/tests/cache_*.rs`. Always feel free to
115add a `tests` directory to a crate, if you want to add a new test and there
116aren't any existing tests.
117
118### Adding Specification-Style Wast Tests
119
120We use the spec testsuite as-is and without custom patches or a forked
121version. This probably isn't what you want to modify when adding a new Wasmtime
122test!
123
124When you have a Wasmtime-specific test that you'd like to write in Wast and use
125the Wast-style assertions, you can add it to our "misc testsuite". The misc
126testsuite uses the same syntax and assertions as the spec testsuite, but lives
127in `tests/misc_testsuite`. Feel free to add new tests to existing
128`tests/misc_testsuite/*.wast` files or create new ones as needed. These tests
129are run as part of the `wasmtime-cli` crate's tests.
130
131If you have a new test that you think really belongs in the spec testsuite, make
132sure it makes sense for every Wasm implementation to run your test (i.e. it
133isn't Wasmtime-specific) and send a pull request
134[upstream](https://github.com/WebAssembly/testsuite/). Once it is accepted in
135the upstream repo, we can update our git submodule and we'll start running the
136new tests.
137
138### Adding WASI Integration Tests
139
140When you have a WASI-specific test program that you'd like to include as a
141test case to run against our WASI implementation, you can add it to our
142`test-programs` crate. In particular, you should drop a main-style Rust source
143file into `crates/test-programs/wasi-tests/src/bin/some_new_test.rs` with a
144name of your choice. And that's it! The build script included in the
145`test-programs` crate will automatically generate the necessary boilerplate
146code for your test program so that it's run on all supported hosts.
147
148If you would like to tweak which host to run the test program against however
149(for instance, only on Unix but on Windows), you can tweak that in the build
150script located under `crates/test-programs/build.rs`.
151