1# Testing
2
3This section describes how to run Wasmtime's tests and add new tests.
4
5Before continuing, make sure you can [build
6Wasmtime](./contributing-building.md) successfully. Can't run the tests if you
7can't build it!
8
9## Running All Tests
10
11To run all of Wasmtime's tests (excluding WASI integration tests), execute this command:
12
13```shell
14cargo test --all
15```
16
17To include WASI integration tests, you'll need `wasm32-wasi` target installed, which,
18assuming you're using [rustup.rs](https://rustup.rs) to manage your Rust versions,
19can be done as follows:
20
21```shell
22rustup target add wasm32-wasi
23```
24
25Next, to run all tests including the WASI integration tests, execute this command:
26
27```shell
28cargo test --features test_programs --all
29```
30
31You can also exclude a particular crate from testing with `--exclude`. For
32example, if you want to avoid testing the `wastime-fuzzing` crate — which
33requires that `libclang` is installed on your system, and for some reason maybe
34you don't have it — you can run:
35
36```shell
37cargo test --all --exclude wasmtime-fuzzing
38```
39
40## Testing a Specific Crate
41
42You can test a particular Wasmtime crate with `cargo test -p
43wasmtime-whatever`. For example, to test the `wasmtime-environ` crate, execute
44this command:
45
46```shell
47cargo test -p wasmtime-environ
48```
49
50Alternatively, you can `cd` into the crate's directory, and run `cargo test`
51there, without needing to supply the `-p` flag:
52
53```shell
54cd crates/environ/
55cargo test
56```
57
58## Running the Wasm Spec Tests
59
60The spec testsuite itself is in a git submodule, so make sure you've
61checked it out and initialized its submodule:
62
63```shell
64git submodule update --init
65```
66
67When the submodule is checked out, Wasmtime runs the Wasm spec testsuite as part
68of testing the `wasmtime-cli` crate:
69
70```shell
71cargo test -p wasmtime-cli
72```
73
74## Running WASI Integration Tests Only
75
76WASI integration tests can be run separately from all other tests which
77can be useful when working on the `wasi-common` crate. This can be done by
78executing this command:
79
80```shell
81cargo test --features test_programs -p test-programs
82```
83
84## Adding New Tests
85
86### Adding Rust's `#[test]`-Style Tests
87
88For very "unit-y" tests, we add `test` modules in the same `.rs` file as the
89code that is being tested. These `test` modules are configured to only get
90compiled during testing with `#[cfg(test)]`.
91
92```rust
93// some code...
94
95#[cfg(test)]
96mod tests {
97    use super::*;
98
99    #[test]
100    fn some_test_for_that_code() {
101        // ...
102    }
103}
104```
105
106If you're writing a unit test and a `test` module doesn't already exist, you can
107create one.
108
109For more "integration-y" tests, we create a `tests` directory within the crate,
110and put the tests inside there. For example, there are various code
111cache-related tests at `crates/environ/tests/cache_*.rs`. Always feel free to
112add a `tests` directory to a crate, if you want to add a new test and there
113aren't any existing tests.
114
115### Adding Specification-Style Wast Tests
116
117We use the spec testsuite as-is and without custom patches or a forked
118version. This probably isn't what you want to modify when adding a new Wasmtime
119test!
120
121When you have a Wasmtime-specific test that you'd like to write in Wast and use
122the Wast-style assertions, you can add it to our "misc testsuite". The misc
123testsuite uses the same syntax and assertions as the spec testsuite, but lives
124in `tests/misc_testsuite`. Feel free to add new tests to existing
125`tests/misc_testsuite/*.wast` files or create new ones as needed. These tests
126are run as part of the `wasmtime-cli` crate's tests.
127
128If you have a new test that you think really belongs in the spec testsuite, make
129sure it makes sense for every Wasm implementation to run your test (i.e. it
130isn't Wasmtime-specific) and send a pull request
131[upstream](https://github.com/WebAssembly/testsuite/). Once it is accepted in
132the upstream repo, we can update our git submodule and we'll start running the
133new tests.
134
135### Adding WASI Integration Tests
136
137When you have a WASI-specific test program that you'd like to include as a
138test case to run against our WASI implementation, you can add it to our
139`test-programs` crate. In particular, you should drop a main-style Rust source
140file into `crates/test-programs/wasi-tests/src/bin/some_new_test.rs` with a
141name of your choice. And that's it! The build script included in the
142`test-programs` crate will automatically generate the necessary boilerplate
143code for your test program so that it's run on all supported hosts.
144
145If you would like to tweak which host to run the test program against however
146(for instance, only on Unix but on Windows), you can tweak that in the build
147script located under `crates/test-programs/build.rs`.
148