# Testing wit-bindgen
## Setup
### Dev container
There are a few pre-requisites to testing the project. The easiest way to get
started is by using the dev container part of this repository. That contains all
the required toolchains in a single Docker container.
### Manual installation
Alternatively, you can install the required tools by hand. You only need the
language compilers that you wish to run test against.
- WASI SDK
- Download from wasi-sdk releases page. If you're using Windows, you need the one with mingw in its name.
- `curl -LO https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-25/wasi-sdk-25.0-x86_64-linux.tar.gz`
- Create an environment variable called `WASI_SDK_PATH` giving the path where you extracted the WASI SDK download, i.e., the directory containing `bin`/`lib`/`share` folders.
- Compilers for the target language:
- Rust - wasi target: `rustup target add wasm32-wasip2`
- C - [Clang](https://clang.llvm.org/) (You can also use the one shipped with WASI SDK)
- C# - [Dotnet 9](https://dotnet.microsoft.com/en-us/download/dotnet/9.0)
- [MoonBit](https://www.moonbitlang.com/download/)
## Test structure
There are two suites of tests: [codegen](#testing-wit-bindgen---codegen) and [runtime](#testing-wit-bindgen---runtime). To run all possible tests, across all supported languages, ensure the dependencies above are installed then run:
```
cargo test --workspace
```
To run just `codegen` tests for a single language (replace rust with language of choice: `c`, `csharp`, etc.):
```
cargo test -p wit-bindgen-rust
```
To run just `codegen` tests for a single language (replace rust with language of choice: `c`, `csharp`, etc.) and a single wit file (replace `flags` with whatever wit file should be tested):
```
cargo test -p wit-bindgen-rust -- flags
```
To run just `runtime` tests for a single language (replace rust with language of choice: `c`, `csharp`, etc.):
```bash
cargo test -p wit-bindgen-cli --no-default-features -F rust
```
Read on to learn more about the testing layout. It's all a bit convoluted so feel free to ask questions on [Zulip](../README.md#about) or open an issue if you're lost.
## Testing wit-bindgen - `codegen`
Any tests placed in to the `tests/codegen` directory should follow either of the following formats:
1. `*.wit` files that are raw wit files that should be executed by the code generator.
2. wit package in it's own directory which must contain a `wit` subdirectory with `*.wit` files and `deps` in it. (e.g. see` tests/codegen/issue569`)
The purpose of these files is to execute language-specific
validation for each bindings generator. Basically if there's a bug where
something generates invalid code then this is probably where the test should go.
Note that this directory can have whatever it wants since nothing implements the
interfaces or tries to call them.
It also contains git submodules for the wasi proposals like `wasi-http`.
Use `git submodule init` + `git submodule update` before running the tests.
The tests are generated by a macro `codegen_tests` in [crates/test-helpers](../crates/test-helpers/).
You can run the `codegen` tests for all languages with the following command:
```sh
cargo run test \
--languages rust,c,cpp,csharp,moonbit \
--artifacts target/artifacts \
--rust-wit-bindgen-path ./crates/guest-rust \
tests/codegen
```
## Testing wit-bindgen - `runtime`
Otherwise tests are organized in `tests/runtime/*`. Inside this directory is a
directory-per-test. These tests are somewhat heavyweight so you may want to
extend existing tests, but it's also fine to add new tests at any time.
The purpose of this directory is to contain code that's actually compiled to
wasm and executed on hosts. The code compiled-to-wasm can be one of:
* `wasm.rs` - compiled with Rust to WebAssembly
* `wasm.c` - compiled with Clang
* `wasm.cs` - compiled with NativeAOT and Mono
Existence of these files indicates that the language should be supported for the
test, and if a file is missing then it's skipped when running other tests. Each
`wasm.*` file is run inside each of the host file under `tests/runtime` directory.
For example, `tests/runtime/variants.rs` is the host file for `tests/runtime/variants/`
Each of these hosts can also be omitted if the host doesn't implement the test
or something like that. Otherwise for each host that exists when the host's
crate generator crate is tested it will run all these tests.
You can run the `runtime` tests for all languages with the following command:
```sh
cargo run test \
--languages rust,c,cpp,csharp,moonbit \
--artifacts target/artifacts \
--rust-wit-bindgen-path ./crates/guest-rust \
tests/runtime
```
## Testing Layout
If you're adding a test, all you should generally have to do is edit files in
`tests/runtime/*`. If you're adding a new test it *should* be along the lines of
just dropping some files in there, but currently if you're adding a
Rust-compiled-to-wasm binary you'll need to edit
`crates/test-rust-wasm/Cargo.toml` and add a corresponding binary to
`crates/test-rust-wasm/src/bin/*.rs` (in the same manner as the other tests).
Other than this though all other generators should automatically pick up new
tests.
The actual way tests are hooked up looks roughly like:
* All generator crates have a `codegen.rs` and a `runtime.rs` integration test
file (typically defined in the crate's own `tests/*.rs` directory).
* All generator crates depend on `crates/test-helpers`. This crate will walk the
appropriate directory in the top-level `tests/*` directory.
* The `test-helpers` crate will generate appropriate `#[test]` functions to
execute tests. For example the JS generator will run `eslint` or `tsc`. Rust
tests for `codegen` are simply that they compile.
* The `test-helpers` crate also builds wasm files at build time, both the Rust
and C versions. These are then encoded into the generated `#[test]` functions
to get executed when testing.