# cargo-ktstr
`cargo ktstr` is a cargo plugin for kernel build, cache, and test
workflow. Subcommands: `test`, `coverage`, `kernel`, `verifier`,
`shell`, `completions`, `test-stats`.
## test
Build the kernel (if needed) and run tests via `cargo nextest run`.
```sh
cargo ktstr test # auto-discover kernel
cargo ktstr test --kernel ../linux # local source tree
cargo ktstr test --kernel 6.14.2 # cached version
cargo ktstr test --kernel 6.14.2-tarball-x86_64-kc... # cache key (from kernel list)
```
`--kernel` accepts a path, version string, or cache key. When absent,
the test framework discovers a kernel via `resolve_test_kernel()`
(`KTSTR_TEST_KERNEL` env var, then `find_kernel()` for cache and
filesystem fallbacks). When `--kernel` is a path, cargo-ktstr configures and
builds the kernel before running tests. Version strings and cache
keys resolve from the cache only -- they error if not cached (run
`cargo ktstr kernel build VERSION` first).
### What it does (path mode only)
These steps run only when `--kernel` is a source directory path.
Version strings and cache keys skip to step 5.
1. **Config check** -- reads `<kernel>/.config` for
`CONFIG_SCHED_CLASS_EXT=y`.
2. **Auto-configure** -- if the config sentinel is missing, runs
`make defconfig` (when no `.config` exists), appends `ktstr.kconfig`
to `.config`, then runs `make olddefconfig`.
3. **Kernel build** -- runs `make -j$(nproc) KCFLAGS=-Wno-error`.
This always runs; `make` handles the no-op case when the kernel
is already built.
4. **compile_commands.json** -- runs `make compile_commands.json` to
generate the compilation database for clangd / LSP.
5. **Test execution** -- execs `cargo nextest run` with `KTSTR_KERNEL`
set in the environment for test kernel discovery.
### Passing nextest arguments
Arguments after `test` are passed through to `cargo nextest run`:
```sh
cargo ktstr test -- -E 'test(my_test)' # nextest filter
cargo ktstr test -- --workspace # all workspace tests
cargo ktstr test -- --retries 2 # nextest retries
```
## coverage
Build the kernel (if needed) and run tests with coverage via
`cargo llvm-cov nextest`. Same kernel resolution as `test`.
```sh
cargo ktstr coverage # auto-discover kernel
cargo ktstr coverage --kernel ../linux # local source tree
cargo ktstr coverage --kernel 6.14.2 # cached version
cargo ktstr coverage -- --workspace --lcov --output-path lcov.info # lcov output
```
Requires `cargo-llvm-cov` and the `llvm-tools-preview` rustup
component:
```sh
cargo install cargo-llvm-cov
rustup component add llvm-tools-preview
```
### Passing arguments
Arguments after `coverage` are passed through to
`cargo llvm-cov nextest`:
```sh
cargo ktstr coverage -- --workspace --profile ci --lcov --output-path lcov.info
cargo ktstr coverage -- --features integration
```
## kernel
Manage cached kernel images. Three subcommands: `list`, `build`,
`clean`. The standalone `ktstr kernel` subcommands are identical.
### kernel list
List cached kernel images, sorted newest first.
```sh
cargo ktstr kernel list
cargo ktstr kernel list --json # JSON output for CI scripting
```
Human-readable output shows key, version, source type, arch, and
build timestamp. Entries built with a different `ktstr.kconfig` are
marked `(stale kconfig)`. Entries built with a different ktstr version
are marked `(stale ktstr)`.
| `--json` | Output in JSON format. Includes all metadata fields. |
### kernel build
Download, build, and cache a kernel image. Three source modes:
version (tarball download), `--source` (local tree), `--git` (clone).
```sh
cargo ktstr kernel build # latest stable from kernel.org
cargo ktstr kernel build 6.14.2 # specific version
cargo ktstr kernel build 6.15-rc3 # RC release
cargo ktstr kernel build 6.12 # latest 6.12.x patch release
cargo ktstr kernel build --source ../linux # local source tree
cargo ktstr kernel build --git URL --ref v6.14 # git clone (shallow, depth 1)
cargo ktstr kernel build --force 6.14.2 # rebuild even if cached
```
When no version or source is given, fetches the latest stable version
from kernel.org's `releases.json`. A major.minor prefix (e.g. `6.12`)
resolves to the latest patch release in that series. Skips building when a cached entry
already exists (use `--force` to override). Stale entries (built with
a different `ktstr.kconfig`) are rebuilt automatically. For
`--source`, generates `compile_commands.json` for LSP support. Dirty
local trees (uncommitted changes to tracked files) are built but not
cached.
| `VERSION` | Kernel version to download (e.g. `6.14.2`, `6.15-rc3`). A major.minor prefix (e.g. `6.12`) resolves to the latest patch release. Conflicts with `--source` and `--git`. |
| `--source PATH` | Path to existing kernel source directory. Conflicts with `VERSION` and `--git`. |
| `--git URL` | Git URL to clone. Requires `--ref`. Conflicts with `VERSION` and `--source`. |
| `--ref REF` | Git ref to checkout (branch, tag, commit). Required with `--git`. |
| `--force` | Rebuild even if a cached image exists. |
| `--clean` | Run `make mrproper` before configuring. Only meaningful with `--source`. |
### kernel clean
Remove cached kernel images.
```sh
cargo ktstr kernel clean # remove all (with confirmation prompt)
cargo ktstr kernel clean --keep 3 # keep 3 most recent
cargo ktstr kernel clean --force # skip confirmation prompt
```
| `--keep N` | Keep the N most recent cached kernels. |
| `--force` | Skip confirmation prompt. Required in non-interactive contexts. |
## verifier
Collect BPF verifier statistics for a scheduler. Builds the
scheduler, boots a VM, loads the BPF programs, and reports per-program
verified instruction counts from host-side memory introspection.
```sh
cargo ktstr verifier --scheduler scx_rustland
cargo ktstr verifier --scheduler-bin ./target/release/scx_rustland
cargo ktstr verifier --scheduler scx_rustland --raw
cargo ktstr verifier --scheduler scx_rustland --all-profiles
cargo ktstr verifier --scheduler scx_rustland --profiles default,llc,llc+steal
```
Either `--scheduler` (package name, built via `cargo build`) or
`--scheduler-bin` (pre-built binary path) is required. They conflict
with each other.
`--all-profiles` discovers flags via `--ktstr-list-flags`, generates
the power set of valid flag combinations (respecting `requires`
constraints), and collects verifier stats for each profile. When more
than one profile is run, a summary table compares `verified_insns`
across profiles.
| `--scheduler PKG` | Scheduler package name to build and analyze. |
| `--scheduler-bin PATH` | Path to pre-built scheduler binary. Conflicts with `--scheduler`. |
| `--kernel ID` | Kernel identifier (path, version, or cache key). Auto-resolves when absent. |
| `--raw` | Print raw verifier output without cycle collapse. |
| `--all-profiles` | Run verifier for all flag profiles (power set). |
| `--profiles LIST` | Run verifier for specific profiles only (comma-separated, e.g. `default,llc,llc+steal`). |
See [BPF Verifier](verifier.md) for the verifier pipeline design and
output format.
## shell
Identical to the standalone `ktstr shell` -- see
[ktstr shell](ktstr.md#shell) for full documentation and flag
reference.
```sh
cargo ktstr shell
cargo ktstr shell --kernel 6.14.2
cargo ktstr shell --topology 1,2,4,1
cargo ktstr shell -i ./my-binary -i strace
```
## completions
Generate shell completions for cargo-ktstr. See
[ktstr completions](ktstr.md#completions) for the base subcommand.
```sh
cargo ktstr completions bash >> ~/.local/share/bash-completion/completions/cargo
cargo ktstr completions zsh > ~/.zfunc/_cargo-ktstr
cargo ktstr completions fish > ~/.config/fish/completions/cargo-ktstr.fish
```
| `SHELL` | Shell to generate completions for (`bash`, `zsh`, `fish`, `elvish`, `powershell`). |
| `--binary NAME` | Binary name for completions. Default: `cargo`. |
## test-stats
Print gauntlet analysis from sidecar JSON files produced by ktstr
tests.
```sh
cargo ktstr test-stats # default sidecar directory
cargo ktstr test-stats --dir path/to/dir # explicit directory
```
The output includes:
- **Gauntlet analysis** -- outlier detection, per-scenario/flags/topology
dimension summaries, stimulus cross-tab.
- **BPF verifier stats** -- per-program verified instruction counts,
warnings for programs near the 1M complexity limit.
- **BPF callback profile** -- per-program invocation counts, total
CPU time, and average nanoseconds per call.
- **KVM stats** -- cross-VM averages for exits, halt polling, host
preemptions.
### Prerequisites
Run tests first to generate sidecar JSON files:
```sh
cargo nextest run --workspace # generates target/ktstr/{branch}-{hash}/*.json
cargo ktstr test-stats # reads them
```
Set `KTSTR_SIDECAR_DIR` to override the default sidecar directory.
| `--dir PATH` | Path to the sidecar directory. Defaults to `KTSTR_SIDECAR_DIR` or `target/ktstr/{branch}-{hash}/`. |
## Deprecated: build-kernel
`cargo ktstr build-kernel` is deprecated. Use `cargo ktstr kernel build --source` instead:
```sh
# Old (deprecated):
cargo ktstr build-kernel --kernel ../linux
# New:
cargo ktstr kernel build --source ../linux
```
## Install
```sh
cargo install ktstr # installs both ktstr and cargo-ktstr
```
Or build from the workspace:
```sh
cargo build --bin cargo-ktstr
```