cargo-impact
Predictive Regression Analysis & Verification Mapping for Rust
Status: v0.3.0 (stable) on crates.io —
cargo install cargo-impact. This README is both the living design spec and the user manual; sections describing yet-unshipped behavior are explicitly called out (§11 has the full shipped-vs-deferred breakdown).
Contents
- Core Philosophy
- Quickstart (Intended UX)
- Technical Architecture
- CLI Interface (UX)
- Vibe Coding Workflow Integration
- Summary Table: Context vs. Impact
- Integration with
cargo-context - MCP Server Surface
- Performance Targets
- Non-Goals
- Implementation Roadmap
- License
1. Core Philosophy
cargo-impact moves the developer from "Running all tests and hoping for the best" to Surgical Verification. It treats a code change as a "stone thrown into a pond" and calculates exactly which ripples hit which shores (tests, docs, APIs).
It answers the critical question: "I changed X; what is the minimum set of things I must check, and how confident can I be in each one?" Every finding is labeled with a confidence tier — static analysis is never certain, and the tool is honest about that.
2. Quickstart
Install
Pick whichever path fits your environment:
# 1. crates.io (once published — see https://crates.io/crates/cargo-impact)
# 2. Pinned from source by tag (works today, no crates.io dependency)
# 3. Prebuilt binary from the GitHub release page
# https://github.com/asmuelle/cargo-impact/releases
# Binaries for linux-x86_64, linux-aarch64, macos-x86_64,
# macos-aarch64, and windows-x86_64 are attached to each release.
First run
Expected first-run output on a clean workspace:
cargo-impact: no Rust files changed relative to HEAD
Make a change, re-run, and you'll get the severity-grouped report. cargo impact --help lists every flag.
Typical AI loop
| # ... give the pack to your AI, have it generate a patch, apply it ...
# ... AI ticks items, flags what it cannot verify ...
(Without cargo-context installed, cargo impact --context | xargs cat is a
plain-text fallback — see §7 for the full integration story.)
For agent-native consumption, start the MCP server: cargo impact mcp speaks JSON-RPC 2.0 over stdio with all six tools from §8.
3. Technical Architecture
cargo-impact is an orchestrator, not a from-scratch analyzer. It composes existing best-in-class Rust tooling into a single blast-radius report, with every finding labeled by confidence tier rather than a binary "affected/not affected" flag.
A. Analysis Backend (The Engine)
Static analysis of Rust requires resolved names, not just syntax. The backend is layered:
| Layer | Tool | Responsibility |
|---|---|---|
| Syntax | syn |
Parse diff hunks → candidate symbols |
| Macro expansion | cargo expand / HIR |
Expand derives, attribute, and fn-like macros before analysis (critical for serde, axum, clap, tokio) |
| Name resolution & call graph | rust-analyzer as library (ra_ap_hir, ra_ap_ide) |
Resolve paths, find references, traverse calls across modules and crates |
| Public API | rustdoc --output-format json + cargo-public-api |
Stable diff of the public surface |
| Semver impact | cargo-semver-checks |
Classify public changes as additive vs. breaking |
| Cache | target/impact-cache/ keyed by content hash + cargo fingerprint |
Sub-second warm runs; incremental invalidation |
Why not syn alone: re-exports, trait method dispatch, generics, and macro-generated code all defeat syntax-only analysis. rust-analyzer gives IDE-grade precision on stable Rust.
B. Blast Radius Detection
For each changed symbol, cargo-impact emits a finding with a confidence tier (see §3F):
- Direct references: Resolved call-graph edges from the reverse-reference index →
Proven. - Trait ripple (differentiated):
- Required method signature changed →
Proven(all impls break at compile time; report as build consequence, not risk). - Default method body changed →
Likelyfor impls that don't override;Provenfor impls that delegate viasuper::. - New method added →
Provencompile break unless defaulted. - Trait bound changed →
Likelyfor downstream generic code.
- Required method signature changed →
- Dynamic dispatch (
dyn Trait): All impls reachable viadyn Traitconstruction sites →Likely. - FFI /
unsafe extern: Any change toextern "C"signatures or#[no_mangle]symbols →ProvenHIGH (blast radius leaves Rust entirely). build.rschanges: Treated asProvenHIGH by default — build scripts can invalidate downstream compilation in non-obvious ways.- Feature-gated code: If the changed symbol is behind
#[cfg(feature = "...")], run analysis per active feature set.--features,--all-features, and--feature-powersetsupported.
C. Surgical Testing
Does not re-implement test selection — orchestrates proven tools:
- Coverage-driven selection: Delegates to
cargo-difftestsfor file-to-test mapping based on actual coverage traces. - Call-graph augmentation: Adds tests that statically reference changed symbols but weren't hit by the last coverage run (catches untested paths).
- Emits nextest filters: Output is a
cargo-nextestfilter expression, e.g.cargo nextest run -E 'test(test_auth_login) + test(test_api_handshake)'. Falls back tocargo testfilters when nextest is absent. - Handles: doctests in triple-backtick blocks inside doc comments,
#[cfg(test)] mod tests, per-member integration tests in workspaces,rstest/proptestparameterization,#[ignore],serial_test.
D. Runtime Surface Mapping (The "Exposed" Layer)
Framework detection runs after macro expansion (handler attributes, derive-based routers are invisible pre-expansion). Pluggable adapters, not hardcoded framework logic:
- HTTP routers:
axum(Router::route, nested routers,#[debug_handler]),actix-web(web::resource, scopes),rocket(#[get]/#[post]),warp. - CLI:
clapderive and builder APIs. - Desktop/mobile:
tauricommands,dioxusroutes. - Public crate API: Delegated to
cargo-semver-checks+cargo-public-api. - Adapter contract: A small trait so third parties can register custom surface mappers for proprietary frameworks.
E. Documentation Drift Detection
Precise, not keyword-based:
- Intra-doc links: Parses rustdoc JSON for
[\PaymentGateway`]-style references in doc comments and/docs/*.md. Exact symbol resolution, not substring match onUser`. - Rustdoc example blocks: Flags doctest examples that exercise changed symbols.
- Changelog heuristic: If a
pubitem changed andCHANGELOG.md/RELEASES.mdwasn't touched in the same diff → flag.
F. Confidence Tiers
Every finding carries a tier. This replaces the fiction that static analysis is ever "99% sure."
| Tier | Score | Meaning | Example source |
|---|---|---|---|
| Proven | 0.95–1.00 | Resolved call-graph edge or rustdoc JSON symbol match | fn a() calls fn b() directly, both resolved |
| Likely | 0.60–0.94 | Trait impl via dyn, feature-gated caller, default-method non-overrider |
Handler registered via axum::Router::route with expanded macro |
| Possible | 0.30–0.59 | Heuristic match, unexpanded macro residue, cross-crate without rustdoc | Identifier appears in doc comment without intra-doc link |
| Unknown | < 0.30 | Listed but not scored | Reflection via Any, runtime config, FFI callbacks, OnceCell mutation |
--confidence-min=0.6 filters output for CI; default shows all tiers with the score attached.
4. CLI Interface (UX)
Flags below reflect the shipping v0.3.0 surface — --context, --features, and the mcp subcommand are all live. Flags still in flight: --checklist (the verification checklist is currently embedded inside --format markdown rather than a dedicated output), --feature-powerset (CI-grade matrix analysis, v0.4 scope). See §11 for the full roadmap.
# Analyze the current working tree against HEAD
# Emit a cargo-nextest filter expression for affected tests only
# Example output: test(auth_roundtrip) + test(api_smoke)
# Analyze against a specific revision (branch, tag, SHA)
# AI-consumable formats
# CI gating
# Feature-aware analysis — cfg(feature = "x") gates are evaluated against
# the resolved active set. Items whose gates don't match are stripped before
# every analyzer sees them.
# Opt-in public-API breakage detection (requires cargo-semver-checks on PATH;
# runs rustdoc twice internally, typically 10–30s).
Sample report (text)
v0.2 is syn-only; no finding reaches the Proven tier — that is reserved for resolved call-graph analysis arriving with rust-analyzer in v0.3. Every score below is the honest ceiling for syntactic analysis.
cargo-impact v0.2.0
Changed files (3):
src/engine.rs
src/ffi.rs
build.rs
Candidate symbols (4):
Greeter
callback_t
process_event
UserProfile
🔴 HIGH (3)
[f-0001] build.rs changed (build.rs) · Likely 0.90
[f-0002] FFI callback_t modified in src/ffi.rs · Likely 0.95
[f-0003] impl Greeter for Foo (src/engine.rs) · Likely 0.80
🟡 MEDIUM (2)
[f-0004] test `api_smoke` (tests/integration.rs) references process_event · Likely 0.85
[f-0005] dyn Greeter used in src/dispatch.rs · Likely 0.75
🔵 LOW (1)
[f-0006] intra-doc link to UserProfile in docs/architecture.md:42 · Likely 0.90
⚪ UNKNOWN (0)
Sample JSON envelope
Stable across releases; matches the MCP tool-call schema (§8) so the CLI and the future MCP server return identical shapes.
Markdown output
--format markdown produces a paste-to-AI-ready document: summary, per-severity sections, and a verification checklist with - [ ] items an agent can tick. Example shape:
- --
- -
- --
5. Vibe Coding Workflow Integration
This completes the Context → Code → Verify loop:
- Context:
cargo context --fix | pbcopy→ AI generates a fix. - Apply: Developer applies the AI's code.
- Impact: Developer runs
cargo impact. - Verify:
cargo impactflags a specific integration test and one API endpoint (with confidence tiers).- Developer runs
cargo impact --test(5 seconds instead of 5 minutes). - Developer runs
cargo impact --checklist --format=markdown | pbcopyand pastes back to the AI: "Here's the verification checklist — tick what you've addressed and tell me what you still need me to test manually."
6. Summary Table: Context vs. Impact
| Feature | cargo-context (The Input) |
cargo-impact (The Output) |
|---|---|---|
| Goal | Maximize AI understanding | Minimize human verification effort |
| Focus | What is the AI looking at? | What did the AI touch? |
| Primary Tools | git diff + cargo metadata + rustdoc JSON |
rust-analyzer call graph + cargo-difftests + cargo-semver-checks |
| Key Output | A Markdown Context Pack | A confidence-tiered Blast Radius Report |
| Vibe Shift | No more copy-pasting files | No more "test all and pray" |
7. Integration with cargo-context
cargo-impact and cargo-context are designed as a bidirectional pair, not two isolated tools. They share cache, symbol index, and MCP surface.
Forward flow: Context → Impact
The normal developer loop. AI edits files inside a context pack; cargo-impact analyzes the resulting diff.
| # ... apply patch ...
Reverse flow: Impact → Context ✅ shipped
cargo-impact --context emits a deduped, newline-delimited list of every file implicated in the blast radius (changed files + each finding's primary path). cargo-context --files-from - consumes that list directly and builds a context pack scoped to exactly those files:
| |
# Pack contains only the blast-radius files, not the whole repo.
cargo-context applies its usual scrubber to each path (so .env etc. never leak raw secrets), skips missing paths with an accounting header, and prioritizes the scoped section at diff-level priority so it survives --budget pressure. Implementation: cargo-context#5.
Scope-limited context packs ⏳ deferred
Passing the full cargo-impact JSON envelope (not just file paths) so cargo-context can prioritize by confidence tier, filter out findings already verified elsewhere, or emit per-finding mini-packs. Tracked as the second half of cargo-context#5.
# Once shipped:
Shared cache ⏳ deferred
The spec imagined both tools reading target/ai-tools-cache/ with namespaced subdirectories (context/, impact/) so rust-analyzer's index is built once and shared. Neither tool ships this yet; each maintains its own cache. A real implementation needs the cache format versioned independently of both tools — tracked for a joint v0.4.
Shared MCP server ⏳ deferred
Combined cargo-ai-tools MCP server that exposes both families under one process (see §8 for the cargo-impact side). Also v0.4+.
8. MCP Server Surface
cargo-impact ships as a CLI and an MCP server. Agents (Claude Code, Cursor, Zed, custom) connect over stdio and call tools directly — no copy-paste, no shell parsing.
Tools exposed
| Tool | Purpose | Returns |
|---|---|---|
impact.analyze |
Run blast radius on current diff or commit range | Structured finding graph with tiers |
impact.test_filter |
Get a cargo-nextest filter expression for affected tests |
String + rationale per test |
impact.checklist |
Generate the verification checklist | Markdown + JSON sibling |
impact.surface |
List affected runtime surfaces (routes, CLI subcommands, FFI) | Structured surface list |
impact.semver |
Classify public API changes | {additive, breaking, none} + reasons |
impact.explain |
Explain why a specific finding was flagged | Trace of the evidence chain |
Tool schema example: impact.analyze
Response envelope
Every tool returns the same outer shape so agents can reason uniformly:
Why not just parse CLI output
Agents parsing pretty-printed CLI text is brittle and burns tokens. MCP tool calls return typed JSON, stream progress for long analyses, and let the agent ask impact.explain(id="f-0007") to drill into a specific finding without re-running the whole analysis.
9. Performance Targets
The "5 seconds, not 5 minutes" claim in §5 needs teeth. These are the SLOs the tool targets — not aspirational, but used as regression thresholds in the benchmark suite.
| Scenario | Target | Measured on |
|---|---|---|
| Warm run, <50 changed files, small workspace (<10 crates) | < 500ms | cargo-impact self-hosting benchmark |
| Warm run, typical workspace (10–30 crates) | < 1.5s | ripgrep, zola-sized repos |
| Warm run, large workspace (100+ crates) | < 5s | rustc, deno, internal monorepos |
| Cold run (first invocation, no cache) | < 30s for 100-crate workspace | includes RA index build |
| MCP response p95 | < 200ms after warm cache | per-tool-call latency |
--feature-powerset on 8 features |
< 60s | CI-only mode, not interactive |
Cache strategy
- Keyed on:
(file_content_hash, cargo_fingerprint, rustc_version, features_hash). - Granularity: Per-file symbol index, per-symbol call-graph edges. A one-line change in
src/utils.rsinvalidates onlyutils's index and its dependents' edges — not the whole crate. - Location:
target/ai-tools-cache/impact/(shared withcargo-context, see §7). - Eviction: LRU by access time; cap at 500MB by default, configurable via
CARGO_IMPACT_CACHE_SIZE. - Invalidation signal:
cargo-impactwatchesCargo.lockand rustc version; bumps purge dependent caches.
When performance degrades
If a run exceeds 2× the target for its scenario, cargo-impact emits a diagnostic:
⚠ cargo-impact: analysis took 4.2s (target < 1.5s for this workspace size).
Likely cause: rust-analyzer index cold (cache dir recently cleared).
Subsequent runs should be fast. Run `cargo impact --bench` to profile.
Benchmark suite
cargo impact --bench runs a built-in benchmark against the current workspace, reports against the SLO table, and writes a JSON trace for regression tracking. Designed to be run in CI on the cargo-impact repo itself — no flaky wall-clock assertions, uses the cargo fingerprint to skip when inputs are unchanged.
10. Non-Goals
Scope discipline matters. cargo-impact is a static impact oracle, not a correctness checker. The following are explicitly out of scope and will not be added:
- Runtime behavior verification. The tool does not execute code, trace syscalls, or observe actual runtime paths. A function can pass every affected test and still be broken in production;
cargo-impactcannot detect that. - Logic bug detection. If the AI writes
a + bwhere it meanta - band the tests don't catch it, neither willcargo-impact. We tell you what to check, not whether the logic is right. - Code review replacement. A human (or reviewing agent) still reads the diff. The blast radius tells them where to focus, not what to conclude.
- Mutation testing. That is
cargo-mutants's job. If you want "would this test catch a bug if one existed," use that. - Fuzzing / property testing integration. Out of scope. Run
cargo-fuzz/proptestseparately and feed their failures back to the AI through whatever channel you already use. - Type-level refactoring guidance. We flag that
UserProfilewas modified; we do not advise on whether a different type design would have been better. - Runtime tracing or profiling. Not a flamegraph, not a tokio console, not a perf tool.
- IDE diagnostics.
rust-analyzeralready does this. We consume its index; we do not compete with it. - Formatting, linting, or style enforcement.
rustfmtandclippyexist. - Dependency vulnerability scanning. That is
cargo-audit/cargo-deny. - Build-time regression detection. Changes that blow up compile time are real but invisible to us — use
cargo-bloator-Z self-profile. - Cross-language impact. A Rust change that breaks a Python FFI consumer is flagged at the
extern "C"boundary (§3B) but we do not trace into the foreign language. - Non-Rust workspaces. We are
cargo-*. Polyglot monorepos are out of scope; runcargo-impactagainst the Rust portion and compose with your own tooling for the rest.
What we will integrate but not reinvent
- Selective testing →
cargo-difftests - Semver classification →
cargo-semver-checks - Public-API diffing →
cargo-public-api - Test execution →
cargo-nextest - Name resolution →
rust-analyzer(as library)
If an existing tool solves a subproblem well, we orchestrate it. If we find ourselves reimplementing one, that is a signal we are off-mission.
11. Implementation Roadmap
The spec is deliberately ambitious. These milestones are the cut points where the tool is genuinely useful to a real user, not a lab demo. Each milestone ships independently.
v0.1 — "Surgical test filter" (the MVP) ✅ shipped
Goal: A Rust developer saves time on cargo test today, with zero AI integration.
- ✅
cargo impactparsesgit diffwithsyn→ candidate symbols - ⏭
rust-analyzer(as library) resolves direct call-graph references — deferred to v0.3; v0.2 uses syn-only token matching honestly tieredLikely - ✅ Emits a
cargo-nextestfilter expression via--test - ✅ Human-readable text output
Deliberately deferred: macros, traits, features, MCP, framework adapters, confidence tiers, public-API analysis. If v0.1 isn't a 2-week project, we're over-engineering.
Success metric: On the ripgrep workspace, typical edits trigger <10% of tests with zero false negatives across 50 seeded changes.
v0.2 — "Honest blast radius" (in progress — core shipped)
Goal: The report earns the name. Confidence tiers, trait handling, and the first AI-consumable format.
- ⚠ Macro expansion —
#[derive(...)]now recognized (matched on last path segment, soserde::Serialize/clap::Parser/ etc. all resolve). Fullcargo expand/ HIR pass for attribute andfn-like macros still deferred; nightly toolchain boundary - ✅ Trait ripple differentiation (§3B): required vs. default vs. new method — per-method HEAD-vs-WT classification shipping in addition to the blanket
TraitImplscan - ✅
dyn Traitdispatch edges asLikely 0.75 - ✅ Confidence tiers (§3F) with numeric scores (
Provenreserved for v0.3 RA integration) - ✅
--format=jsonand--format=markdown - ✅
--features/--all-features/--no-default-features— cfg-aware AST filtering against the resolved feature set - ✅
cargo-semver-checksintegration (opt-in via--semver-checks) - ✅
--confidence-minand--fail-on={high,medium,low}for CI - ✅ Documentation drift via intra-doc links (plus length-gated keyword fallback)
- ✅ Bonus: diff-aware candidate symbols (HEAD-vs-WT per-item comparison) + FFI signature tracking +
build.rschange detection
Success metric: On the cargo-impact repo itself, 95% of Proven-tier findings correspond to tests that actually fail when the finding is seeded as a regression. (v0.2 emits no Proven findings — the metric re-activates once rust-analyzer integration lands in v0.3.)
v0.3 — "Agent-native"
Goal: First-class AI integration. The tool is now consumed by agents, not just humans.
- ✅ MCP server (
cargo impact mcp) — all six §8 tools (impact_analyze,impact_test_filter,impact_surface,impact_semver,impact_explain,impact_version) ship in v0.3.0. - ✅ Rust-analyzer integration for the
Proventier — LSP stdio client with Content-Length framing, initialize handshake, indexing-progress wait,documentSymbol+referencesqueries, emittingResolvedReferencefindings atTier::Proven. - ✅ Content-hashed finding IDs so
impact_explaincan round-trip by ID across runs. - ✅
--contextbridge tocargo-context(forward-flow shipped viacargo-context --files-from -; JSON-envelope--impact-scopestill deferred) - ⏳ Framework adapters:
axum,clap(reference implementations); documented adapter trait for third parties - ⏳
cargo impact log-missfor ground-truth collection - ⏳ Token budgeting on markdown output
- ⏳ Configuration file (
cargo-impact.toml) +.impactignore
Success metric: A Claude Code / Cursor session can complete a non-trivial Rust refactor using only MCP tool calls — no shell output parsing, no manual context assembly.
v0.4 — "Production-grade"
Goal: Ready for use in serious CI pipelines and large workspaces.
- Adapters:
actix-web,rocket,tauri,dioxus --feature-powerset(CI mode)- Cross-target support (
--target wasm32-unknown-unknown,no_std) - Streaming progress over MCP for long analyses
- Deterministic output for reproducible CI artifacts
cargo-difftestscoverage integration (if stable by then)- Benchmark suite hardening; SLO regression gates in CI
- Pre-push and GitHub Actions recipes shipped as templates
Success metric: Deployable on a 100+ crate workspace with a stable, cached p50 under 2 seconds.
Beyond v0.4
Uncertain and deliberately unplanned. Candidates (prioritized by user demand, not roadmap):
- IDE integration (VS Code extension, Zed)
git bisectdriver that uses impact data to narrow the search- Historical blast-radius mining from the full repo history (find under-tested hot spots)
- Cross-workspace impact for polyrepo setups with path dependencies
What could kill each milestone
Honest risk log, not hand-wave:
| Milestone | Biggest risk | Mitigation |
|---|---|---|
| v0.1 | ra_ap_* API churn between rust-analyzer releases |
Pin RA version per release; fall back to LSP protocol if library API breaks |
| v0.2 | Macro expansion is too slow or too incomplete on real workspaces | Downgrade confidence on unexpanded macros rather than fail; document known-bad proc macros |
| v0.3 | MCP ecosystem fragments before stabilizing | Ship stdio + Streamable HTTP; keep CLI first-class so tool isn't MCP-dependent |
| v0.4 | cargo-difftests doesn't mature / is abandoned |
Fall back to call-graph-only test selection (less precise but still useful) |
License
Dual-licensed under either of
- Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
- MIT License (LICENSE-MIT or http://opensource.org/licenses/MIT)
at your option. Per Rust ecosystem convention.
Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.