dry4rust

Created by Matjaz Domen Pecan as cargo-dupes, forked and extended by Umberto Gotti under the MIT licence.

A cargo subcommand for detecting duplicated code patterns in Rust -- DRY (Don't Repeat Yourself) analysis.

Provenance

dry4rust is a fork of cargo-dupes by Matjaz Domen Pecan, used under the MIT licence. The duplicate-detection engine documented below — AST normalisation, fingerprint hashing and Dice-coefficient similarity — is his work, and both copyright notices are carried in LICENSE and in every source file.

The fork keeps the Rust analyser and drops the tree-sitter and Python backends, so this is a single-language tool by intent rather than a general one. What is added beyond that point is noted in CHANGELOG.md.

Family

Part of the same family as grip (testability), braintax (cognitive load), and crap4rust (change-risk complexity × coverage). Where those three measure how safe, understandable, and risky a codebase is, dry4rust measures how much of it is needlessly repeated.

Status

The engine is upstream's, with four corrections it does not have — an exact size bound, aligned sequence children, a fingerprint format that survives a toolchain upgrade, and near-duplicate detection that no longer hides what exact detection reports. Added on top: baseline mode, and configuration that cannot hold an impossible threshold.

The surrounding repository is under this family's house rules and gates: cargo stern4rust reports no offences with all twenty-two rules applied to every workspace member and none skipped -- its one baseline holds the ten end-to-end scenario files that have no source file to be named after -- cargo crap4rust finds no function at or above 15, every source file has a mirrored test file, no file reaches 10 on cargo iceberg4rust, and dry4rust run on its own source finds nothing beyond the two near-duplicate groups its baseline records.

Commands below are cargo dry4rust; the upstream cargo dupes spelling is gone.

Install

cargo install cargo-dry4rust

License

MIT — see LICENSE, which carries both copyright notices.

How It Works

dry4rust parses Rust source files into ASTs using syn, then normalizes each function, method, and closure into a canonical form where:

• Identifiers are replaced with positional placeholders (so foo(x) and bar(y) are identical)
• Literal values are erased but types preserved (42 and 99 are both "integer literal")
• Control flow structure is preserved exactly
• Macro invocations become opaque nodes

This normalized AST is hashed into a fingerprint for exact duplicate detection, and compared tree-by-tree using the Dice coefficient for near-duplicate detection.

Usage

cargo dry4rust [OPTIONS] [COMMAND]

Commands:
  stats     Show duplication statistics only
  report    Show full duplication report (default)
  check     Check for duplicates and exit with non-zero if thresholds exceeded
  ignore    Add a fingerprint to the ignore list
  ignored   List all ignored fingerprints
  cleanup   Remove ignore entries that no longer match anything
  baseline  Record the duplication that is already there

Options:
  -p, --path <PATH>                Path to analyze (defaults to current directory)
      --min-nodes <MIN_NODES>      Minimum AST node count for analysis
      --min-lines <MIN_LINES>      Minimum source line count for analysis
      --threshold <THRESHOLD>      Similarity threshold (0.0-1.0)
      --format <FORMAT>            Output format [default: text] [possible values: text, json]
      --exclude <EXCLUDE>          Exclude patterns (can be repeated)
      --exclude-tests              Exclude test code (#[test] functions and #[cfg(test)] modules)
  -s, --sub-function               Also analyse if-branches, match arms, loop bodies and closure bodies
      --min-sub-nodes <N>          Minimum AST node count for a sub-function unit [default: 5]
      --baseline <PATH>            Judge the run against a recorded baseline of inherited duplication
  -h, --help                       Print help
  -V, --version                    Print version

Every example below is real output from fixture/exact_dupes, which ships in the repository -- cargo run -p cargo-dry4rust -- --path fixture/exact_dupes report reproduces it.

--threshold and the two --max-*-percent ceilings are checked against their ranges. A threshold outside 0.0..=1.0, or a percentage outside 0.0..=100.0, fails the run with a message naming the field rather than being accepted and quietly finding nothing.

Examples

Full report:

$ cargo dry4rust report
Duplication Statistics
=====================
Total code units analyzed: 3

Exact duplicates: 1 groups (3 code units)
Near duplicates:  0 groups (0 code units)

Duplicated lines (exact): 27
Duplicated lines (near):  0
Duplication: 100.0% exact, 0.0% near (of 27 total lines)

Exact Duplicates
================

Group 1 (fingerprint: 396fa8f6b728ff01, 3 members):
  - process_data (function) at src/target.rs:6-14
  - compute_total (function) at src/target.rs:16-24
  - aggregate (function) at src/target.rs:26-34

Statistics only:

$ cargo dry4rust stats
Duplication Statistics
=====================
Total code units analyzed: 3

Exact duplicates: 1 groups (3 code units)
Near duplicates:  0 groups (0 code units)

Duplicated lines (exact): 27
Duplicated lines (near):  0
Duplication: 100.0% exact, 0.0% near (of 27 total lines)

JSON output:

$ cargo dry4rust --format json stats
{
  "total_code_units": 3,
  "total_lines": 27,
  "exact_duplicate_groups": 1,
  "exact_duplicate_units": 3,
  "near_duplicate_groups": 0,
  "near_duplicate_units": 0,
  "exact_duplicate_lines": 27,
  "near_duplicate_lines": 0,
  "exact_duplicate_percent": 100.0,
  "near_duplicate_percent": 0.0
}

Every --format json run is a single JSON document, so jq and any ordinary parser read it in one call. report and check name their sections:

$ cargo dry4rust --format json report | jq '.exact | length'
$ cargo dry4rust --format json check --max-exact 0 | jq '.passed, .breaches'
false
[
  "1 exact duplicate groups (max: 0)"
]

command	keys
stats	the summary fields, at the top level
report	stats, exact, near, and sub_exact/sub_near when sub-function analysis found any
check	stats, passed, breaches, and exact/near when a ceiling was breached

exact and near are always present on report because those are always analysed. The sub-function sections are absent rather than empty when the analysis was not asked for, so [] never stands in for "not looked at". On check, the groups behind a breach are listed once even when two ceilings on the same set are breached together.

CI check (fail if any exact duplicates exist):

$ cargo dry4rust check --max-exact 0
# Exits with code 1 if exact duplicate groups > 0
# Exits with code 0 if within thresholds

CI check with percentage thresholds (fail if >5% of lines are exact duplicates):

$ cargo dry4rust check --max-exact-percent 5.0
# Exits with code 1 if exact duplicate lines exceed 5% of total lines

Exclude test code (inline #[cfg(test)] modules and #[test] functions):

$ cargo dry4rust --exclude-tests report

Exclude test directories by path:

$ cargo dry4rust --exclude tests --exclude benches report

Only report duplicates that are at least 10 lines long:

$ cargo dry4rust --min-lines 10 report

Lower the similarity threshold:

$ cargo dry4rust --threshold 0.7 report

Sub-function Analysis

By default a code unit is a whole function, method or closure. Two functions that share a copy-pasted match arm but differ elsewhere are not duplicates of each other, and nothing is reported.

--sub-function (or -s) also treats each if-branch, match arm, loop body and closure body as a unit in its own right:

$ cargo dry4rust --sub-function report
...
Sub-function exact: 3 groups (6 units)
Sub-function near:  0 groups (0 units)
...
Sub-function Exact Duplicates
=============================

Group 1 (fingerprint: 004522adf0425ce1, 2 members):
  - for body (loop body) in sum_doubled at src/target.rs:70-78
  - for body (loop body) in accumulate at src/target.rs:80-88

Group 2 (fingerprint: 847270a821ab17a2, 2 members):
  - match arm 2 (match arm) in classify_number at src/target.rs:33-53
  - match arm 2 (match arm) in describe_value at src/target.rs:55-66

Each member names the function it came from, and the line range shown is that parent function's — not the branch's. Sub-function units are grouped separately from top-level ones and counted under their own headings, so a function never shares a group with its own branch. --min-sub-nodes (default 5) is the floor a branch must reach to be considered at all; raise it when small branches produce noise.

Two limits are worth knowing before trusting the output, both in docs/OPEN_POINTS.md: sub-function findings restate a function-level one when two functions are already duplicates of each other, and the parent line range means a report cannot be read straight to the branch.

Configuration

Configuration can be provided in three ways (in order of precedence):

1. CLI flags (highest priority)
2. dry4rust.toml in the project root
3. Cargo.toml under [package.metadata.dry4rust]

dry4rust.toml

min_nodes = 15
min_lines = 5
similarity_threshold = 0.85
exclude = ["tests", "benches"]
exclude_tests = true
max_exact_duplicates = 0
max_near_duplicates = 10
max_exact_percent = 5.0
max_near_percent = 10.0

Cargo.toml

[package.metadata.dry4rust]
min_nodes = 15
similarity_threshold = 0.85
exclude = ["tests"]

Configuration Options

Option	Default	Description
min_nodes	10	Minimum AST node count for a code unit to be analyzed. Increase to skip trivial functions.
min_lines	0	Minimum source line count for a code unit to be analyzed. 0 means disabled.
similarity_threshold	0.9	Minimum similarity score for near-duplicate detection. Must be within 0.0..=1.0.
exclude	[]	Path patterns to exclude from scanning (substring match).
exclude_tests	false	Exclude #[test] functions and #[cfg(test)] modules from analysis.
sub_function	false	Also analyse if-branches, match arms, loop bodies and closure bodies.
min_sub_nodes	5	Minimum AST node count for a sub-function unit to be analyzed.
baseline	None	Path to a recorded baseline of inherited duplication, relative to the analysed root.
max_exact_duplicates	None	For check subcommand: maximum allowed exact duplicate groups.
max_near_duplicates	None	For check subcommand: maximum allowed near-duplicate groups.
max_exact_percent	None	For check subcommand: maximum allowed exact duplicate line percentage. Must be within 0.0..=100.0.
max_near_percent	None	For check subcommand: maximum allowed near-duplicate line percentage. Must be within 0.0..=100.0.

A value outside the range its field allows fails the run with a message naming the field, and exits 2. A config file that cannot be read or parsed is passed over, because a project with no configuration looks the same.

Ignoring Duplicates

Some duplicates are intentional (e.g., test helpers, trait implementations). You can ignore them by fingerprint:

# Add a fingerprint to the ignore list
$ cargo dry4rust ignore 396fa8f6b728ff01 --reason "Intentional test helpers"
Added 396fa8f6b728ff01 to ignore list.

# List ignored fingerprints
$ cargo dry4rust ignored
Ignored fingerprints:
  396fa8f6b728ff01 (reason: Intentional test helpers)

# Ignored groups are automatically filtered from reports and checks
$ cargo dry4rust report
# The ignored group will not appear

The ignore list is stored in .dry4rust-ignore.toml in the project root. When cleanup prunes the last entry it removes the file rather than leaving an empty one behind: an empty suppression list says exactly what no file says.

Entries whose fingerprint no longer matches anything go stale — after a refactor, or after an upgrade that changed the fingerprint format. cleanup prunes them:

$ cargo dry4rust cleanup --dry-run   # list them
$ cargo dry4rust cleanup             # remove them

Adopting on a Codebase That Already Has Duplication

ignore is for duplication that is meant to be there. Duplication nobody has got to yet is a different thing, and recording it as intentional — one fingerprint at a time, with a reason invented to fill the field — writes a lie into a file that outlives whoever wrote it.

Record it as a baseline instead. check then fails on what is added, not on what was inherited:

# Record what is already there
$ cargo dry4rust baseline
  exact 396fa8f6b728ff01 (3 members: process_data, compute_total, aggregate)
Recorded 1 groups in dry4rust-baseline.json.

# From now on, a zero ceiling is a gate rather than a wall
$ cargo dry4rust --baseline dry4rust-baseline.json check --max-exact 0
...
Baseline: 1 groups suppressed

Check passed.

Commit dry4rust-baseline.json, or put baseline = "dry4rust-baseline.json" in dry4rust.toml so every run picks it up without the flag. baseline --dry-run lists what would be recorded without writing anything.

A baseline records a group's fingerprint and its member count. A third copy of an already-recorded duplicate makes the group larger than what was recorded, so it is reported — an exact group keeps its fingerprint when a copy joins it, and a baseline keyed on the fingerprint alone would inherit every future copy. Deleting a copy is admitted: progress is not something to fail on.

A baseline that cannot be read — missing, malformed, or written by a version with a different format — fails the run and says so, rather than being treated as empty. Every summary a baseline touched states how many groups it suppressed, so a stale baseline is visible rather than silent.

The reasoning is in ADR-BaselineIsInheritedNotForgiven.

CI Integration

Use the check subcommand in CI pipelines:

# GitHub Actions example
- name: Check for code duplication
  run: cargo dry4rust check --max-exact 0 --max-exact-percent 5.0

On a codebase with existing duplication, add a baseline so the gate measures what the change introduced:

- name: Check for new code duplication
  run: cargo dry4rust --baseline dry4rust-baseline.json check --max-exact 0

Exit codes:

• 0 — Check passed (within thresholds)
• 1 — Check failed (thresholds exceeded)
• 2 — Error (no source files, invalid path, etc.)

What Gets Analyzed

Code Unit	Description
Functions	Top-level fn items
Methods	fn items inside impl blocks
Trait impls	fn items inside impl Trait for Type blocks
Closures	Closure expressions (above the min node threshold)

The scanner automatically:

• Skips target/ directories
• Skips hidden directories (starting with .)
• Respects exclude patterns
• Handles parse errors gracefully (skips unparseable files with a warning)

Documentation

Where	What
docs/ARCHITECTURE.md	The pipeline, the components, and the data model
docs/FORMULA.md	Fingerprinting and the similarity score, precisely
docs/ADRs/	The load-bearing decisions and why they were forced
docs/OPEN_POINTS.md	Where the model is known to be thin, including two silent false-negative sources
docs/ROADMAP.md	Direction, current baseline, and what is left
docs/IMPLEMENTED-FEATURES.md	What each version added, upstream's included
docs/dry4rust-dossier.md	Research on where this tool is going, and what already holds the ground

Development

Requirements: Rust 1.91+ (edition 2024), matching the rust-version every workspace crate declares.

The repository is a workspace: core/ is the published crate, validation/ holds the end-to-end tests, xtask/ runs the stage 2 gates, and fixture/ is the corpus both are pointed at.

cargo build          # Build
cargo test           # Run the suite
cargo clippy         # Lint check
cargo fmt --check    # Format check

Both gates are mandatory after any change under src/ or tests/, and both run the same way on Windows, Linux and macOS:

just stage1   # fmt, clippy under -D warnings across all targets, tests
just stage2   # stern4rust, dry4rust on itself, crap4rust, twin4rust, iceberg4rust

Stage 1 is cargo built-ins only, so it works on a fresh checkout with none of the tools below installed. Note that it lints test targets too (--all-targets) — combined with the pedantic and nursery groups that core and xtask enable, that holds the test files to the same bar as the sources.

Stage 2 is cargo xtask stage2, a real crate rather than a script, so the gate argument lists and failure messages are covered by its own integration tests. xtask is itself a workspace member and is gated like everything else — the crate that runs the gates is not exempt from them.

Its second gate is dry4rust run on itself: this checkout's binary, built from source rather than installed, over core/src with zero ceilings against dry4rust-baseline.json, at a floor of 25 AST nodes. A tool that enforces a check it does not pass is not worth installing. It needs nothing beyond cargo, which is why it is not in the table below.

Everything the two stages need, none of which ships with cargo:

Tool	Install	Needed by
just	cargo install just	both stages
cargo-llvm-cov	cargo install cargo-llvm-cov	stage 2
llvm-tools rustup component	rustup component add llvm-tools	stage 2
cargo-stern4rust	cargo install cargo-stern4rust	stage 2
cargo-crap4rust	cargo install cargo-crap4rust	stage 2
cargo-twin4rust	cargo install cargo-twin4rust	stage 2
cargo-iceberg4rust	cargo install cargo-iceberg4rust	stage 2

cargo-llvm-cov and llvm-tools are what the CRAP gate needs; without them it fails with a bare exit code that says nothing about a missing install.

CI (.github/workflows/ci.yml) runs both stages on Ubuntu, Windows and macOS for every pull request and every push to main.