# hunkpick
[](https://crates.io/crates/hunkpick)
[](https://docs.rs/hunkpick)
[](https://github.com/VitalyOstanin/hunkpick/actions/workflows/ci.yml?query=branch%3Amaster)
[](https://codecov.io/gh/VitalyOstanin/hunkpick)
[](https://github.com/VitalyOstanin/hunkpick/blob/master/LICENSE)
Non-interactive unified-diff hunk picker and splitter — a pure stdin→stdout filter for
staging subsets of changes without interactive prompts. It is the scriptable,
non-interactive alternative to `git add -p`: pick or split hunks by index, range, or
content id inside a pipeline, with no prompts and machine-readable output.
## Table of Contents
- [Why / Motivation](#why--motivation)
- [Installation](#installation)
- [Usage](#usage)
- [list](#list)
- [select](#select)
- [split](#split)
- [Staging recipe](#staging-recipe)
- [Selectors](#selectors)
- [Content ids](#content-ids)
- [Splitting by individual changed lines: `INDEX@L<set>`](#splitting-by-individual-changed-lines-indexlset)
- [Verification](#verification)
- [Input handling](#input-handling)
- [Auto-split and non-overlap](#auto-split-and-non-overlap)
- [Exit codes](#exit-codes)
- [Comparison to filterdiff](#comparison-to-filterdiff)
- [Development](#development)
- [License](#license)
## Why / Motivation
Git's built-in way to stage a subset of hunks is the interactive `git add -p`. It
prompts for each hunk, so it cannot be driven from a script, a Makefile, or an
automated coding agent. The standard **non-interactive** substitute for `git add -p`
has been [`filterdiff`](https://linux.die.net/man/1/filterdiff) from the
[patchutils](https://cyberelk.net/tim/software/patchutils/) suite:
```sh
`filterdiff` works at the granularity of whole hunks as they appear in the diff.
If a single hunk contains multiple independent change runs separated by context
lines, `filterdiff` cannot address them individually — the entire hunk is either
included or excluded.
`hunkpick` fills this gap:
- **Auto-split**: each hunk is automatically decomposed into minimal sub-hunks,
one per contiguous change run. The resulting sub-hunks are addressable individually
by a stable 1-based per-file index.
- **Per-file addressing**: selectors use `path:1,3` syntax, which is unambiguous in
multi-file diffs and composable in scripts. A `*` selects every sub-hunk of a file.
- **Content ids**: each sub-hunk also carries a content-derived `@<id>`. It hashes only
the file paths and the sub-hunk's changed (`+`/`-`) lines — not its context or the `@@`
line numbers — so the id stays the same across a re-diff even when an edit elsewhere
shifts its line numbers or staging a neighbour rewrites its surrounding context. An
agent can capture `@<id>` once and keep using it across a staging loop. (Byte-identical
changes share an id; `list --json` reports `id_count`. See [Content ids](#content-ids).)
- **Correct anchors for a partial selection**: leaving a sub-hunk out changes how many
lines the result adds or removes, so every later hunk's new-side (`+`) start is
recomputed from the emitted hunks alone. `git apply` searches from that position; a
value carried over from the input diff would drift and, where the surrounding context
repeats, land the change in the wrong place.
- **Built-in verification**: the result diff is checked for internal consistency by
default; an optional `git apply --check` run is available on demand.
- **Git-agnostic**: `hunkpick` reads a diff from stdin and writes to stdout. It does
not call `git diff` itself and works with any diff source (git, Mercurial, SVN, or
plain `diff -u` output). Application to the index is left to the caller via
`git apply --cached`.
- **Encoding-agnostic**: the diff is processed as raw bytes end to end. Content in any
encoding — including invalid UTF-8 — round-trips byte-for-byte; only the path and
preview shown by `list` are decoded lossily for display.
- **Cross-platform, including Windows**: `filterdiff`/`patchutils` is a Unix toolchain
that is awkward to obtain and run on Windows. `hunkpick` is a single self-contained
binary built for Linux, macOS, and Windows (`x86_64-pc-windows-msvc`), with no runtime
dependencies.
- **AI-agent integration**: the first consumer is an automated coding agent. Staging a
precise subset of a diff programmatically needs non-interactive operation (no
`git add -p` prompts), a stable machine-readable `--json` listing, deterministic
per-file sub-hunk addressing, and structured exit codes — none of which the
interactive `git add -p` or the whole-hunk-only `filterdiff` provides.
## Installation
**From crates.io:**
```sh
cargo install hunkpick
```
**Prebuilt binary via [cargo-binstall](https://github.com/cargo-bins/cargo-binstall)** (downloads the release artifact from GitHub instead of compiling):
```sh
cargo binstall hunkpick
```
Prebuilt binaries are published for `x86_64-unknown-linux-gnu`, `aarch64-apple-darwin`, `x86_64-apple-darwin`, and `x86_64-pc-windows-msvc`. On other targets `cargo binstall` falls back to a source build.
Every archive on the [releases page](https://github.com/VitalyOstanin/hunkpick/releases) carries a `.sha256` companion and a build provenance attestation. The checksum says the download arrived intact; the attestation says which workflow run, at which commit of this repository, produced those exact bytes — which the checksum cannot, since it is served from the same page as the archive:
```sh
gh attestation verify hunkpick-<version>-x86_64-unknown-linux-gnu.tar.gz --repo VitalyOstanin/hunkpick
```
**From source:**
```sh
git clone https://github.com/VitalyOstanin/hunkpick.git
cd hunkpick
cargo build --release
# binary is at target/release/hunkpick
```
Minimum supported Rust version: **1.85**.
## Usage
All subcommands read a unified diff from **stdin** by default and write to **stdout**.
Use `-i, --input FILE` to read from a file instead (`-` means stdin). See
[Input handling](#input-handling) for the size limit.
### list
Parse the diff, auto-split each hunk into minimal sub-hunks, and list them per file
with their 1-based per-file index.
```sh
# Human-readable output (default)
# Machine-readable JSON
# Control colorisation
Colour in the default `--color auto` mode follows stdout: on when it is a
terminal, off when piped. The `NO_COLOR` environment variable (any non-empty
value) forces it off; `CLICOLOR_FORCE` (any non-empty value) forces it on even
when piped. `NO_COLOR` takes precedence when both are set. An explicit
`--color always|never` overrides all of these.
**Example human output:**
```
src/main.rs
[1] 114ccaaa7ce6c0f1 @@ -10,4 +10,4 @@ +1 -1 +let x = 1;
[2] 8002dd73f0dfd2f4 @@ -20,6 +20,6 @@ +1 -1 +fn bar() {
```
The 16-hex token after the index is the sub-hunk's **content id** (see
[Selectors](#selectors)). Each line then shows the hunk header, the `+N -M` change
counts, and a preview of the first changed line. A sub-hunk that only adds lines (a
file creation or a pure append, with no context and no deletions) is flagged `[+add]`
between the counts and the preview — the same property the JSON listing reports as
`addition_only`:
```
src/new_file.rs
[1] 3f1c0a52d7b94e68 @@ -0,0 +1,12 @@ +12 -0 [+add] +fn main() {
```
Text a terminal would act on rather than show — escape sequences, control bytes,
bidirectional overrides — is escaped in this listing (`\x1b`, `\u{202e}`), so a diff
being filtered cannot repaint or reorder what you are reading.
**The JSON listing does not do this.** Its text fields (`path`, `header`, `preview`,
`changed_lines[].text`) reproduce the diff's own bytes: JSON escaping covers control
characters, but a bidirectional override survives it and comes back out of any parser as the
character it was. That is deliberate — the machine-readable mode reports what the diff says,
not a display-safe rendering of it — so **a consumer that prints these fields to a terminal
must escape them itself**. Prefer the human listing when a person is reading the output.
All four are also lossy for a path or a line that is not valid UTF-8 (legal on
Unix): JSON must be UTF-8, so undecodable bytes become `U+FFFD`. A path taken from `list
--json` therefore does not necessarily round-trip back into a `path:N` selector. For such a
file, address the sub-hunk by its content id (`@id`), which is computed over the raw bytes and
is exact.
**JSON schema** (`--json`): array of file objects, each with `path`, `binary`, and
`hunks` (array of sub-hunk objects with `index`, `id`, `id_count`, `old_start`,
`old_lines`, `new_start`, `new_lines`, `added`, `deleted`, `addition_only`,
`changed_lines`, `header`, `preview`). `id_count` is how many sub-hunks across the whole
patch share that `id` (`1` = unique). `addition_only` is `true` when the sub-hunk is all
additions (a file-creation or pure-append block). `changed_lines` is the sub-hunk's
changed (`+`/`-`) lines in body order, each `{ i, kind, text }`: `i` is the 1-based index
for `select INDEX@L<set>`, `kind` is `"add"` or `"del"`. The `i` indices are positional —
they renumber after each staged round, unlike the sub-hunk `id`, so re-run `list --json`
each round (there is no stable per-line id).
Binary files are listed with `"binary": true` and an empty `hunks` array.
### select
Emit only the chosen sub-hunks as a valid unified diff.
```sh
# Select sub-hunks 1 and 3 from a single-file diff
# Select sub-hunks from specific files in a multi-file diff
# Same when the diff is taken over an explicit file list (git diff file1 file2 fileN).
# With more than one file, every selector must carry a path: prefix (a bare index is
# only allowed for a single-file diff).
# Select a range
# Select every sub-hunk of a file (or the whole single-file diff)
# Select by content id (from `list --json`), stable across re-diffs
# Content ids work across a multi-file diff too: the file path is part of the id, so
# an id addresses the change in its own file (the same edit elsewhere gets another id).
# Several ids at once, mixed with path: selectors. Read the ids from `list --json` first
# (the machine-readable form, intended for tooling):
```
A binary file referenced by any selector index is emitted whole.
### split
Split one original hunk (addressed by its 1-based index over the file's original
hunks, before auto-splitting) at specified new-file line numbers. The line numbers
must fall on context lines. The result is the complete patch with that hunk replaced
by the pieces.
```sh
# Split original hunk 1 in a single-file diff at new-file line 5
# Same for a named file in a multi-file diff
# With git verification. The check reads the working tree, so it belongs on a patch file
# against a tree at the pre-patch state — not on a `git diff |` pipeline, where it would
# reject a correct result. See "Git apply check (optional)".
hunkpick split 1 --at 5 -i patch.diff --verify-result-diff-git -C /path/to/clean/checkout
```
### Staging recipe
```sh
# 1. Inspect what sub-hunks are available
# 2. Stage only sub-hunks 1 and 3
Splitting one file's mixed changes into several semantic commits, addressing
sub-hunks by content id. Bare indices renumber after each staging, but a `@<id>`
stays valid across the re-diff (see [Content ids](#content-ids)), so the listing
is captured once and never re-read:
```sh
# 1. Capture the ids once. `id_count` flags any id that selects more than one.
# 2. Stage and commit each group by @id (one or more ids each), re-running git
# diff each round. The ids from step 1 remain valid even though staging
# renumbers the bare indices.
# 3. Whatever is left is the last group; `*` takes the remaining sub-hunks.
the 30 lines the first round left unstaged, numbered from 1 again. Reusing the first round's
numbers is a usage error (exit 2, "changed-line index 120 is out of range"). Selecting the
whole remainder — `1` or `*` — works as well.
A sub-hunk addressed by `@L` must be addressed **once per invocation**: combining it with
another `@L`, or with a whole selection of the same sub-hunk, is a usage error
(exit 2) — the pieces would carry inconsistent line numbers. Stage further pieces in
later `diff → stage → re-diff` rounds. A partial `@L` on an entry that deletes the file is a
usage error for the reason given above.
Example — separate a replacement's removals from its insertions. `list --json` shows the
changed lines and their indices:
```json
"changed_lines": [
{ "i": 1, "kind": "del", "text": "a" },
{ "i": 2, "kind": "del", "text": "b" },
{ "i": 3, "kind": "add", "text": "A" },
{ "i": 4, "kind": "add", "text": "B" }
]
```
```sh
```
The second round re-runs `git diff`: after the deletions are committed the sub-hunk holds
only the two additions, now numbered 1 and 2.
## Verification
### Internal consistency check (default)
After `select` or `split`, `hunkpick` verifies the result diff for internal
consistency: `@@` header counts match the body line counts, hunks within each file
are ordered, their old-file ranges do not overlap, and each hunk's new-side (`+`) start
follows from its old-side start plus the net size of the hunks emitted before it. This
check runs by default and requires no git repository.
To disable it:
```sh
### Git apply check (optional)
Pass `--verify-result-diff-git` to additionally run `git apply --check` on the result
diff before emitting it. This confirms the diff applies cleanly to the working tree.
**Read that literally: to the working tree.** `git apply --check` without `--index` compares
against the files on disk, not against the index. In the staging pipeline this README is built
around — `git diff | hunkpick select ... | git apply --cached` — the working tree already
contains the edits the diff describes, so git reports `patch does not apply` and hunkpick exits
70 for a result that is perfectly correct. The flag is for the case where the tree *is* at the
state the diff expects: checking a patch file against a clean checkout, or pointing `-C` at
such a tree. It is not a routine safety net for the staging loop — the internal check above is,
and it runs by default.
```sh
# The tree is at the pre-patch state: the check is meaningful here.
hunkpick select 1 -i patch.diff --verify-result-diff-git -C /path/to/clean/checkout
```
Use `-C <DIR>` to specify the working tree directory the check runs against (default:
current directory). `-C` requires `--verify-result-diff-git`; passing `-C` alone is a usage
error.
```sh
hunkpick select 1 -i patch.diff --verify-result-diff-git -C /path/to/clean/checkout
```
### Verification failure
On any verification failure, `hunkpick` writes a diagnostic to stderr, writes
nothing to stdout, and exits with code **70**. A `git` that cannot be started at all is a
different matter — the check never ran, so that is exit **74**, not a verdict on the diff.
## Input handling
### Source
By default the diff is read from stdin. `-i, --input FILE` reads from a file instead;
`-i -` is an explicit stdin. The flag is available on every subcommand and may appear
after it:
```sh
hunkpick list --input changes.diff
```
Run without a pipe and without `-i` and hunkpick reads the terminal, as any filter does, after
writing one line to stderr saying so — a paste-and-Ctrl-D still works, but a forgotten pipe no
longer looks like a hang. In a pipeline stdin is not a terminal, so nothing is printed.
### What the input may be
hunkpick reads a two-sided unified diff: what `git diff`, `git format-patch`, `diff -u`, and
Mercurial or Subversion produce. Within that, the input is passed through unchanged — what
comes in comes back out byte for byte, including CRLF endings, a `\ No newline at end of file`
marker, the mail head and footer of a `format-patch` output, a full binary patch
(`git diff --binary`), and a diff that arrived without a final newline.
The diff has to arrive as a UTF-8 (or otherwise ASCII-compatible) byte stream. A UTF-8 BOM is
harmless and comes back out unchanged, but a UTF-16 or UTF-32 one is refused with a message
naming the encoding — `git diff > patch.diff` in Windows PowerShell 5.1 produces UTF-16LE, and
`iconv -f UTF-16LE -t UTF-8` makes it readable. hunkpick does not re-encode input: the whole
point of the byte-for-byte pass-through is that what comes out is what went in.
One format is deliberately not read: the **combined diff** git writes for a merge commit
(`git show <merge>`, `git diff --cc`, `@@@` headers). Its body has one marker column per
parent, so a sub-hunk of it is not a two-sided change and cannot be addressed or sliced.
hunkpick rejects such input as a usage error (exit code 2) rather than reading it as
something it is not.
### Size limit
Input (from stdin or a file) is capped at **64 MiB** by default to guard against an
accidentally unbounded stream. Exceeding the limit is a usage error (exit code 2).
Override with `--max-input-bytes N`; `0` disables the limit.
The limit bounds the input, not the memory: hunkpick keeps the whole parsed diff in memory
with one allocation per line, so peak RSS is a multiple of the input size, and the multiple
grows as the average line gets shorter. Measured on a release build (`/usr/bin/time -f %M`):
| 18 MB, lines of ~13 bytes | 196 MiB | 349 MiB |
| 61 MiB, lines of ~60 bytes | 341 MiB | 534 MiB |
| 63 MiB (at the limit), ~40 B | 427 MiB | 713 MiB |
That is 6x–19x the input, and the run at the default limit takes well under a second
(0.49 s and 0.88 s for the row above). Plan for around a gigabyte when the lines are short,
and raise `--max-input-bytes` only with that in mind.
```sh
hunkpick list --max-input-bytes 268435456 -i huge.diff # raise to 256 MiB
hunkpick list --max-input-bytes 0 -i huge.diff # no limit
```
Note: the working-set memory is several times the input size (the input buffer, the
parsed model, and the emitted diff coexist), so a 64 MiB input corresponds to a few
hundred MiB of peak RAM. Lower the limit if you run in a memory-constrained environment.
One other limit exists and is not configurable: the selectors of one invocation may name at
most **1 048 576** (2^20) indices *between them*, so `1-99999999` is a usage error (exit code 2)
rather than an allocation, and so is a long list of smaller ranges that adds up past the
ceiling. The allowance is shared because the number of selectors is bounded only by the length
of the command line. It is far above the sub-hunk count of any real diff — there is no
legitimate reason to raise it, hence no flag.
### Validation
`hunkpick` reads the input as raw bytes and validates it before parsing:
- **Empty or whitespace-only input** is a no-op: nothing is written and the exit code
is 0, for every subcommand.
- **Binary input** (any NUL byte) is rejected with a diagnostic and exit code 2.
- **Text with no diff marker** (no line starting with `diff --git `, `--- `, `+++ `,
`@@ `, or `Binary files `) is rejected with exit code 2.
Valid diff content is never decoded as UTF-8 internally, so lines in any byte encoding
(or with invalid UTF-8) pass through unchanged.
## Auto-split and non-overlap
`hunkpick` decomposes each hunk into sub-hunks automatically at boundaries between
adjacent change runs. A "change run" is a maximal contiguous sequence of `+`/`-`
lines. Context lines between change runs become the split boundary.
**Non-overlap guarantee**: sub-hunk old-file ranges are strictly non-overlapping.
The boundary context (lines between two change runs) becomes the *trailing* context
of the earlier sub-hunk. The later sub-hunk starts directly at its change run, with
no leading copy of the boundary context.
This differs from `git add -p`, which can share context between adjacent hunks
because it applies each hunk individually. `hunkpick select` emits all selected
sub-hunks as a single combined patch applied in one `git apply` call; overlapping
old-file ranges would cause `git apply` to reject the patch.
**Round-trip property**: selecting all sub-hunks for a file produces a diff that
applies equivalently to the original hunk. The output is not byte-identical to the
original (one hunk becomes several), but the applied result is the same.
## Exit codes
| 0 | Success |
| 2 | Usage error: bad flag, bad selector, parse error, binary/non-diff input, input over size limit, `-C DIR` not naming a directory |
| 70 | Verification failure (internal consistency, `git apply --check` rejecting the result, or a violated internal invariant) |
| 74 | I/O error: reading stdin, writing stdout, or not getting a verdict out of `git` for `--verify-result-diff-git` — it would not start, or it failed on its own before looking at the diff |
| 130 | Interrupted by SIGINT (default signal disposition: 128 + 2) |
| 143 | Terminated by SIGTERM (default signal disposition: 128 + 15) |
A reader that closes the pipe first (`hunkpick list | head`) is not an error: the write
ends the run with code 0, so the tool composes with `set -o pipefail`.
## Comparison to filterdiff
| Binary file pass-through | ✅ | ✅ |
| Select whole hunks from a diff | ✅ | ✅ |
| Works with any diff source (not git-specific) | ✅ | ✅ |
| Address sub-hunks by per-file index | ❌ | ✅ |
| Auto-split hunks at change-run boundaries | ❌ | ✅ |
| Built-in result verification | ❌ | ✅ |
| Explicit hunk split at a named line | ❌ | ✅ |
| Machine-readable listing (JSON) | ❌ | ✅ |
| Split any sub-hunk by individual changed lines | ❌ | ✅ |
## Development
Contributions are welcome. The crate has no build-time code generation and no external
runtime dependencies, so the standard cargo workflow applies.
```sh
# Run the unit and integration tests through nextest (the documented runner).
cargo t
# Run the doc tests; nextest does not execute them.
cargo t-doc
# Lint with all warnings denied (the CI gate).
cargo clippy --all-targets --all-features -- -D warnings
# Check formatting (CI verifies this; use `cargo fmt --all` to apply).
cargo fmt --all --check
# Verify the code still compiles on the minimum supported Rust version (1.85).
# `--all-targets` includes the tests, so the dev-dependencies are checked too.
cargo +1.85 check --all-targets --all-features
# Check the public API against the version on crates.io (needs cargo-semver-checks
# and a network connection). Before 1.0 a breaking change is allowed, provided the
# minor version goes up with it; this gate checks that it did.
cargo semver-checks check-release
```
`t` and `t-doc` are aliases from [`.cargo/config.toml`](https://github.com/VitalyOstanin/hunkpick/blob/master/.cargo/config.toml). The list above is
the everyday loop, not the full set of gates: CI also lints the `fuzz` workspace, which declares
a workspace of its own that no `--all` here reaches, generates the docs and checks the
third-party licences. [`CONTRIBUTING.md`](https://github.com/VitalyOstanin/hunkpick/blob/master/CONTRIBUTING.md) carries the complete list. The CI workflow
([`.github/workflows/ci.yml`](https://github.com/VitalyOstanin/hunkpick/blob/master/.github/workflows/ci.yml)) runs these checks with
[`cargo-nextest`](https://nexte.st/) for the unit/integration tests and `cargo test --doc` for
doc tests. Test runner limits (per-test timeout and thread count) live in
[`.config/nextest.toml`](https://github.com/VitalyOstanin/hunkpick/blob/master/.config/nextest.toml) and apply only under nextest; without it
installed, use `cargo test --all-features -- --test-threads=4`. Please keep tests fast and
hermetic — several tests shell out to `git apply --check` and require `git` on `PATH`.
`cargo t` includes generated tests: a differential suite that compares hunkpick with real git
over generated diffs, and property tests over shapes git will not produce on demand. The fuzz
targets in [`fuzz/`](https://github.com/VitalyOstanin/hunkpick/blob/master/fuzz) need nightly and are run separately, through
[`scripts/fuzz-all.sh`](https://github.com/VitalyOstanin/hunkpick/blob/master/scripts/fuzz-all.sh) (`FUZZ_SECONDS=60 scripts/fuzz-all.sh parse`
for one target, one minute); CI builds each target on every push and runs a longer search twice
a week. See [`CONTRIBUTING.md`](https://github.com/VitalyOstanin/hunkpick/blob/master/CONTRIBUTING.md) for what each kind covers, why the
command needs a toolchain override, an explicit triple, a corpus directory and a hang timeout,
and how a failure is reproduced.
## License
MIT. See [LICENSE](LICENSE).
The released binary statically links third-party crates (all MIT / Apache-2.0 / Unicode-3.0).
Their license texts and copyright notices ship inside every release archive as
`THIRD-PARTY-NOTICES.md`, generated at release time by
[`scripts/generate-notices.sh`](https://github.com/VitalyOstanin/hunkpick/blob/master/scripts/generate-notices.sh) from
[`about.toml`](https://github.com/VitalyOstanin/hunkpick/blob/master/about.toml). Installing from crates.io needs no such file — cargo resolves the
dependencies' own licenses from `Cargo.lock`.