# AGENTS.md - csift operating manual
Project-specific operating manual for any AI agent (Claude Code, Codex, Cursor) working in this repo. Read this first when a conversation opens. This file is sufficient for **repo mechanics + invariants** - directory map, dispatch wiring, test harness, the rules you must not break - so a fresh mid-size model can add/modify a subcommand or fix a bug respecting every invariant without spelunking the source. For the exact **behavioural contract** of a subcommand (precise flag semantics, output shape, edge cases), open the matching `SPEC.md` section 6 subsection; SPEC is the authority on *what* each command does, this file on *how* to work in the repo.
> **About this file.** `AGENTS.md` is the canonical, vendor-neutral filename. `CLAUDE.md` is a **symlink to `AGENTS.md`** - edit `AGENTS.md` only; the symlink follows. **Companion doc:** [`SPEC.md`](./SPEC.md) (~175KB / ~1240 lines) is the product/behaviour spec - open it whenever you change search/verbatim/recover/etc. behaviour. Concrete pointers: SPEC **section 4** = record semantics, **section 5** = the full label taxonomy (`search -t` role.class.sub), **section 6** = per-subcommand specifications (flags, output shape, edge cases), **section 7** = the performance contract, **section 8.2** = the JSON envelope v2; `## 6 Subcommand specifications` opens with the per-version CHANGE LEDGERS, ordered newest-first (the newest entry is authoritative; historical entries describe their own versions' surfaces). This file = how to work in the repo; SPEC.md = what to build. **The FIVE-DOCUMENT CONTRACT** (user-ruled, v0.5.1): `SKILL.md` = the LLM manual (density + weak-attention retrievability) · `--help` = the human (CLI-proficient) manual — INFORMATION-PARITY with SKILL, never thinner, but plain prose with stronger structure/layout (help IS surface: changing it bumps the version triple) · `README.md` = promotion · `SPEC.md` = design intent · this file = maintenance. When code and a doc disagree, **code wins** - re-verify, then fix the doc.
---
## 1. What csift is
**csift - "ripgrep for Claude Code session transcripts".** A fast Rust CLI that **lists** and **regex-searches** Claude Code session `.jsonl` files under `~/.claude/projects/`, plus recovers files, reconstructs compaction-clipped turns, maps subagents, and extracts pasted images.
- **Primary consumer is an LLM** - a Claude Code agent searching/recovering its own or a peer session. Output is clean, token-efficient, regex-driven. Default output is human/LLM-readable (session/turn/label/timestamp headers); `--format json` is the machine format (there is no bare `--json` flag). `search --raw` / `show --raw` emit VERBATIM jsonl lines (the escape hatch for unrendered fields).
- **Explicitly NO BM25 / embeddings / semantic search.** Pure regex only. Lexical tokenisation across scripts (CJK / multi-byte) is intractable for scoring; regex is the strength and the whole point. Never add semantic search.
- **Thirteen subcommands** (`Command` enum, `cli.rs`): `list`, `search`, `show`, `stats`, `agents`, `whoami`, `files`, `recover`, `plan`, `verbatim`, `image`, `status`, `wait` (+ ONE hidden `Turns` tombstone variant, v0.6.4 — never runs, always bails with the rename pointer to `verbatim`; keep it bare + hidden). `show` = record FETCH (by `--line`/`--uuid`/`--turn`, one transcript, rendered full or `--raw` verbatim bytes — search no longer fetches; `--turn N|A..B` fetches EVERY record of the named turn(s) in the SAME `·tN` numbering `search`'s `<tok>·tN` headers print (v0.7.0: `<tok>` = a STABLE first-8 id-prefix token, an `@` target as-is; the `sN` legend/ordinals are GONE), so `show --turn -3..` is the tail-peek/monitoring path — "read a session's recent turns from the live transcript"); `verbatim` = the compaction-fidelity specialist (reconstruct the verbatim user/assistant turns a compaction summary clipped — the command FORMERLY named `turns`, a zero-BC rename: `csift turns …` is now an unrecognized subcommand; the module file stays `src/turns.rs`, handler `turns::run_verbatim`, args `cli::VerbatimArgs`, variant `Command::Verbatim`); `stats` = one-scan per-session aggregates (tokens by model — usage counted ONCE per API message via `message.id` per-field-MAX dedupe, v0.9.2: CC repeats the identical usage object on every per-block record, so per-record sums over-read 2-3.5x; dedupe is PER FILE, the scope TOTAL sums files / narration blocks per model / tools / turns / span / compactions / a whole-file line-type census). (There is NO `pending` subcommand — the elicitation sidecar it once read is merged TRANSPARENTLY into every record-reading surface; see §3.10.) `status`/`wait` (v0.9.0) are the LIVE-TRUTH pair — point-in-time, explicitly non-reproducible, the one documented departure from the forensic contract: `status` joins the harness session registry (`<claude-home>/sessions/<pid>.json`, transition-writes only) + the transcript tail state machine (unreturned tool call = in flight) + a per-platform pid probe (`ps`/`/proc` on unix, PowerShell `Get-Process`/`tasklist` on Windows; reuse-guarded via procStart - UTC asctime on unix, a FILETIME tick count on Windows - vs the probe's own start instant; a foreign `pidDomain` row is never probed) into one of SEVEN verdicts with named evidence (v0.10.0 adds `idle-background-open`: a clean end_turn with counted background tasks not returned - never a `stop`) — child lanes classify `in-flight|generating|settled` (v0.9.4: `generating` = tail-record age <=300s AND last stop_reason != end_turn; the retired mtime `active` state is GONE), settled lanes FOLD to a count, a tasks section reads `<claude-home>/tasks/` (both the full-uuid and `session-<first8>` dir forms), a BACKGROUND section (v0.10.0, `live/background.rs` + `background_scan.rs`) scans the WHOLE main transcript for backgrounded shells / async agents / Monitor arms and their `<task-notification>` completions (joined by the launching tool_use id across three carriers incl. `queue-operation` lines and `queued_command` attachments; a subagent target also reads its PARENT main; the harness writes nothing about a running shell at EOT, so this scan is the only instrument), under the LENS `--background-since WHEN` / `--ignore-background RE` shared by both commands, and a LAST section prints the newest prompt + reply as excerpts under the not-a-review warning; `wait` polls with STRICT post-start baseline semantics, REQUIRES `--timeout` (v0.10.0 BREAKING - a background task may never return by design), censuses the activity that landed while it waited, and exits 124 on timeout with the at-exit report (the one exit-code exception). `--until notification` and the census read the same three pulse carriers the background scan joins (a user record for the idle delivery; a `queue-operation` enqueue line and a `queued_command` attachment for a pulse absorbed mid-turn, which never becomes a user record - v0.10.2).
- **Subagent spanning default:** `list`/`search`/`stats`/`files`/`recover`/`plan`/`image`/`status`/`wait` span each session's subagent transcripts by default. EVERY span command answers the SAME two switches: `--no-subagents` (restrict) and `--subagents` (affirm — a no-op on the default-on commands, kept so the pair is uniform; the contradictory pair is a clap `conflicts_with` parse error). `verbatim` is the exception (its per-session budget MULTIPLIES, so it defaults to top-level thread only; `--subagents` opts IN — the field is named `include_subagents` but the FLAG is `--subagents` — and a target is REQUIRED). `agents` LISTS subagents as its targets, so it hard-rejects BOTH span flags with a pointed error (its `--subagents` twin is a hidden no-op that exists only to be rejected). EVERY default-on `*Args` exposes `want_subagents() -> bool` (= `self.subagents || !self.no_subagents`) feeding `SubagentScope::from` (`true` -> `WithSubagents`, `false` -> `TopLevelOnly`); `verbatim` returns `self.include_subagents && !self.no_subagents`. `SubagentScope` has exactly those two variants. (There is no `--subagents-only` mode.)
---
## 2. Targeting grammar (@-tokens) - shared resolver `path::resolve_session_files`
Every session-operating subcommand resolves its target(s) through ONE function:
`path::resolve_session_files(paths: &[PathBuf], scope: SubagentScope, caller: Caller) -> Result<Vec<PathBuf>>` (src/path/). It returns the concrete jsonl files to scan. Multi-target commands actually call the thin wrapper `path::resolve_targets_with_session_list` (src/path/), which first UNIONs in the `--sessions-from <FILE|->` id list (`extend_with_session_list`, src/path/ — bare or `@`-prefixed uuid/prefix/agent-id tokens, each appended as an `@<id>` positional; an EXPLICIT empty list resolves to an EMPTY scope, never all-projects). `whoami` is the only subcommand with no target (reads `$CLAUDE_CODE_SESSION_ID`).
Each positional token (for `search` the FIRST positional is PATTERN, sessions come as later `@<uuid>` positionals):
- **real cwd path** / `.` -> encode it, locate the matching `~/.claude/projects/<encoded>` dir.
- **encoded dir token** (`-Users-...`, or the Windows drive shape `C--Users-...`) -> used directly; a bare `--`-leading token is rejected as a mistyped flag by `parse_project_target` (src/cli/ — a UNC-encoded `--server-...` dir is targeted via `@--server-...`); a drive-shaped token that matches no projects dir falls through to real-path resolution (it can be a genuine relative path).
- **`~/.claude/projects/<encoded>` path** -> used as-is.
- **`@<uuid>`** -> that one top-level session (`is_uuid`).
- **`@<uuid-prefix>`** e.g. `@13d9645a` -> the leading hex of a uuid (4..=11 dashless chars); unique-resolved against enumerated sessions, else ambiguity error (`is_uuid_prefix`).
- **`@main`** -> the CALLING TOP-LEVEL session, read from `$CLAUDE_CODE_SESSION_ID` (`resolve_env_session`).
- **`@trap:<marker>`** -> the CALLING SUBAGENT (or main thread). CC withholds a subagent's OWN id from its Bash env, so a running subagent cannot name itself. The caller INVENTS a unique literal marker, embeds it in *this very* csift command; csift finds the transcript whose SHELL `tool_use` carries it (`resolve_trap` — `Bash`, or Windows' separate `PowerShell` tool, v0.7.4). STRICT marker grammar (`validate_trap_marker`, src/path/): EXACTLY 3 CamelCase words (no single-letter / ALLCAPS like `HTML`/`USB`) + exactly 4 non-trivial trailing digits, hand-invented one-shot + CONTEXT-INDEPENDENT, shaped like `@trap:JollyShinyBrook4283` - a RESERVED example csift hard-refuses (don't copy it; a pasted example self-collides, so invent your own). Routes on EVERY target-taking subcommand via the shared `resolve_session_files`, AND on `whoami` (`whoami @trap:<marker>` -> the subagent's full UPSTREAM ancestry chain self -> ... -> top-level root, the walk-UP mirror of `agents`' walk-DOWN; env-independent; `path::resolve_trap_who` reuses `agents::topology_for_session` + `parent_agent_id`). Malformed/ambiguous -> hard bail with guidance. **TIMING (subagent verified live 2026-07-12; main-lane mechanism re-measured 2026-08-29):** a SUBAGENT's transcript flushes per content block, so the launching tool_use is on disk at dispatch and a first try resolves. The MAIN conversation's record is an ASYNC FLUSH of the completed assistant message, landing ~1-3.4s AFTER dispatch - a RACE, not a wait (the 2026-07-12 receipt, a 3s in-command sleep seeing 0 on disk, sat exactly at the window's edge and had no resolving power; a 263s command was later observed with its unpaired tool_use on disk 39s in). csift finishes inside that window, so a top-level FIRST use normally misses and a re-run of the SAME marker resolves - the no-match error routes both paths, `@main` FIRST (the main thread's direct answer, no race), then the literal-marker check, then retry-as-lane-confirmation (a NEW, SEPARATE invocation: a second attempt inside ONE script runs in that same unlanded window, whose width is invisible from inside; a FRESH marker restarts the race). A @trap that RESOLVES to the main transcript prints a stderr lane note (stdout unchanged, exit 0). The same flush window bites a UserPromptSubmit hook asking for the PREVIOUS prompt: at that instant the CURRENT prompt's record may or may not be on disk yet (both observed live), so a hook-side "previous prompt" query excludes hits younger than now-3s (the measured window is ~1-3.4s) and takes the newest survivor.
- **`@<agent-id>`** -> a subagent + its TOPOLOGICAL subtree (unless `--no-subagents`). Accepts EITHER a bare hex (built-in/workflow, len >=12, `is_bare_subagent_hex`) OR a name-embedded **teammate** id like `aVSRepro-68a2a1661c9390c1` (`is_teammate_agent_id`, path.rs). The shared gate is `path::is_subagent_id` (= `is_bare_subagent_hex || is_teammate_agent_id`); use it (NOT `is_bare_subagent_hex`) for any new `@<id>` / pin routing. EVERY id `csift agents` prints round-trips here regardless of shape — a teammate id used to fall through to the project-dir branch and fail; the `is_subagent_id` gate (src/path/) via `pins_single_session` (src/path/) is what fixed it, and the teammate arm tolerates dashes in the NAME (`aP1-engine-9cf2f06d6235ca64` is real data — the head accepts `[A-Za-z0-9-]` with an explicit `!is_uuid` guard so an `a`-led uuid can't slip in). Downstream `resolve_agent_subtree` matches a subagent by EXACT `session_id_from_path`, so it already handled non-hex ids; only the entry gate needed widening.
- **`*.jsonl` file** -> one transcript; a path containing a `subagents` component is treated as a subagent (+ subtree), else a top-level session.
- **bare uuid (no `@`)** is NOT special - it falls to the path branch and fails as "no project dir named <uuid>". Always prefix with `@`.
- **unrecognized `@`-shape** (a 1-3-char hex prefix, a dashed fragment, any non-id token not leading with `-`) -> HARD ERROR naming the @-grammar (the `resolve_session_files` catch-all arm, v0.6.1) - it never falls through to path resolution; only a `-`-leading token OR the Windows drive shape (`<letter>--...`, v0.7.3) is treated as an encoded project-dir name (`@-Users-...` and `@C--Users-...` both work). A 1-3-char hex token gets the dedicated "too short for a session-uuid prefix - needs 4-11" message.
Resolution is **fail-closed/fail-loud**: an id that was pinned but matches no file bails (never a silent empty). `0 paths` => every project under the root. `Caller` is a subcommand discriminator threaded through for FUTURE subcommand-aware remediation text - but it is currently **INERT** inside `resolve_session_files` (`let _ = caller;`, src/path/); pass the right variant for forward-compat, but no remediation text varies by it today. `SubagentScope` (src/path/) has exactly two variants: `WithSubagents` (default) / `TopLevelOnly`; `SubagentScope::from(bool)` maps `true->WithSubagents`, `false->TopLevelOnly`. (The former `SubagentsOnly` variant + `files`' `--subagents-only` flag were both removed in the crate-wide span-flag cleanup.)
---
## 3. The Claude Code jsonl domain model (load-bearing - verified against real data)
### 3.1 Data location + path encoding
```
~/.claude/projects/<ENCODED_CWD>/<session-uuid>.jsonl # a session transcript
~/.claude/projects/<ENCODED>/<uuid>/subagents/agent-<hex>.jsonl # (A) built-in Task/Agent subagent
~/.claude/projects/<ENCODED>/<uuid>/subagents/workflows/wf_<id>/agent-<hex>.jsonl # (B) workflow / OMC subagent (dominant)
~/.claude/projects/<ENCODED>/<uuid>/subagents/workflows/wf_<id>/journal.jsonl # (C) workflow EVENT log - NOT a transcript (excluded)
~/.claude/projects/<ENCODED>/<uuid>/subagents/**/*.meta.json # {agentType, description?} companions
~/.claude/projects/<ENCODED>/<uuid>/tool-results/<id>.txt # externalised tool output
```
**Encoding** (deterministic forward, lossy reverse; CC-exact since v0.7.3, extracted from the 2.1.228 binary): the cwd is NFC-normalized, then every UTF-16 CODE UNIT outside `[A-Za-z0-9]` becomes a single `-` (an astral char = two surrogate units = TWO dashes). NO consecutive-dash collapsing; `.`,`/`,`_`,space all -> `-`. E.g. `/Users/u/Projects/w_app` -> `-Users-u-Projects-w-app`; a `/.claude/` segment -> `--claude-`; WINDOWS `C:\Users\x` -> `C--Users-x` (letter-led!) and UNC `\\server\share` -> `--server-share`. Reverse is lossy (`-` could be `/`,`_`,`.`) so we NEVER reverse. Accept either a real fs path (encode + locate) or a direct encoded dir. **>200-char cap:** CC caps the encoded name at `MAX_SANITIZED_LENGTH = 200` (src/path/); a longer cwd is stored `<first-200>-<hash>`. The hash is NOT reconstructible (CLI uses Bun.hash, SDK djb2 - different digests), so a long path is resolved by **prefix-scanning** the projects root for `<first-200>-` (src/path/, `find_dir_by_prefix`); among collisions, disambiguate by the in-record `cwd` (authoritative).
### 3.2 Record model (one JSON object per line)
Top-level fields used: `type`, `uuid`, `parentUuid`, `timestamp` (ISO8601 UTC), `sessionId`, `cwd`, `version`, `gitBranch`, `isSidechain`, `userType`, `message`, + `subtype`/`content` on system records, `isCompactSummary` on compaction summaries. Many more exist and are IGNORED by the default scanning surfaces (`attachment`, `file-history-snapshot`, `queue-operation`, `isMeta`, `sourceToolAssistantUUID`, `slug`, `promptId`, ...; since v0.10.0 a `queue-operation` line with content, the `turn_duration`/`away_summary`/`stop_hook_summary` system records and the `file-history-snapshot`/`-delta` lines are the GATED promoted leaves, parsed only under an explicit selector or a `show` address - §3.3a) - `toolUseResult` is NOT ignored: it is kept UNPARSED (`Box<RawValue>`, section 4 perf) and deep-read by `files`/`recover` (create-vs-edit, content anchors, the section 3.11 freshness signals) — with ONE opt-in exception: a `type:"attachment"` record whose payload is `{"type":"hook_additional_context","content":[...]}` (the context a SessionStart/UserPromptSubmit/... hook injected) classifies `harness.meta.hook` and is scanned by `search --additional-context` (default OFF — a default scan never parses attachment lines; the candidate needle is `&&`-gated like the D7 boundary keep) and rendered by an explicit `show --line`/`--uuid` address flag-free (the refetch law). `Record::hook_additional_context_text` (model.rs) is the one extractor — `content` is a string ARRAY joined with `\n` (bare string tolerated).
`type` values: `user`, `assistant`, `system`, plus metadata-only `last-prompt`/`ai-title`/`agent-name`/`mode`/`permission-mode`/`attachment`/`file-history-snapshot`/`queue-operation` (these often have **no `timestamp`** - skip in time logic, never crash).
**Block types** (`Block` enum, model.rs): `{type:text,text}`, `{type:thinking,thinking,signature?}`, `{type:tool_use,id,name,input}`, `{type:tool_result,tool_use_id,content,is_error?}`, `{type:image,source}`. `tool_result.content` may be a string OR an array of `{type:text,text}`/`{type:image}`. The enum has a `#[serde(other)] Unknown` arm - new/unknown block types parse, never crash.
### 3.3 Turn boundary = `opens_turn` (4 cases), NOT just `is_genuine_user`; classification is a SEPARATE axis
A `type:"user"` record's `message.content` is EITHER a string (genuine text, older format) OR an array of blocks. **CRUCIAL: a "user" record is NOT always a human turn** - `tool_result` blocks ride on `role:user` records too. Real ratio in one session: ~393 genuine vs ~1619 tool_result-carriers. `model::Record::is_genuine_user` (src/model/) returns true iff: `type:"user"` AND `role=="user"` AND `isCompactSummary` falsey AND `isMeta` falsey AND (content is a string OR blocks contain a `text` block and NO `tool_result` block) AND the string is NOT a synthetic marker AND **NOT `is_peer_message`** (the GOLD §1 + FINDING-2 fix - an inbound `<teammate-message>` OR `<agent-message>` is `type:user`/`role:user`/string and matches no synthetic marker, so it used to slip through as the human; 106 peer messages mislabeled in one real session). **FINDING-1:** peer/notification tags are detected ONLY at a SECTION BOUNDARY (`is_section_boundary` - content-start, just after the relay preamble `Another Claude session sent a message:`, or right after a prior section's close tag), so a genuine message that merely QUOTES `<teammate-message>`/`<task-notification>` mid-prose (common in csift's OWN dev sessions) stays `user.message`. A tool_result-carrier is NOT genuine.
**But `is_genuine_user` is NOT the turn delimiter.** The single boundary predicate every surface keys on is `model::Record::opens_turn` (src/model/):
```
opens_turn() = is_genuine_user() OR is_auq_answer_boundary() OR is_plan_rejection_boundary() OR is_peer_message_record()
```
A turn opens on ANY of four cases (using ONLY `is_genuine_user` drops three of them):
1. **genuine human message** (`is_genuine_user`).
2. **answered AskUserQuestion** (`is_auq_answer_boundary`, src/model/) - the user's ANSWER is their message; rides on a non-errored `tool_result` carrier (a cancelled / `is_error:true` / `Cancelled...` / `<tool_use_error>` AUQ is NOT a boundary).
3. **tool-use rejection-with-message** (`is_plan_rejection_boundary`, src/model/) - a rejection carrying a typed `To tell you how to proceed, the user said:\n` tail; a bare rejection (no typed tail) is NOT a boundary.
4. **inbound peer message** (`is_peer_message_record`) - a `<teammate-message>` OR `<agent-message>` (FINDING-2 folds the agent-message peer form in) from another session, at a section boundary: a real delivered message, so it STAYS a turn-opener (segmentation / turn-COUNT stays byte-stable where peers already opened turns - a non-isMeta teammate/agent message; an isMeta `<agent-message>` newly opens one), but `is_genuine_user` excludes it so it never counts as the human. Body render comes from `inbound_comm_preview` (`verbatim`/`list`) / `record_text_sections` (`search`), never the raw XML.
**Opening a turn ≠ the `-t` label.** `opens_turn` segments the transcript; `Record::classify` (the role.class.sub engine, §3.3a) is a SEPARATE axis that assigns the searchable label(s). A record that opens a turn classifies as `user.message` (genuine), `user.answer` / `user.rejection` (the AUQ/rejection duals), `agent.communication.inbox` (a teammate-message), or `harness.notification.<kind>` (a `<task-notification>`) - the opener body still renders via `reconstructed_user_text`, but its `-t` bucket follows `classify`, not `opens_turn`.
**EXCLUDED from opening a turn** (machine pseudo-turns that LOOK human - never delimiters, dropped/folded as turn MEMBERS): `isMeta` records; the two interrupt markers (`[Request interrupted by user]` / `...for tool use]`); `<local-command-stdout>...`; slash-command wrappers — `is_slash_command_wrapper` detects BOTH tag orders, a leading `<command-name>...` OR the current-CC `<command-message>...` (the new `COMMAND_MESSAGE_PREFIX`; verified 14 new-order vs 35 old-order sessions), and `is_genuine_user` now excludes BOTH so a new-order wrapper no longer masquerades as the human or opens a turn (a v0.5 correctness fix — turn NUMBERING may shift on transcripts carrying new-order wrappers); compaction summaries (`isCompactSummary`). **Slash-wrapper classify:** a WITH-args wrapper carries BOTH labels `[user.message, harness.command.invocation]` with `user.message` FIRST (richest-view law), so the UNFILTERED render is the extracted `/name args` (`slash_command_name`), never the wrapper XML; a NO-args wrapper → `harness.command.invocation` ONLY; an explicit `-t harness.command.invocation` still renders the raw wrapper. (SPEC §4.2.3 is the authority.)
**`<task-notification>...` automation pulse** (src/model/) passes every `is_genuine_user` gate so it DOES open a turn, but is a machine trigger, not operator prose - so `classify` buckets it under **`harness.notification.<kind>`** (NOT `user`; the GOLD §1 reparent), `<kind>` = one of FIVE `AutomationKind` slugs `background-command`/`workflow`/`subagent`/`monitor`/`task` (the role-`agent`-colliding `agent` slug renders as the `subagent` leaf). Render it via `Record::automation_label()` -> `[<kind> <task-id> <status>] <summary>` (parsed from the summary, src/model/; status falls back to `<event>` then `completed`) - NEVER dump the raw `<task-id>`/`<output-file>`/`<status>` XML. `monitor` is the Monitor/ScheduleWakeup cadence pulse (from_summary src/model//1676; for it the status slot renders the `<event>`, NOT a fabricated `completed`, src/model/ - directly relevant to monitoring/cron transcripts). A pulse carrying a `<result>` is a background-agent REPORT, so it ALSO carries `agent.communication.inbox` (child ⇨ self, G1). **Standing caution: these section-3 enumerations can LAG the enum - verify the live set against `AutomationKind::slug` (src/model/) before trusting a fixed list.**
**Render + group via the shared helpers, never hand-roll:** `Record::reconstructed_user_text(plan_index)` (src/model/) yields the normalized opener body for ALL three cases (genuine text / full AUQ Q+options+answer unit / rejection text + optional `[plan: <path>]` pointer when `plan_index` resolves the rejected id; `None` = does not open a turn). Delimit with `group_turn_indices_deduped` (src/model/, production - drops esc-cancel/edit-resend superseded-draft openers via shared `parentUuid`) or the bare `group_turn_indices` (test-only bool-fixture base). Both `search` exchange reconstruction and `files` mutation attribution route through these so they never drift. C-18 (v0.8.2) + v0.9.2: a superseded draft stays OUTSIDE turn numbering but is searchable/censusable under its own leaf `user.unsent` (footer discloses the set + names the leaf; JSON summary `superseded_drafts`); an explicitly ADDRESSED draft always fetches via `show --line`/`--uuid` (JSON `superseded_draft:true` + null `turn_index`); a `--turn` window suppresses draft units (they belong to no numbered turn).
### 3.3a The `role.class.sub` classification (model.rs `Role`/`Class`/`classify`; SPEC §5 is the authority)
`Record::classify(ctx) -> Vec<Class>` (src/model/) is the engine `search -t` keys on. It is **multi-label** (one physical record can carry >1 leaf), pure, tolerant (an unmodeled record → empty `Vec`, never a crash). Three ROLES, **34 leaf classes** (`Class::ALL`, src/model/ - the single source of truth; a drift-guard test pins the count):
```
user (the human) user.message · user.answer · user.rejection · user.unsent · user.queued
agent (the assistant) agent.message · agent.thinking · agent.thinking.narration · agent.tool.use · agent.tool.result
communication agent.communication.{inbox, sent, signal}
harness (CC machinery) harness.notification.{workflow, monitor, subagent, background-command, task}
harness.compaction.{summary, boundary}
harness.command.{invocation, stdout}
harness.interrupt.{user, tool}
harness.schedule.{wakeup, continuation}
harness.meta.{hook, loop, attachment}
harness.meta.{turn-duration, away-summary, stop-hooks, snapshot} (v0.10.0 promoted, gated)
harness.meta.system (v0.10.1 catch-all for every other system subtype, gated)
```
- **`-t` selector = a dotted path matched by dot-SEGMENT prefix** (cli.rs `category_selectors`/`label_selected`, derived from `Class::ALL`): `-t agent` ⇒ the whole agent role, `-t agent.tool` ⇒ use+result, `-t agent.communication` ⇒ all three comm leaves, a full leaf path ⇒ just it. NO `-t` ⇒ every label. The **old flat values** `thinking`/`tool`/`tool-response` HARD-error (0 back-compat, like `--full`) — since v0.6.3 the error names the successor path (`agent.thinking`/`agent.tool`/`agent.tool.result`) ahead of the full leaf list; `user`/`agent` keep working as ROLE selectors.
- **Gated meta leaves**: `harness.meta.hook` is scanned only under `search --additional-context` (or the superset `--attachments`); `harness.meta.attachment` (any OTHER attachment payload; matchable text = the VERBATIM payload JSON, a byte substring of the raw line so the prefilter laws hold with no synth markers) only under `--attachments` / `--count-by attachment` - a default scan never parses attachment lines; an explicit `show --line`/`--uuid` address renders any of them flag-free (the refetch law). Both keeps are `&&`-gated candidate needles like the D7 boundary keep (`needs_hook_context` / `needs_attachments`, scan.rs).
- **Gated PROMOTED leaves (v0.10.0; `harness.meta.system` added v0.10.1 as the catch-all for every OTHER `type:"system"` subtype - `informational`, `api_error`, `model_refusal_*`, `agents_killed`, `local_command`, `scheduled_task_fire`, future ones - rendered `[<subtype> <level>] <content>`, admitted by the key-only `"subtype"` needle, the compaction boundary excluded)**: `user.queued` + `harness.meta.{turn-duration, away-summary, stop-hooks, snapshot, system}` are the former non-record line types (`queue-operation` with content, `system`/`turn_duration`/`away_summary`/`stop_hook_summary`, `file-history-snapshot`/`-delta`), classified by `Record::promoted_class` (src/model/classify_promoted.rs - one leaf each, `Class::llm_visible` false for all five: NONE carries a `message{}` field, the boundary's instrument; parentUuid threading and `preservedMessages` membership are NOT instruments). They are admitted by the prefilter ONLY when `SearchArgs::reaches_gated(leaf)` holds (an EXPLICIT `-t` reaching the leaf: full path, glob, or the `harness.meta` prefix) or a `show` address is present - a bare scan, bare role, `-T`-only filter and flagless `--count-by label` never parse them (this is STRICTER than the boundary keep, which a match-all scan admits). The gates live in `scan::CandidateGates` (one `&&`-gated memmem per leaf: `"queue-operation"`, `turn_duration`, `away_summary`, `stop_hook_summary`, `"file-history-`). Render text = `record_text::promoted_record_text` (verbatim `content` for queued/away-summary; fabricated key=value excerpts for turn-duration/stop-hooks/snapshot - each fabricated form registers its type value as a VERIFIABLE synth marker in `synth_marker_finders`, gated on the same predicate, so the §7f whole-file gate stays sound). A queued hit carries `Hit.queue_operation`/`queue_reason` (label zone `[enqueue]` / `[remove · reason]`, JSON `queue_operation`/`queue_reason`); a queued `<task-notification>` / peer message (a harness RIDER, 87% of content-bearing queue lines) carries NO label - the same content-shape law that reparents those user records. `dispatched` is never asserted (no join key on the queue line). The zero-match diagnosis adds a stderr note + JSON `gated_leaves_unreached` when no selector reached a gated leaf. Deliberately NOT promoted: the session-state cache lines (`last-prompt`/`mode`/`ai-title`/`agent-name`/`permission-mode` - no uuid/timestamp/message, `lastPrompt` duplicates `user.message`), the unpromoted system subtypes and the other bookkeeping types (SPEC v0.10.0 ledger ④ lists them with counts).
- **Multi-label + Q4 dedup**: a record selected by ≥2 of its labels emits ONCE, richest view (AUQ answer → `user.answer` not `agent.tool.result`; a `SendMessage`/spawn → the `agent.communication.*` view not `agent.tool.use`; a `<result>` pulse / subagent return → the comm view). JSON carries the matched leaf as `label` + the full set as `labels[]`.
- **Direction (`from ⇨ to`)** on every `agent.communication.*` hit (`Record::direction`, src/model/): teammate-message `teammate_id ⇨ self`; `SendMessage` `self ⇨ input.to`; spawn `self ⇨ child`; subagent opener `parent ⇨ self`; subagent/`<result>` return `child ⇨ self`. The owner's own id renders as the literal `self`.
- **Render markers**: a ROLE glyph leads each hit line - **`◂` user · `▸` agent · `⚙` harness** (gear = machinery); `⇨` (hollow arrow) = the comm direction; `▹` = ONLY the `agent.tool.use ▹ agent.tool.result` pairing (joined by `tool_use_id`; an unreturned use → `(no result — pending)`, an orphan result → `(use not in scope)`).
- **Notable mappings**: a SUPERSEDED turn-opener draft (esc-recall + edit-resend, shared parentUuid — the C-18 set) → `user.unsent` (v0.9.2: assigned at the SCAN layer, where `superseded_draft_indices` sees the LATER sibling a pure per-record classify cannot; SINGLE label so `user.message` counts stay pure; searchable + censusable, still OUTSIDE turn numbering, suppressed under a `--turn` window, always reachable by address; a recalled-then-abandoned message has no sibling = undetectable; a queued text edited before dispatch never becomes a user record — it survives only in `queue-operation` non-record lines); a narration-TAGGED thinking block → `agent.thinking.narration` (v0.9.2: the tag hides in the base64 `signature`, protobuf path 2→1→8 LAST-field, decoded by `model/narration.rs` — SIGNATURE ONLY, never adjacency; renders the `[narration summary]` LABEL-zone marker, display-only because the tag word is base64-hidden in raw bytes so matchable marker text would violate the §7 prefilter laws; sibling cap 1; excluded from `verbatim` by construction — `agent_text` collects only text blocks; `agent.thinking` is the first leaf that PREFIXES another leaf, handled by segment-prefix selection); `redacted_thinking` block → `agent.thinking` (opaque, renders `[redacted thinking]`); a pending elicitation sidecar marker (AUQ/ExitPlanMode/MCP) → `agent.tool.use`; an `isMeta` record that matches no harness marker is **EXCLUDED** (empty - never `user.message`); a `<task-notification>` → `harness.notification.<kind>` (§3.3); the harness's plain-text agents-stopped notice (`is_agents_stopped_notice`, markers.rs: `N background agents were stopped by the user: …` / `Background agent "…" was stopped by the user.`) → `harness.notification.subagent` (v0.10.0: a synthetic marker, so NEVER genuine and never a turn opener; renders `[subagent stopped] …` via `automation_label`, its fabricated head covered by the `stopped by the user` synth marker AND by `synth_texts_match` re-rendering `automation_label`); a `Background command "…"` pulse → always `.background-command` (v0.10.0 retired the quoted-name monitor heuristic; `.monitor` = summary-prefix `Monitor`/`Scheduled`/`cron` only); a `queue-operation` line with the human's content → `user.queued`, `system`/`turn_duration` → `harness.meta.turn-duration`, `away_summary` → `harness.meta.away-summary`, `stop_hook_summary` → `harness.meta.stop-hooks`, `file-history-snapshot`/`-delta` → `harness.meta.snapshot` (v0.10.0, all gated - the bullet above).
- **`search`-surfacing note**: `classify` runs on every record. `search`'s §7 stage-1 prefilter (`line_is_transcript_candidate`) keeps `role:user`/`role:assistant` lines AND — since D7 — the rare `compact_boundary` `type:"system"` record — but the boundary keep is itself **`-t`-GATED**: a `needs_compact_boundary` bool (`label_selected(&args.categories, "harness.compaction.boundary")`) is derived ONCE before the scan, and the extra `memmem` is `&&`-gated behind it, so a `-t user`/`-t agent.*`/any non-boundary query pays ZERO (the `memmem` is never reached); only no-`-t` (match-all) or a selector reaching `harness.compaction.boundary` (`harness` / `harness.compaction` / `harness.compaction.boundary`) keeps it. The role keeps ALWAYS run (a conservative superset). **R13 needle law:** the role keeps are the SHARED serialization-tolerant matchers `parse::line_has_role_marker` / `line_has_user_role_marker` (JSON whitespace around the colon is the SAME record — the old exact compact bytes `"role":"user"` silently dropped a reserialized line, e.g. python `json.dumps` defaults, one layer BEFORE any malformed counter could see it: no preview, no count, no match, zero disclosure; found independently by R12's own fixture accident and R13's witness). Every OTHER prefilter needle in the tree is serialization-safe by construction (a bare value substring like `tool_use`/`plan_mode`/`compact_boundary`, or a key-only `"media_type"`/`"filePath"` form). When adding a NEW candidate needle: value-substring or key-only, or route it through a tolerant matcher — NEVER a raw compact `"key":"value"` byte pair for candidate SELECTION (a compact pair is fine for a redundant belt where a tolerant hook already covers the record). When it IS run, the `||` chain reaches the `memmem` only on lines that already failed both role checks, and boundaries are rare — so the §7 perf contract holds. So `search -t harness.compaction.boundary` IS surfaced: for a message-less system record `record_raw_text` falls back to the boundary's top-level `content` PLUS a readable `compactMetadata` excerpt (`[compaction boundary: trigger=… preTokens=… postTokens=… durationMs=…]`) as the match + excerpt, so compaction points are enumerable + inspectable. Its sibling `harness.compaction.summary` (a `type:"user"` record) is searchable too — so every leaf is now search-reachable (the boundary was the last exception).
### 3.4 AskUserQuestion
A `tool_use` block with `name=="AskUserQuestion"`. **HARD-WON, RE-MEASURED at CC 2.1.258 (v0.10.1): a PENDING single-question AskUserQuestion is NEVER flushed to jsonl** - the whole assistant turn stays buffered until it is answered or cancelled, so a one-question option-picker freeze looks identical to a stall on disk; but a **MULTI-question ask (`input.questions` length >= 2) IS written at question time** as an unreturned `tool_use` (measured live on macOS and Windows: the record sits on disk while the dialog is up, the single-question ask does not). `live::tail` therefore reads an unreturned `AskUserQuestion`/`ExitPlanMode` at the main tail as `waiting-hitl`; the elicitation sidecar (§3.10) stays the instrument for the single-question shape. The registry row's `status:"waiting"` (§3.9) covers both plus permission prompts. The user's answer to an AskUserQuestion classifies as **`user.answer`** (dual-labeled with `agent.tool.result` since it rides on the answering tool_result carrier; deduped to the richer `user.answer`, §3.3a). **The answer's structured signal is EITHER of two `toolUseResult` fields (v0.10.3):** a non-empty `answers` map (the TUI dialog: a listed label, a typed "Other" text, or the `(notes only)` sentinel plus `annotations.notes`), OR a non-blank freeform `response` string (the binary's `The user responded: <text>` branch, written by an answerer other than the TUI dialog and paired with an EMPTY `answers`); `is_auq_answer_boundary` accepts both, and `auq_exchange` renders the questions plus a `response:` line for the second. The synthesized text prefixes (`markers::AUQ_ANSWER_MARKERS`) are the fallback for carriers without `toolUseResult`; the idle-timeout branch (`afkTimeoutMs` set, text `No response after ...`) is NOT an answer and never opens a turn.
### 3.5 Compaction shape
The summary is a `type:"user"` record with `isCompactSummary:true` + `isVisibleInTranscriptOnly:true` carrying **string** content (NOT a `type:"summary"` record). A separate `type:"system"` `subtype:"compact_boundary"` record carries `compactMetadata:{trigger,preTokens,postTokens,durationMs}` PLUS a top-level `logicalParentUuid` (the true predecessor record the compaction re-links to; the boundary's own `parentUuid` is null) - surfaced in the boundary's rendered excerpt as `[logicalParent=<uuid>]`. A compaction summary MUST be excluded from "genuine user". `verbatim` reconstructs the verbatim exchange a summary clipped. **Clone law (v0.9.4, `session::clone_head_boundary`):** a transcript whose FIRST TIMESTAMPED record is a `compact_boundary` was minted by COPYING another session at that compaction (background-job forks preserve uuids, strip the slug, carry timestamps predating the file — never trust file birthtimes); `list` detects it and joins the origin (`clone_of`), and a clone DOUBLE-COUNTS its inherited records on every spanning surface until scoped away. The PLAN corollary: a clone has no `plan_mode` attachment, so `plan` falls back to the first-VALID-slug law (CC's own binding rule; `binding_source:"slug-only"`, `minted_at_compaction` when the first carrier is the boundary).
### 3.6 Externalised output
Large tool outputs are moved to a sibling `tool-results/<id>.txt`, leaving an inline `<persisted-output>` pointer (absolute path + preview). `search --resolve-persisted` replaces the pointer with full file content BEFORE matching: prefers structured `toolUseResult.persistedOutputPath` (src/model/), falls back to scraping `Full output saved to: <ABS_PATH>`. Read failure is non-fatal - keeps inline + appends `[csift: could not resolve persisted output ...]`.
### 3.7 Subagent on-disk shapes, flat nesting, topology
Three on-disk shapes: (A) built-in Task/Agent under `subagents/agent-<hex>.jsonl`; (B) workflow/OMC under `subagents/workflows/wf_*/agent-<hex>.jsonl`; (C) `journal.jsonl` event log = NOT a transcript (read only for completion status, never listed/searched). **Kind = on-disk location, NOT `agentType`** — `BuiltinTask` vs `Workflow` (subagent.rs) — **with ONE meta-driven exception: `Teammate`.** Canonical agent id = bare `<hex>` (= the record/journal `agentId`) for A/B. The parent uuid is the dir segment above `subagents/`.
**Teammate subagents (the `in_process_teammate` kind, the new Claude Code "teammates"/FleetView feature).** A teammate lands at the BUILT-IN location (`subagents/agent-<id>.jsonl`) so location alone can't classify it — the discriminator is its `meta.json` `taskKind:"in_process_teammate"` (`make_subagent` upgrades `BuiltinTask` -> `Teammate` when it sees that). Distinctive shape, all verified against real data: (1) the canonical id EMBEDS the teammate name — `aVSRepro-68a2a1661c9390c1` = `a<Name>-<16hex>`, NOT a bare hex (so `is_bare_subagent_hex` rejects it; `is_teammate_agent_id`/`is_subagent_id` accept it — see §2); (2) the meta carries `{name, taskKind, teamName, color, model, permissionMode, spawnDepth}` and the `agentType` is OVERLOADED with the teammate NAME (e.g. `"VSRepro"`), NOT the real subagent type; (3) the meta has **NO `toolUseId`** — the usual id-join to the spawning tool_use can't reach it; (4) it is spawned by an `Agent` tool_use whose `input.name` == the teammate name AND `input.subagent_type` == the REAL type (`oh-my-claudecode:qa-tester`); (5) the team lead talks to it via `<teammate-message teammate_id="team-lead">` blocks + the `SendMessage` tool (bidirectional, persistent — unlike a one-shot Task subagent). **Spawn linkage = a NAME-join, not an id-join** (`ParentSpawnIndex::by_name` / `spawn_id_for_name`, joined in `node_for`): the teammate's `meta.name` -> the `Agent` spawn's `input.name` recovers the spawn `tool_use_id`, the TRUE trigger ts, the parent agent, and the real `subagent_type` (which `node_for` then prefers over the meta's overloaded `agentType` ONLY for a teammate). The node also carries `name` + `team_name`. A teammate is a normal topology PARENT — children it spawns nest under it by the usual issuer link. `agents --shape teammate` filters to them. **Control hint (NOT a csift op — csift is read-only):** `agents` surfaces a pointer to the CORRECT control tool because a real session burned ~30 min trying to `TaskStop`/`pkill` a runaway teammate — feeding it the name, the `Name@team` form, AND the exact `aName-<hash>` agentId csift prints, all rejected (`No task found with ID`). The ids were CORRECT; the TOOL was wrong. A teammate is steered/terminated via `SendMessage` addressed BY NAME (`message:{type:"shutdown_request"}` terminates — verified from the live SendMessage schema), never `TaskStop` (it only resolves `run_in_background` `task_id`s) or `pkill` (in-process, shares the orchestrator PID). Surfaced as a text footer when ≥1 teammate is in scope (`agents.rs` `TEAMMATE_CONTROL_HINT_*` / `any_teammate`) and as each teammate node's JSON `control_hint`. Keep the wording FACT-led (in-process Agent subagent, addressed by name) so it survives a tool rename.
**Workflow RUN `status` is journal-verbatim (open set).** An `agents` `kind:"run"` row's `status` is read straight off the workflow `journal.jsonl` (`str_f("status")`, subagent.rs) — observed values include `completed` and `killed`; it is NOT a csift enum and is distinct from the per-agent csift-computed lifecycle status. Never tighten it into a closed set.
**Flat on disk, nesting reconstructed:** all subagent transcripts sit flat under one session; depth>1 nesting (an agent spawning an agent) is NOT encoded in the path. `build_topology(session_jsonl, with_files)` (src/subagent/) rebuilds the tree: build a GLOBAL spawn index (`build_global_spawn_index`, src/subagent/) by forward-scanning the main transcript PLUS every subagent transcript for `Task`/`Agent`/`Workflow` `tool_use` spawn records, keyed by `spawn_tool_use_id`; each child's `spawn_tool_use_id` (from its head record / meta) joins to its issuer -> `parent_agent_id`; `assign_depths` walks the id->parent chain (cycle-guarded). `SubagentNode` carries `agent_id`, `kind`, `parent_session_id`, `parent_agent_id`, `spawn_tool_use_id`, `spawn_tool`, `workflow_id`, `agent_type`, `description`, `trigger_utc` (true spawn = parent tool_use ts; falls back to child-head `started_utc`), `completed_utc`, `returned_message` (+`_source`), `status`, `files_changed`, `depth`, `children`. **Subagent env quirk:** a workflow `agent()` subagent's `$CLAUDE_CODE_SESSION_ID` holds the PARENT session uuid, not its own - and current CC hands an Agent-tool subagent the PARENT uuid too (verified live 2026-07-12; the subagent's own id is withheld from its env; older builds handed a built-in Task subagent its own id) - never assume env == self; `@trap` is the env-independent self-resolution.
**r5 id shape - EVERY emitted row carries `is_subagent` + `parent_session_id` (REQUIRED, not optional).** A subagent transcript's `session_id` is a bare `<hex>` - it is NOT a re-feedable `@<uuid>` target (`csift verbatim <hex>` fails). So `list`/`search`/`files`/`verbatim`/`recover` rows are a TRIO, not a lone id: `session_id` (display) + `is_subagent` (id-domain discriminator) + `parent_session_id` (the always-re-feedable owning uuid; == `session_id` for a top-level row). Since v0.6.5 every search HIT (and sibling) object inside an exchange row carries the trio too — bare `.hits[]` flattening keeps real ids (jq cannot fail loud on a missing key, so the data matches the natural access pattern). Emitting only the hex = unusable rows. **Derive all three with `crate::subagent` helpers - NEVER read the path stem yourself** (the stem is `agent-<hex>` -> reading it gives the wrong `agent-`-prefixed id): `session_id_from_path` (src/subagent/ - strips the `agent-` prefix to the bare-hex canonical `agentId`), `is_subagent_path` (src/subagent/ - true iff a `subagents/` path component), `parent_session_id_from_path` (src/subagent/ - the dir component just before `subagents/`; `None` for a top-level file, so callers `.unwrap_or_else(|| session_id.clone())`). To re-feed a subagent match downstream, use `parent_session_id`, never the bare hex. **envelope v2 (SPEC §8.2) is built ONLY through `text::envelope_header`/`envelope_scope_header`/`envelope_summary`** — every `--format json` stream is exactly ONE `{"kind":"header","command":…}` line + kind-tagged rows + ONE `{"kind":"summary"}` line; `kind` belongs to the envelope exclusively (a transcript's shape is `shape`, a boundary's classifier is `cause`), and the jsonl-line key is `line` everywhere. The row TRIO keys themselves are still hand-built in each module's `json!` projector — when adding a spanning JSON surface, emit the envelope through the text.rs builders and replicate all three trio keys, deriving values via the `crate::subagent` helpers above.
### 3.8 whoami detection
CC exports `$CLAUDE_CODE_SESSION_ID` into the Bash tool env == the session's own jsonl basename. Definitive, per-session, version-independent, zero false positives. Use it and nothing else. When absent/empty, **DO NOT GUESS** (concurrent sessions; most-recent-mtime is a false-positive trap) - error with guidance to pass `@<uuid>`. whoami saying "ambiguous" is acceptable. LANE HONESTY (v0.8.2): the env names the TOP-LEVEL session in EVERY lane, so the env form's `is_subagent`/`parent_session_id`/`depth` are NULL (unknowable, never fabricated), text carries a `lane unknown` line, stderr names the resolution path; every `@main` resolution prints an unconditional stderr lane note (csift cannot know the caller's lane). (`CODEX_COMPANION_SESSION_ID` mirrors it but is Codex-specific.)
### 3.9 Pending tool_use / "frozen lane" (a permission ESCALATION leaves NO jsonl trace)
A pending tool-approval lives ONLY in CC process memory - it is in NEITHER transcript NOR any per-lane control file. The ONE surface that shows it is the top-level session's registry row (`<claude-home>/sessions/<pid>.json`, v0.10.1, binary 2.1.258): `status` is the closed set `busy | shell | idle | waiting`, computed as `waiting` (with an unpersisted `waitingFor` kind) whenever a dialog blocks the session - a queued elicitation, a permission prompt, a plan approval, a sandbox or worker request - else `busy` while loading or delegating, else `idle`, which is relabeled **`shell`** while a background shell task is open (so `shell` = idle + a dev server, NEVER a running shape). The row is transition-written, so a stale `waiting` from a dead session is refuted only by the pid probe (`live::registry::probe_pid`: `ps` + `/proc` on unix; PowerShell `Get-Process` + `tasklist` on Windows, where `procStart` is a FILETIME tick count instead of asctime; a row whose `pidDomain` - `darwin` / `linux` / `win32:<host>` - is not ours is never probed). Per LANE, on disk a blocked lane looks like: the assistant's `tool_use` record (`stop_reason:"tool_use"`, the command in `message.content[].input.command`) is the LAST record, FROZEN, with NO following `tool_result` for its `tool_use_id` (the block carries only `{type,id,name,input,caller}` - no `permission`/`isEscalated`/`requires_approval`); minutes later the `tool_result` is appended once a human answers. So THREE states share ONE on-disk signature: ① escalation-blocked (waiting for a human to approve) ② awaiting-execution (a slow tool) ③ wedged/dead. **Worse, the OLD status logic mis-reported a frozen lane as `completed`** - the tail walk-back for an end-of-turn text found the assistant text that PRECEDES the frozen `tool_use` (the L629->L630 shape) and called it a clean finish. `lifecycle` now detects the frozen lane from the NEWEST meaningful record (an unreturned `tool_use` ⇒ blocked, NEVER `completed` - status forced `Running`) and classifies it: a Bash command CC's `dangerous-rm` classifier ([`bash_danger`]) would HOIST even under bypass ⇒ **escalation-blocked** (the one state jsonl can POSITIVELY confirm); otherwise **awaiting-execution** (slow OR wedged - jsonl can't tell these apart, so don't pretend; the caller weighs elapsed-since-`pending_since_utc`). Surfaced on `agents` nodes as `pending_tool_use_id`/`pending_tool_name`/`pending_classification`/`pending_since_utc`. On EVERY agent node `completed_utc/_local` (+`duration`) are non-null ONLY when `status==completed` (a frozen lane is never "done" — the JSON now matches the text tree's suppression), and `last_activity_utc/_local` carries the tail newest-record instant regardless of status (== `pending_since_*` when frozen); `--order-by completion` windows on that terminal instant, so frozen lanes window on their freeze instant. `bash_danger` is a 1:1 port of CC's `Ywa`/`egp`/`Zhp` (the dangerous-rm path is pure static regex - no LLM, no fs); it is PURELY LEXICAL (flags `rm $VAR/{*,$,/,",',end}` WITHOUT checking if `$VAR` is empty - mirror CC exactly, never "improve" it). Higher-fidelity live signal would need a pre-installed CC `Notification`/`PreToolUse` hook writing a sidecar marker; the heuristic is the zero-setup retroactive path.
**Windows shell reality (extracted from the CC 2.1.228 binary, v0.7.4):** on Windows CC ships a SEPARATE first-class tool literally named **`PowerShell`** (tool registry `["Bash","BashOutput","KillShell","PowerShell",…]`; same `input.command` field; "Executes a given PowerShell command with optional timeout…"). Selection: `CLAUDE_CODE_USE_POWERSHELL_TOOL` explicit override → else PowerShell is FORCED ON when no Git-for-Windows bash is found (detection: `CLAUDE_CODE_GIT_BASH_PATH` override → `C:\Program Files\Git\bin\bash.exe` → the `(x86)` sibling) → else a feature gate decides. The Windows `Bash` tool runs the REAL Git-for-Windows (MSYS2) bash — POSIX syntax, so csift's lexical layers stay valid on those records. csift handling: `@trap` matches BOTH shell tools; the bash-LEXICAL layers (bash_danger escalation, bash_mutations attribution in files/recover) deliberately do NOT run on `PowerShell` records — a pending PowerShell lane classifies `awaiting-execution`, and PS shell-side file mutations are invisible to the heuristic (structured Read/Write/Edit attribution is unaffected); `--count-by tool` reports `PowerShell` as its own key. ALSO NOTED (bash_danger.rs header): 2.1.228's dangerous-rm has EVOLVED past the ported 2.1.x generation (fixpoint `$(…)` stripping; a tree-sitter pass bails to explicit approval at >64 command substitutions) — a port refresh is a recorded follow-up. **MEASURED LIVE on a Windows 11 ARM64 guest at CC 2.1.258 (v0.10.1):** with an msys2 `bash.exe` merely on PATH (no Git-for-Windows install) the session offered BOTH tools at once - `Bash` (POSIX paths like `/c/work/todo`) and `PowerShell` - so the "forced when no bash" rule above is the bashless case only; `PowerShell` accepts `run_in_background:true` and returns the SAME result grammar as Bash (`Command running in background with ID: <id>. Output is being written to: <path>`), so `live::background_scan` ingests both by name; task output files live under `%LOCALAPPDATA%\Temp\claude\<encoded-project>\<session>\tasks\<id>.output` (macOS: `/private/tmp/claude-<uid>/<encoded-project>/<session>/tasks/`), and an async agent's `outputFile` there stats 0 bytes on Windows (no symlink to the child transcript); records carry `cwd` as `C:\work\todo` (backslashes) and `gitBranch:"HEAD"`; the Windows line-type census adds `atis-latch`, `bridge-session` and `file-history-delta` beside the usual set.
### 3.10 Elicitation sidecar - the TRANSPARENT merge (AskUserQuestion / ExitPlanMode / MCP)
Three elicitations BLOCK a session on a human yet leave NO usable live trace in the native jsonl: **AskUserQuestion** + **ExitPlanMode** (CC buffers the WHOLE assistant turn until answered - the tool_use is committed to the persisted message array only AFTER the interactive tool resolves; nothing on disk during pending - see §3.4) and **MCP Elicitation** (the inner `elicitation/create` request is an in-memory MCP-client callback, never a transcript record). A CC hook (SKILL.md recipe) records each to an append-only **SIDECAR** `<session-sidecar-dir>/elicitations.jsonl` (the `<uuid>/` dir beside `subagents/`, via `crate::subagent::sidecar_dir_for_session` / `crate::elicitation::sidecar_path`) - NEVER the native transcript (no pollution). Each line carries `csift:"elicitation-marker-v1"` + `csiftPhase`∈{`pending`,`resolved`} + `csiftKind` + `csiftKey`; a `pending` line is shaped like the NATIVE record CC will eventually write (an AUQ/ExitPlanMode `tool_use` on a `type:"assistant"` record; an MCP `type:"system"` record), a `resolved` line is a lightweight `type:"csift-elicitation-resolved"` close marker. **Keyed by the TOP-LEVEL session** (the hook's `session_id` is always the top-level/leader uuid, never a subagent's). The tolerant `csift*` fields parse onto `model::Record` (`is_elicitation_marker()` gates them).
**The merge (NO subcommand - this REPLACED the former `pending` subcommand).** `crate::elicitation::unresolved_pending(session)` groups the sidecar by `csiftKey`; a key with a `pending` and NO `resolved` is UNRESOLVED → its pending record is emitted (exactly the one MISSING from the native transcript - once resolved CC wrote the real record, so the pending is paired off and DROPPED ⇒ no duplicates, the auto-dedup is the whole point). **GHOST-PENDING guard (v0.6.3):** CC fires NO PostToolUse for a REJECTED AUQ/ExitPlanMode, so the hook can never write `resolved` on that path — `unresolved_pending` therefore ALSO drops an AUQ/EPM pending whose `csiftKey` appears on a native record as an actual `tool_use` block id / `tool_result` `tool_use_id` (`native_closes`, elicitation.rs — STRUCTURAL check, a key quoted in prose does not count; cost paid only when ≥1 such key is sidecar-unresolved). The native transcript outranks the sidecar; MCP markers (no native form, key may be a server NAME) are exempt and stay sidecar-paired only. **search / show / verbatim / list** read the sidecar when they read a TOP-LEVEL session (subagent transcripts have none — `show` merges too, via `fetch_records`; a merged record is addressable by `--uuid`) and merge the unresolved-pending records as if native: `classify` labels a pending marker **`agent.tool.use`**, so `search` matches all three kinds under `-t agent.tool.use` (an AUQ/ExitPlanMode `tool_use` via the `Block::ToolUse` arm; the MCP `system` record - which has no `tool_use` block - via a guarded §3.10 arm in `collect_record_hits` that matches its top-level `content` string and tags it `agent.tool.use` named by `csiftKind`, so `search MCPPROBE` and `-t agent.tool.use` both find it; that arm is gated to a marker with NO tool_use block so AUQ/ExitPlanMode never double-emit), `verbatim` appends each as its own pending turn unit (ranked most-recent), `list` annotates the row. A merged record has NO physical line - it renders `(elicitation sidecar)` in place of `Lnnnn` (never a fabricated L), carries JSON `source:"elicitation-sidecar"` + null `line`/`line_no`, and every surface that merged ≥1 record prints the EXACT note **`with elicitation sidecar`** (JSON `with_elicitation_sidecar:true`). Malformed sidecar lines are skipped + COUNTED (folded into the surface's `skipped_lines`, never silent) — and so is a SENTINEL-bearing marker whose `csiftPhase` the current schema cannot read (schema skew: a pre-release fossil under old field names `phase`/`kind`/`key` — provably ours, uninterpretable ⇒ counted, never merged, never invisible; R12, the `_` arm of `pair_unresolved`); a missing sidecar dir/file ⇒ no merge (not an error); near-free when nothing is pending (typically 0 records). **files / recover** carry no file ops for these records, so the merge would produce no output there - they deliberately do NOT read the sidecar (a no-op for uniformity). **Targeting rejection:** `resolve_session_files`' `*.jsonl` branch `bail!`s when the target `is_sidecar_path` (basename `elicitations.jsonl` OR a content-sniff: every line is a `csift`-marked record) - the sidecar is read automatically, never searched directly.
### 3.11 The Bash shell cwd + file freshness (verified against CC 2.1.237; SPEC 4.9 is the authority)
**cwd:** CC spawns a FRESH shell per Bash call at a TRACKED cwd and stamps that value on EVERY record as the top-level `cwd` field (subagents included, compaction-stable). The tracker advances via a `&& pwd -P >|`-chained read-back (non-zero exit / backgrounded commands never advance it); a cd into a project SUBDIR persists silently across calls; a reset fires only outside originalCwd + the `/add-dir` set ("Shell cwd was reset to <path>" in the result). **ANCHOR LAW: read `cwd` off the TOOL_USE record, never the result record** (~0.24% async lag). So bash-operand resolution needs NO cross-command state: `bash_mutations::cwd` joins each operand against the record's own cwd through the in-command cd checkpoints, with an explicit `Resolution` class per row (`absolute`/`cwd-joined`/`cd-tracked`/`unresolved` - verbatim, never guessed). Class-marker pseudo-paths (`git:`/`fmt:`/`interp:`/`pkg:`/`extract:`, `is_class_marker`) are flags, never resolved or path-matched.
**Freshness:** Write/Edit gate on mtime vs the last Read with a content-hash absolution for FULL reads only; bash never invalidates readFileState (bash `cat` SEEDS it); rejections always land `is_error:true`. Three stream signals are consumed by `recover` (v0.8.0): `toolUseResult.staleReadFileStateHint` on Bash results (CC names the modified files, paths relative to the shell cwd -> resolved via the CARRYING record's cwd -> hard `hint_modified` boundary; `parse_stale_read_hint`, recover/carriers.rs), `toolUseResult.staleRecovered:true` on successful Edits (-> `stale_recovered` annotation), and the `edited_text_file` attachment (16KB budget; an EMPTY snippet = the over-budget degraded form). A change producing NONE of these leaves no transcript trace - absence of signal is never evidence of absence, hence recover's per-window opaque accounting (`ScanResult::opaque`, the M/K/P disclosure + suggested-search).
---
## 4. Invariants + design decisions (with rationale)
- **No `unwrap`/`expect` in library/hot paths.** Propagate via `anyhow::Result` + `?`. Tests may `unwrap`. Only `main.rs` turns an error into an exit code. Rationale: a panic in a transcript-recovery tool destroys the recovery; surface the full chain instead.
- **No silent truncation / no silent failure.** Any cap (`--max-count`) MUST report the drop count; a skipped malformed line is counted, never hidden — since v0.6.6 this includes NON-candidate lines: every byte-prefilter rejection path runs the O(1) `parse::line_shape_malformed` shape check (non-blank but not `{…}`-framed ⇒ counted; catches free-text garbage + crash-truncation at zero §7 cost). Residue: a `{…}`-framed invalid INTERIOR is only counted on a parse candidate - EXCEPT on `stats` (v0.8.1): its non-candidate type probe (`line_type_probe`, stats.rs) fully validates every line for the line-type census, so the named census authority's count is exact. **SCOPE (R12): whole-file census vs window census.** The count is a WHOLE-FILE census only on the full-scan commands (`search`/`stats`/`show`/`files`/`recover`/`verbatim`/`image`); the head/tail readers — `list` rows and `agents` lifecycle — census the LINES THEY READ: `parse::head_records*` returns `(skipped, consumed_end)` and the paired tail scan FLOORS at that offset, so the two windows are DISJOINT and every malformed line in them is booked exactly ONCE (pre-v0.6.8 an all-garbage file reported exactly 2× — both scans, finding no anchor, walked the whole file and double-booked every line; the tail still walks below the floor for missing anchors, it just never re-counts). A mid-file tear is OUTSIDE those windows BY DESIGN (§7 — full coverage measured ~4× the unscoped `list` runtime), so `list`/`agents` `skipped_lines: 0` is NOT a whole-file verdict: the text note scope-qualifies (`among the head/tail lines read — full census: csift stats`) and `stats` is the named census authority. When pairing a head and a tail scan over ONE file, ALWAYS pass the head's `consumed_end` as the tail `floor` — summing two unfloored scans reopens the R12 double-book. When adding a scan with a byte prefilter, route the non-candidate arm through `parse::non_candidate_verdict` — a bare `LineVerdict::Ignore` there reopens the R10 hole. `resolve_session_files` bails when a pinned id matches nothing rather than returning empty. Rationale: a quietly-shortened search result is a correctness bug for an LLM consumer that trusts completeness. **Context-safety output caps.** `list` carries a DEFAULT cap of `session::DEFAULT_LIST_CAP = 50` rows when listing ALL projects (no target / no `--sessions-from`) — the flood guard against a ~69k-line all-projects dump; a scoped query is UNCAPPED, and `--max-count N` overrides. `stats` gained an opt-in `--max-count N` (default unlimited). Both keep the MOST-RECENTLY-active rows (`session::last_activity`) and report the drop (text footer + JSON summary `dropped_by_cap`), never silent — and `list`'s scope banner / JSON header `sessions_in_scope` are computed PRE-cap (v0.6.3: the row flood-guard never shrinks the scope numbers; only the rows are capped). `show` carries a DEFAULT cap of `show::DEFAULT_SHOW_CAP = 200` record units (open ranges like `--turn ..` address a whole transcript) — keep-FIRST; the drop reports the EXACT continuation command (`+N more record unit(s) beyond the 200-unit cap · continue: csift show @<id> --line A..B (or --max-count 0 = uncapped)`; JSON `dropped_by_cap`/`refetch_remainder`). **`--max-count 0` = UNCAPPED uniformly** on `list`/`stats`/`search`/`show` (a bare `Some(0)` used to be an absurd literal zero-cap). `files` (a compact `--by summary` rollup) and `agents` (a tree scoped to its target) were left scope-bounded, so neither took a default cap.
- **Tolerant parsing.** Real jsonl carries far more fields than documented and some records lack `timestamp`. Deserialize only what's used, ignore the rest, never crash on a new field/block type (the `#[serde(other)] Unknown` arm). EXTEND the model tolerantly when CC evolves; never tighten it into a crash.
- **JSON output = the `serde_json::json!` macro ONLY, never `#[derive(Serialize)]`.** There is ZERO `Serialize` derive in `src/` (verify: `grep -rn "derive(Serialize" src/` is empty). A fresh model's instinct - "add `#[derive(Serialize)]` to the row struct" - is WRONG and breaks the house style. Every JSON line is hand-built with `json!({...})` in a per-module `*_json` projector (`hit_json`/`preview_json`/`node_json`/...) then `serde_json::to_string(&value)?` + `println!`. **One JSON object per line** (JSONL, not a pretty array). Render structs stay plain (`#[derive(Debug, Clone)]`); the projector maps struct -> `json!` explicitly, so the wire shape is decoupled from field layout. **Envelope (search + verbatim):** the FIRST line is ALWAYS the shared `text::envelope_scope_header(command, top, sub)` record `{kind:"header", command, sessions_in_scope, top_level_sessions, subagent_sessions}` (text.rs) — emitted UNCONDITIONALLY, NOT gated on `sub > 0`; `list`/`files`/`recover` emit it too. `search` ALSO prints a TRAILING summary object `{matched, sessions, transcript_ids, transcript_ids_truncated, dropped_by_cap, skipped_lines, with_elicitation_sidecar, excerpts_truncated}` as the last line (search.rs `render_json`). `transcript_ids` (+ `transcript_ids_truncated`, first-100 cap) = the distinct MATCHING-TRANSCRIPT ids — DELIBERATELY named apart from `-l`, which emits the OWNING-session `parent_session_id`s; the two answer different "which sessions?" questions. On a ZERO-match result the summary gains `definitive_absence:true` + `active_filters` + `excluded_by_label` (§ empty-result self-diagnosis, below). `excerpts_truncated` = ≥1 emitted excerpt was clipped to the cap (machine echo of the text reader-caution; always false under `--no-truncate` — and under `show`'s `--line`/`--uuid`/`--turn` — since those lift the cap to `usize::MAX` so `Hit::truncated` is never set). Reproduce BOTH when adding a JSON surface that spans/caps. **v0.5 row-kind set** (`kind` belongs to the envelope alone): `search --count-by <axis>` emits `census` rows `{axis, key, records}` + a `{axis, matched_records, distinct_keys, excluded_records, dropped_by_cap, skipped_lines}` summary; **`agents --format json` is FLAT** — `{kind:"session"}` count row → `{kind:"run"}` rows → each subagent a `{kind:"agent"}` row in tree PRE-ORDER (nesting is TEXT-only, rebuilt from `parent_agent_id`/`depth`; NO nested `workflow_runs`/`children` row field, and `--agent` is no longer a bare-node exception); a node's transcript shape is `shape` (not `kind`). `files` summary gained `sessions` (distinct owning sessions; deliberately NO `dropped_by_cap` — files has no cap); `show` summary is `{records, dropped_by_cap, refetch_remainder, non_record_lines, skipped_lines, with_elicitation_sidecar}`. **`pairing` is a closed machine enum** `paired`|`pending`|`orphan`|`null`. **Timestamp PAIR LAW:** every machine instant is a `_utc`+`_local` PAIR — a record's OWN instant is `ts_utc`+`ts_local`; a NAMED instant is `<name>_utc`+`<name>_local` (`first_*`/`last_*`/`trigger_*`/`started_*`/`completed_*`/`last_activity_*`/`pending_since_*`; on an agents node `completed_*` is STATUS-GATED — non-null only when `status==completed` — and `last_activity_*` carries the tail instant on every timestamped lane). The text form derives from these but drops the UTC copy (§4 timez helpers).
- **Shared `text::` + `timez::` helpers are REQUIRED, not optional.** Do NOT hand-roll truncation, the scope banner, the malformed-line note, or a range parser - a divergent fourth copy (`agents::one_line` dropping the elision count) is exactly the silent-truncation contract bug this consolidation killed. REUSE:
| Need | Helper | Notes |
| --- | --- | --- |
| excerpt + explicit `... (+N chars)` | `text::truncate_excerpt(s, max)` | CHAR-counted (codepoint-safe); cap = **200** for `list`/`agents` scannable previews, **400** for `search`/`recover` context excerpts |
| flatten multiline + truncate | `text::collapse_and_truncate(s, max)` | the `agents` returned-message preview |
| malformed-line note fragment | `text::malformed_note(n)` | bare `N malformed line(s) skipped`; caller frames it |
| scope span wording | `text::scope_span_fragment(top, sub)` | the one phrasing |
| text scope banner (suppress when `sub==0`) | `text::emit_scope_banner(top, sub)` | `list`/`files`/`search`/`recover` |
| JSON scope header line (ALWAYS the envelope's line 1) | `text::envelope_scope_header(command, top, sub)` | `{kind:"header", command, sessions_in_scope, top_level_sessions, subagent_sessions}` (text.rs); emitted unconditionally |
| range spec: PARSE, then RESOLVE against a domain length | `text::parse_range_spec(s, label, one_based) -> RangeSpec` then `RangeSpec::resolve(len, one_based) -> (usize, usize)` | THE one range grammar (`--line` · `--turn` on EVERY command that windows on turns — `search`/`show`/`stats`/`files`/`recover`/`verbatim`/`image` · `--file-lines`): `N` (single) · `A..B` (closed) · `N..` (to end) · `..N` (from start) · `..` (all) · negative `-k` = k-th from the END (`-3..` = last 3, `-1` = the last). Types: `text::Endpoint` (At / FromEnd / Open) + `text::RangeSpec {start, end}`. A statically-detectable reversal (`9..3`, both explicit) errors at PARSE; a len-dependent reversal resolves to an EMPTY range. Each consumer resolves the spec against its OWN per-file domain length (turn count / line count) — `search`/`stats`/`files`/`recover`/`verbatim`/`image` resolve `--turn` per-transcript; `show --line` / recover `--file-lines` against the file's line count. The dash form `A-B` still hard-errors with the correct spelling. The range-valued clap args carry `allow_hyphen_values = true` so the space form `--turn -3..` parses (not mistaken for a flag). |
| timestamp — the ONE canonical local text form `YYYY-MM-DD HH:MM:SS <TZAB>(UTC±offset)` | `timez::format_timestamp(raw)` (seconds) | system-local via jiff, infallible; NO raw-UTC copy (that lives only in JSON `ts_utc`) — shares `tz_marker` with the ms variant |
| compact local instant `…HH:MM:SS.mmm <TZAB>(UTC±offset)` (no UTC copy) | `timez::format_local_compact(raw)` (ms) | for token-lean text; same renderer + `tz_marker` as `format_timestamp` |
| JSON `ts_local` ISO+offset | `timez::local_iso(raw)` | `None` if absent/unparseable |
| the detected local zone | `timez::local_tz()` | `TimeZone::system()` |
- **Performance is a contract** (SPEC section 7). `list`/`search` stay fast on 200MB+ files via: **mmap** (`parse::mmap_bytes`) + **`memchr` SIMD newline scan** (`scan_lines_bytes`, `memchr_iter`) + a cheap **byte/regex prefilter** with full `serde_json::from_slice` ONLY on candidate lines (`parse_candidates_parallel`) + **tail reads that SEEK from EOF backward** newest-first (`tail_records`, never parse the whole file to find the last user/agent msg; `list` additionally byte-prefilters head/tail lines via `head_records_prefiltered`/`tail_records_prefiltered` so megabyte metadata lines are never parsed) + **`rayon`** fan-out across files AND within one giant file (`scan_lines_parallel` — `search`/`verbatim`/`files`/`recover` all use it) + the **§7f whole-file gate** (see the landmine below) + **`mimalloc`** as the global allocator (macOS libmalloc contends across rayon workers) + heavy raw fields kept UNPARSED (`Record.tool_use_result`/`attachment` are `Box<RawValue>`; the hot paths read small fields via `Record::tur_probe`, deep consumers parse on demand via `tool_use_result_value`/`attachment_value` — never revert these to eager `Value`) + **`subagent::JournalCache`** (each workflow `journal.jsonl` read+parsed ONCE per topology build, not once per agent). Don't regress these: don't `BufReader`-copy a whole file, don't full-parse every line, don't lose the prefilter. **Prefilter silent-drop landmine (where "no silent truncation" actually bites).** `search`'s literal prefilter (`required_literal` + the `Prefilter` enum, search.rs) is the ONE place a naive optimisation silently drops matches. It scans RAW JSON line bytes, where string content is JSON-encoded (`"` -> `\"`, every control char `< 0x20` + DEL -> `\uXXXX`/`\n`/...), AND several render paths rewrite whitespace before matching (`normalize_line` collapses runs; multi-part texts join with `' '`/`'\n'` seams). So prefilter needles are derived NECESSITY-ONLY by `required_needles` (matcher.rs, v0.9.4): the whole-pattern plain literal stays the fast path, and a pattern WITH metachars goes through the `hir_needles` walk — a literal contributes its longest SAFE run (whitespace + JSON-escaped chars SPLIT runs; a sub-run of a required literal is itself required), a concat contributes its strongest part, an alternation gates ONLY when EVERY branch contributes (union, <=8 needles), min-0 repetitions/classes/anchors contribute nothing. The safety predicate applies PER NEEDLE: no JSON-escaped char (`json_escapes_in_string`: `"` / control / DEL), no whitespace (a rendered `hello world` can be raw `hello\nworld`), >=3 bytes. An eligible needle set gets a byte prefilter in BOTH case modes: case-sensitive -> per-needle `memmem` (`Prefilter::Literal`/`AnyLiteral`); smart-case-insensitive -> ONE `(?i)`-escaped-literal-alternation bytes regex (`Prefilter::CaselessLiteral`, Teddy-class caseless scan). Non-ASCII (`>= 0x80`) survives verbatim as UTF-8, so it stays prefilter-eligible. **Synthesized-text escape hatch (`Matcher::synth`, REQUIRED for correctness):** a small closed set of render paths fabricates matchable text that is NOT a verbatim substring of the raw line (automation labels' kind slug/status, the AUQ scaffold, a rejection's `[plan: ...]` pointer resolved from a DIFFERENT record, the compact-boundary `trigger=...` excerpt, `--resolve-persisted` external content); each is detectable by a raw marker its carrier ALWAYS bears, checked via per-needle `memmem` finders (NOT Aho-Corasick — the quote-leading `"answers"` needle degenerates AC's start-byte prefilter on JSON). Markers split two tiers (`Matcher::synth_verifiable`/`synth_conservative`): VERIFIABLE markers (`<task-notification>`, `"answers"`, the AUQ answer markers, `-t`-gated `compact_boundary`) re-render JUST the marker lines through the SHARED engines (`record_text_sections`/`auq_exchange`/`record_raw_text`) and regex-check the synthesized texts — so a marker-heavy main session still gates when none of them can match; CONSERVATIVE markers (`To tell you how to proceed` — its `[plan: …]` pointer resolves through ANOTHER record — and the flag-gated persisted pointers — external file content) force the full scan. (No `AskUserQuestion` needle: the question-side `tool_use`'s matchable text is the verbatim `name` + re-serialized `input`, and the name bytes sit in the raw line — a bare needle would disable the gate for every file whose injected context merely mentions the tool.) **Whole-file gate (§7f):** when a PARALLEL per-line pre-scan (same rayon chunking as the full scan; a relaxed AtomicBool short-circuits it on the first may-match line) proves a whole file can't match (and no `--line`/`--uuid` addressing), `search` skips building records for it entirely — the gated file's candidate lines were each syntax-validated (`parse::validate_line_syntax`, no Record build) so the malformed-line count stays exact (a TESTED contract). Extraction is NECESSITY-ONLY and safety-gated per needle — NEVER emit a needle a match could lack (an alternation with one needle-less branch gets NO gate), NEVER skip the per-needle safety predicate, NEVER prefilter decoded content, and NEVER add a synthesized render path without adding its raw marker to `synth_marker_finders` — when no safe needle set exists, fall back to running the regex on the raw bytes (still pre-JSON, just no cheap short-circuit).
- **`unsafe_code = "deny"` crate-wide, ONE exception.** `deny` (not `forbid`) is deliberate: it bans every `unsafe` block EXCEPT the single audited mmap site in `src/parse/` (`#[allow(unsafe_code)] unsafe { Mmap::map(&file) }`), which is irreducibly unsafe and SPEC-mandated. `forbid` cannot be locally overridden so it would refuse to compile mmap. Any OTHER `unsafe` without an explicit allow is a hard error. Do NOT add a second allow.
- **No crate-level `#![allow(dead_code)]`.** The only `#[allow(dead_code)]` is targeted on `model::Record`/`Block` fields deserialized for tolerance but not yet read. A crate-wide allow would mask real dead code.
- **Exit codes** (main.rs): `Ok(()) -> ExitCode::SUCCESS` (0); any `Err -> eprintln!("csift: error: {err:#}")` (full anyhow chain, alternate `:#`) + `ExitCode::FAILURE` (non-zero, no custom numeric codes) — with ONE documented exception: `wait`'s timeout exits **124** (the GNU timeout convention; a monitor's timeout is a normal outcome scripts branch on, rendered via `std::process::exit` after the output flushes). Do not extend 124 to any other command or outcome. De facto (v0.6.4, informational): a clap USAGE error exits 2 (clap's own convention), a csift `bail!` exits 1 — never build on the split, the contract is 0-vs-non-zero. NOT errors (exit 0): `verbatim --slice` out-of-range prints nothing; a ZERO-match `search` still exits 0 (a DEFINITIVE absence, NOT an error) and prints its NORMAL empty-result output on STDOUT (`no matching exchanges` in text; the `kind:"header"` + `kind:"summary"` envelope lines in json) — only the `csift: 0 matches …` self-diagnosis rides STDERR (see §5). HARD non-zero: `show` address-miss (an explicit line/uuid/turn resolving to no record → `bail!("no such record(s): …")`); `recover` partial/no-history (`bail!` pointing at `--salvage`/`--patches`/`--coverage`); `image --id` no-match or ambiguous `#N` (`bail!`).
---
## 5. Module map (`src/` — directory modules after the 2026-08 modularization)
Every former >600-line single-file module is a DIRECTORY module with a PATH-NAMED root: `src/<m>.rs` beside `src/<m>/` (the root carries the module doc, shared imports, `mod` decls + `pub(crate) use` re-exports — every `crate::<m>::item` path still resolves), children are focused submodules (~200-500 lines, each opening `use super::*`), and a module's unit tests live in `src/<m>/tests.rs` (shared fixture helpers) + `src/<m>/tests/<feature>.rs` (feature-named files that mirror the source children — never numbered slices). **`mod.rs` is BANNED crate-wide** (the legacy layout): enforced twice, by clippy `mod_module_files` and by the pre-commit structure gate. Dispatch conventions are unchanged (`Command::Foo(args) => foo::run_foo(&args)`; variant->module still not 1:1 — `list` lives in `session/`). HARD structure limits (husky-enforced, §7): <=600 lines per `.rs`, <=20 `.rs` per folder (a paired module root `name.rs`+`name/` counts with its directory, not against the parent), <=16 subfolders per folder (raised from 15 at v0.9.0: src/ grows one dir per command family by design and sits AT the cap - the next command module must consolidate an existing dir first), <=5 levels below `src`/`tests`; per-function cognitive complexity <=25 (clippy.toml); no `mod.rs`.
```
main.rs # binary entrypoint: parse_argv -> install --claude-home override -> dispatch -> ExitCode
cli/ # clap surface: argv (the P0 normalize_argv machinery), root (Cli + Command),
# selectors (-t/-T label machinery), formats, one *_args file per command family
path/ # home (CC-exact cwd encoding + config home), project_dirs (>200 prefix scan),
# scope, trap (@trap self-id), targets (--sessions-from), resolver (the
# @-grammar: collect_targets/scan_top_level/resolve_prefix_uniqueness), ids
model/ # markers, record (serde model), predicates (is_genuine_user &c.), exchange
# (AUQ/opens_turn/reconstructed text), mutation (FileOp), grouping (turns),
# automation (pulse model), taxonomy (Role/Class::ALL), peer, classify_support,
# classify_promoted (the v0.10.0 non-record leaves: queue/turn-duration/away/stop-hooks/snapshot),
# classify (the role.class.sub engine), narration (thinking-signature tag decoder)
parse/ # lines (mmap + tolerant role matchers [the ONE unsafe mmap site], lazy parse),
# parallel (rayon scans + malformed accounting), readers (RevLines, head/tail)
session/ # `list`: rows, run (DEFAULT_LIST_CAP flood guard), summarize (+ the
# C-19 clone probe + origin join), render
search/ # types (Hit/Exchange), matcher (prefilter + synth markers, SPEC 7d/7f), census
# (-c/-l/--count-by + zero-match diagnosis), run (run_search), scan
# (search_one_file + fetch_records — show's engine), turns_match, hits (the
# emission engine), record_text, render (headers/tokens/envelope)
show/ # addr (line/uuid/turn specs + caps), run (raw/rendered fetch), render,
# branch (--branch-points fork facts)
stats.rs # one-scan per-session aggregates
subagent/ # ids (the r5 trio helpers), types, discover, meta (+JournalCache), lifecycle
# (frozen-lane detection), spawn (ParentSpawnIndex/global index), topology
agents/ # run (filters/windows/teammate hint), render (text tree), json (flat rows)
files/ # types, run, mutations (structured + bash heuristics), rollup, render
bash_mutations/ # entry, heredoc, mask, commands, outputs, redirect, cwd (CwdAt/Resolution
# tracking + resolve), interp (write-idiom analysis), classes (fmt:/pkg:/
# extract: markers), anchors (content-anchor classifier: the v0.9.4
# deterministic read/write shapes) - the lexical bash parser
bash_danger.rs # faithful 1:1 port of CC's dangerous-rm classifier
recover/ # types, run (batch driver), backups (--list-backups store listing),
# bash_anchors (per-turn content-anchor gates + suppression),
# scan (+opaque accounting), events (extraction),
# carriers (toolUseResult/attachment + freshness signals), buffer (SparseBuffer),
# replay, diff (LCS/unified), render (disclosure + merge), restore, report, timeline
plan.rs + plan/ # plan forward/reverse resolution (reuses whoami's detection + guidance);
# audit (--audit: edits joined against corpus bindings)
live.rs + live/ # `status` + `wait` (the live-truth pair): background (types/lens/report) +
# background_scan (needles/ingesters/join/stat), last (the newest prompt +
# reply excerpts), activity (the wait census), tasks (the harness task list),
# registry (sessions/<pid>.json
# + ps pid probe), tail (bounded tail state machine), children
# (subagent tails + incremental journal), verdict (six-verdict join),
# conditions (--until grammar), status, wait (poll loop), render
turns/ # `verbatim`: config, units, run, build, richness, select, planning
# (plan_session), render, scope, json
image/ # refs (base64 probes), selection (#N), run, convert (transcode), render
elicitation.rs # transparent elicitation-sidecar merge (no subcommand)
time_window.rs # --since/--until parsing (shared by all WHEN consumers)
timez.rs # REQUIRED shared time helpers
text.rs # REQUIRED shared helpers: excerpts, banners, envelope, the range grammar
whoami.rs # `whoami`: CLAUDE_CODE_SESSION_ID detection, false-positive-safe
```
**E2E tests** live in ONE integration binary `tests/cli/` (main.rs declares the modules): `harness.rs` + `harness/` (the `Home` isolation harness in `home.rs` + the shared fixture builders in `fixtures.rs` — all helpers live HERE, feature files carry no cross-file dependencies). Per-SUBCOMMAND modules own that command's tests (`search/` split by aspect — basics/filters/census/headers/taxonomy/classify/output/scope; `agents/`, `verbatim/`, `recover/`, `files/`, `targeting/` as feature dirs; `list`/`stats`/`show`/`image`/`plan_whoami` as single files), and CROSS-COMMAND sweeps get their own root files: `argv.rs` (flag ordering, exit conventions), `contracts.rs` (range grammar, timestamps, honest empties), `spanning.rs` (span-switch/scope-banner/cap uniformity), `elicitation.rs` (EVERY sidecar-merge test across surfaces). A new e2e test goes in the module whose FEATURE it exercises, never a grab-bag file. The golden baseline stays at `tests/turns_pre_feature_baseline.txt`.
**`search` flags from the verified audit** (cli.rs `SearchArgs`, search.rs): `-c/--count-only` prints ONLY the integer total `exchanges.len() + dropped_by_cap`; `-l/--sessions-with-matches` prints ONLY the distinct OWNING session ids (uncapped, stderr-noted on a `--max-count` drop — the pipe into `--sessions-from -`); the JSON summary's `transcript_ids` stays the per-transcript detail set (sorted, cap-100 + explicit `transcript_ids_truncated`) — DELIBERATELY named apart from `-l` (which emits the OWNING-session `parent_session_id`s), so the two "which sessions?" answers never collide. `--count-by <AXIS>` is a THIRD terminal mode (like `-c`/`-l`): a per-KEY CENSUS of the matched records along ONE closed axis (`label`|`tool`|`turn`|`session`|`pairing`|`model`|`attachment`|`version`|`result`; an empty pattern = a whole-scope census; the `attachment` axis IMPLIES the `--attachments` gate; `result` buckets tool results `ok`|`error` - pairing answers "did a result come back", result answers "was it good"; an errored result also renders an inline `[error]` and hit JSON carries `is_error`). On `label` each record counts under every leaf it carries THAT SURVIVES the active `-t`/`-T` (v0.6.3: census keys pass the same `LabelFilter::selected` predicate that admits the record's views — a dual-labeled record never leaks its filtered-out twin into the keys; no filter ⇒ the full label set, and a leaf's count = how many records `-t <leaf>` would surface, via `search::label_census` — also the zero-match probe engine, which passes `LabelFilter::all()` since it exists to name what the DROPPED filter excluded); every OTHER axis counts each record ONCE and EXCLUDES records outside its domain (no tool/pairing/model), reporting the excluded count (`search::axis_census`). A multi-SECTION record (text+tool_use, a batched notification) emits one hit per section but is ONE census record (`record_groups`, search.rs). `set_pairing` rides the tool BLOCK through the comm views (a SendMessage/spawn tool_use → sent/signal, a subagent-return tool_result → inbox), so a frozen SendMessage is `pending` under ANY selector; record-text comm units (no tool_use_id) stay outside the axis. `turn` sorts ascending (histogram); the rest count-desc except `label` (richest-first). JSON emits `census` rows `{axis, key, records}` + a `{axis, matched_records, distinct_keys, excluded_records, …}` summary. `-T/--label-not` excludes labels (same selector grammar as `-t`, `LabelFilter` include-minus-exclude, richest-SURVIVING-view dedup, statically-empty combos bail). `--raw` emits matched records' VERBATIM jsonl lines (per-file mmap backfill onto `Hit.raw`; stdout pure, notes on stderr; sidecar hits omitted + noted). Every JSON hit carries `refetch` (the ready-to-run `csift show` command addressed at the hit's own `session_id`); in TEXT output a SUBAGENT hit line ALSO prints a `↳ csift show @<agent-id> --line N` continuation — the exact refetch with the CORRECT id, closing the text-mode ID-law hole where a parent-uuid + subagent-line would silently fetch the WRONG record (a line number is per-FILE); top-level hits stay terse (safe via the header's id-prefix token — a valid `@` target since v0.7.0). `--siblings` is a ZERO-ARG flag with a FIXED policy (`sibling_cap`): message classes uncapped, thinking≤2 / thinking.narration≤1 / tool.use≤3 / tool.result≤3 / harness≤2 per leaf; the capped remainder renders an explicit `(+N more · csift show @<id> --line A..B)` pointer and rides JSON as `siblings_hidden` + `turn_lines`. Record FETCHING moved to `csift show` (search has no `--line`/`--uuid`; `AddressSet` + `fetch_records` remain in search.rs as show's engine). `--resolve-persisted` rewrites tool-result text pre-match (failures soft-noted; under `--raw` it affects matching only). PATTERN guardrails: leading `@` = hard error; uuid-shaped + no session target = stderr note.
**Empty-result self-diagnosis (the keystone, `search`).** A ZERO-match search is a DEFINITIVE absence, NOT an error — exit 0, with stdout carrying the NORMAL empty-result output (`no matching exchanges` in text; the `kind:"header"` + `kind:"summary"` envelope lines in json). The diagnosis rides STDERR: a line framing the result as `0 matches — a definitive absence (exit 0), NOT an error`, the active filters (`active_filters_str`), and — WHEN a `-t`/`-T` label filter was active — an ACTIVE PROBE: a re-scan of the SAME pattern + scope with the label filter DROPPED that NAMES the label(s) the pattern DOES occur under (e.g. `⚠ but "X" DOES occur — N record(s) under: agent.tool.use`). The probe cost is paid ONLY on a zero-hit + label-filtered query. JSON echoes it in the summary: `definitive_absence:true` + `active_filters` + `excluded_by_label` (`{records, by_label}`, null when no label filter). Structures: `search::EmptyDiagnosis` / `emit_empty_diagnosis` / `active_filters_str`. Rationale: the observed #1 slippage cause is a model reading an empty result as a SYNTAX error and abandoning csift — this makes absence machine-legible and self-correctable.
**argv normalization:** the entrypoint is `cli::parse_argv` (NOT `Cli::parse`): it runs `cli::normalize_argv` to reorder declared flags ahead of leading-`-` encoded-project positionals, fixing clap's `allow_hyphen_values` greedy-absorb bug (#3880). Flag discovery is zero-drift via `Cli::command()` introspection (action `Set`/`Append` take a value; `SetTrue`/`SetFalse`/`Count`/help/version do not). **v0.5 P0 fix:** the subcommand is now located by SCANNING PAST any declared ROOT flags (+ their value tokens; a `--flag=value` inline form spans one token) rather than assuming `argv[1]` is the subcommand — so a pre-subcommand GLOBAL flag no longer disables normalization. Previously `csift --claude-home DIR list @x --max-count 3` mis-parsed (the trailing flag got swallowed by the PATH positional with a misleading "not a project target" error); now "flag order is free" and "`--claude-home` in any position" hold IN COMBINATION. So `csift list <ENCODED> --format json` and `csift --claude-home DIR list @x --max-count 3` both work.
---
## 6. How to add or modify a subcommand
1. **Args struct** in `cli.rs`: add `pub struct FooArgs` (`#[derive(Args, Debug)]`). Positionals that target sessions use `#[arg(value_parser = parse_project_target)]` so a `--`-typo errors cleanly. Put example-rich text in `#[command(after_help = "...")]`/doc comments (the `--help` is the LLM's manual - keep it usable from `--help` alone). If the subcommand spans subagents, add the span flag(s) + a `pub fn want_subagents(&self) -> bool` mirroring the existing ones (src/cli//1048/1220/...).
2. **Command variant**: add `Foo(FooArgs)` to `enum Command` (src/cli/).
3. **Dispatch**: add one arm to `run()` in `main.rs`. The real arm shape is `Command::Foo(args) => foo::run_foo(&args)` (handler fn convention is `run_<subcommand>`: `run_list`/`run_search`/`run_agents`/...). **The dispatch MODULE need not be named after the `Command` variant** - the precedent is `Command::List(args) => session::run_list(&args)` (main.rs): `list` lives in `session.rs` (`pub fn run_list`, src/session/); there is NO `src/list.rs`. Variant->module is NOT 1:1; pick whatever module already owns the concern. Every handler returns `anyhow::Result<()>`, so `?`/`bail!` funnel to the single FAILURE arm; never `eprintln!`+exit yourself.
4. **Module**: host the handler in a `src/*.rs` (a new `src/foo.rs`, or fold into an existing module as `list` did into `session.rs`); declare `mod foo;` in `main.rs` if new (keep alphabetical-ish). The handler `run_foo` then resolves targets via the shared resolver:
`let scope = SubagentScope::from(args.want_subagents()); let files = path::resolve_session_files(&args.paths, scope, path::Caller::Other)?;`
- **Resolve via `&args.paths`** - that is the convention for `list`/`agents`/`files`/`recover`/`verbatim`/`image`/`plan` (src/session/, src/agents/, src/files/, src/recover//373, src/turns/, src/image/, plan.rs/263). The ONE exception is `search`: its first positional is `PATTERN`, so `SearchArgs` adds the SOLE `targets()` helper (src/cli/ - the only `fn targets` in the tree) that returns the path positionals only (`self.paths.clone()`, excluding the pattern); `search` resolves `&args.targets()` (src/search/). Do NOT add a `targets()` method to a non-`search` struct (it has none -> compile error); use `args.paths`.
- **Scope resolution.** Every default-on span command (`list`/`search`/`stats`/`files`/`recover`/`image`/`plan`) exposes `want_subagents() -> bool` (= `self.subagents || !self.no_subagents`) and resolves `SubagentScope::from(args.want_subagents())` (`true->WithSubagents`, `false->TopLevelOnly`). `verbatim` is the opt-in exception (`self.include_subagents && !self.no_subagents`; flag spelled `--subagents`). `SubagentScope` has only those two variants; there is no subagents-only mode.
- `agents` passes `false.into()` because it LISTS subagents rather than spanning them.
- The `path::Caller` arg is currently **INERT** - `resolve_session_files` does `let _ = caller;` (src/path/, "reserved for future subcommand-aware guidance"). Pass `path::Caller::Other` (or the matching variant) to stay future-proof, but do NOT expect it to change remediation text today; the doc's "Caller tunes messages" is forward-looking, not live.
Read files via `parse::mmap_bytes` + the head/tail/parallel scanners; lazy-parse only candidate lines; **derive each row's id trio with the `crate::subagent` helpers** (`session_id_from_path` / `is_subagent_path` / `parent_session_id_from_path` - never read the stem, see section 3.7); render via `text::` excerpt helpers + `timez::` timestamps. Honor: no `unwrap`, report any cap's drop count, support `--format json` via a `json!` projector (section 4 JSON invariant - no `derive(Serialize)`).
5. **Tests**: add an end-to-end case to the matching FEATURE module under `tests/cli/` (one integration binary driving the built csift against synthetic transcripts; per-subcommand dirs and the cross-command root files are mapped in §5 — put a new test in the module whose feature it exercises, never a grab-bag file). **Isolation mechanism (load-bearing - reuse it, don't hand-roll a fixture scheme):** the e2e harness never touches the real `~/.claude`. The `Home` struct (`tests/cli/harness/home.rs`) makes a TempDir, creates `.claude/projects` under it, and `Home::write(rel, contents)` drops a fixture jsonl at `$HOME/.claude/projects/<rel>` (e.g. `<ENC>/<uuid>.jsonl`). Its `run`/`run_with_env`/`run_full` invoke the binary via `env!("CARGO_BIN_EXE_csift")` (the exact build cargo produced) with `.env("HOME", &self.root)` and `.env_remove("CLAUDE_CODE_SESSION_ID").env_remove("CODEX_COMPANION_SESSION_ID")` for deterministic `whoami` (a test that needs a session id sets it back via `run_with_env`). This works because `claude_home()` (src/path/) falls back to `$HOME/.claude` when no `--claude-home`/`$CLAUDE_CONFIG_DIR` is set. Start from the `populated_home()` builder (`tests/cli/harness/fixtures.rs`) - the canonical fixture, used in ~198 cases (`populated_home()`/`Home::new()` combined) - rather than inventing a new tree.
Unit tests local to a module go in `src/<m>/tests.rs` (shared fixture helpers + `mod` decls, wired by a plain `#[cfg(test)] mod tests;` at the bottom of `src/<m>.rs` - **no `#[path = "..."]` attribute needed**) with feature-named children under `src/<m>/tests/` that mirror the source submodules; child files never depend on each other. A SMALL module may keep an inline `#[cfg(test)] mod tests { ... }` while it fits the 600-line cap. Run `cargo test`.
---
## 7. Commands + quality gate
```bash
cargo build # debug (GATE: must succeed)
cargo build --release # optimised (thin-LTO, 1 cgu) for real scans
cargo run -- <subcommand> ... # run a subcommand
cargo fmt --all -- --check # format gate
cargo clippy --all-targets -- -D warnings # lint gate (warnings-as-errors)
cargo test # unit + integration tests (also installs the hook on first run)
```
**Pointing csift at a fixture tree (not the real `~/.claude`).** csift resolves its data root (`claude_home()`, src/path/) with precedence `--claude-home <DIR>` > `$CLAUDE_CONFIG_DIR` > the OS home's `.claude` (`$HOME` on Unix, `%USERPROFILE%` on Windows — the split mirrors CC's own `os.homedir()`; a stray Git-Bash `HOME` on Windows is deliberately ignored, v0.7.1), then reads `<root>/projects/<encoded>/*.jsonl` (`projects_root`). `--claude-home` is a clap **global** flag (`global = true`, src/cli/; installed in `main` before dispatch, main.rs), so it is honored by EVERY subcommand and `normalize_argv` lets it sit before OR after the subcommand: `csift --claude-home /tmp/fix list` and `csift list --claude-home /tmp/fix` both work. Use `--claude-home <dir>` (or export `$CLAUDE_CONFIG_DIR`) to run a manual repro against a synthetic projects dir instead of your live transcripts. All three positions are exercised by the `custom_claude_home_via_env_var_and_flag` e2e test. **Test-isolation trap:** `--claude-home` installs via `set_claude_home_override`, which writes a SET-ONCE process-global `OnceLock<PathBuf>` (`CLAUDE_HOME_OVERRIDE`, src/path/; `let _ = ...set(dir)` silently ignores later sets, src/path/). So an IN-PROCESS unit test CANNOT relocate the data root via the override - the first set wins for the whole test binary (all tests share one process). For path-resolution unit tests call the PURE `resolve_claude_home(flag, env, home)` (src/path/, factored out precisely so precedence is testable WITHOUT touching the OnceLock); for e2e use the `Home` harness, which spawns a fresh subprocess per test with `$HOME` set (`tests/cli/harness/home.rs` `run_full`). Don't write a flaky test that calls the global `claude_home()`.
**Pre-commit gate (cargo-husky).** On the first `cargo test`/`cargo build` after checkout, cargo-husky (dev-dep, `user-hooks` feature) installs `.git/hooks/pre-commit` from `.cargo-husky/hooks/pre-commit`. It runs, in order: the STRUCTURE GATE (hard limits: <=600 lines per `.rs` file [Markdown exempt], <=20 `.rs` files per folder, <=16 subfolders per folder, <=5 subfolder levels below `src`/`tests`) -> `cargo fmt --all -- --check` -> `cargo clippy --all-targets -- -D warnings` (which also enforces per-function cognitive complexity <=25 via clippy.toml + the `[lints.clippy] cognitive_complexity` warn) -> `cargo test`. A failure blocks the commit. No CI service runs - this hook is the ENTIRE gate. Edit the **source** hook (`.cargo-husky/hooks/pre-commit`), never the installed `.git/hooks` copy, then re-run `cargo test` to reinstall. Genuine-WIP bypass: `git commit --no-verify` (sparingly). Test layout: `tests/cli/` (the ONE e2e binary, feature modules + `harness/`), `tests/turns_pre_feature_baseline.txt` (golden), and per-module unit tests under `src/<m>/tests/`.
**Versioning + release discipline (LOAD-BEARING — the SKILL's staleness guard depends on it).** The version is TRIPLE-LOCKED and the three MUST move together in the SAME commit: `Cargo.toml` `version` ≡ `SKILL.md` `Surface: **vX.Y.Z**` header ≡ `csift --version`. This is not cosmetic: `SKILL.md` line 3 tells the LLM consumer "Surface vX must == `csift --version`; if an invocation you were confident about errors, your knowledge is stale — re-read this file", so it uses version==surface as its ONLY staleness detector. Shipping a changed surface under an unchanged version silently defeats that guard and puts two DIFFERENT binaries in the wild claiming the same version — a correctness bug for the primary (LLM) consumer, not a nicety.
- **What is a "surface change" (⇒ REQUIRES a version bump):** any change to a command, a flag, a flag's semantics, a default value, a JSON field name/shape, a row `kind`, output text a consumer keys on, OR the `--help` text (SKILL calls `csift <cmd> --help` "the authoritative flag manual", so help IS surface). Pure internal refactors, tests, comments, and non-user-facing docs are NOT surface changes and do not require a bump.
- **Which number (0.x policy — csift is pre-1.0, so per SemVer the MINOR carries breaking changes):** bump the **MINOR** (`0.Y.0`) for a BREAKING surface change — a removed/renamed command or flag, a renamed/removed JSON field, a changed default or exit semantics (e.g. v0.3→v0.4 = `turns`→`verbatim`, `session_ids`→`transcript_ids`, removed `--subagent`). Bump the **PATCH** (`0.4.Z`) for a NON-breaking surface change — an added flag, a corrected help string, a fixed output, a bug fix (e.g. v0.4.0→v0.4.1 = help/doc drift corrections). At 1.0+, breaking ⇒ MAJOR.
- **Tag discipline:** every released version gets an ANNOTATED git tag `vX.Y.Z` on its release commit (`git tag -a vX.Y.Z -m "…"`), and tags are pushed (`git push --tags`). The release commit is `chore(release): vX.Y.Z`. Never leave a shipped version untagged.
- **Changelog discipline:** `CHANGELOG.md` (repo root) carries ONE digest entry per released version, newest first (`## [X.Y.Z] - YYYY-MM-DD`), written IN the release commit — never after the fact. The SPEC §6 per-version ledger stays the authoritative long-form record; the changelog is its scannable digest, and the two must agree. Entries are contemporaneous: name flags/fields by their names AT that version (no forward references to later renames).
- **Introspection ledger audit (v0.10.1, a RELEASE GATE).** `INTROSPECTION.json` at the repo root lists every Claude Code behavior csift depends on, one claim per entry, and the README's "verified against Claude Code X" badge is admitted ONLY through it: `tests/cli/ledger.rs` (run by name in the pre-commit hook, and again under `cargo test`) refuses a badge version any claim lacks a check at. BEFORE EVERY RELEASE a verification subagent per area re-runs each claim's instrument against the CURRENT Claude Code binary + live corpus and writes that claim's own check record - **mechanical replacement of the check records is forbidden in every case**. A `drifted` verdict is a correctness task for the same release. The full schema, the gate rules and the four procedures (new behavior, moved code, release audit, retirement) are in §7.1.
- **Release flow (do ALL, in order):** (0) run the introspection ledger audit (above) at the Claude Code version the badge will name, write each claim's check, bump the README badge + `verified_claude_code` together; (1) bump the triple (Cargo.toml + SKILL header) AND add the version's CHANGELOG.md entry in one commit; (2) `cargo build --release && cargo install --path .`; (3) confirm `csift --version` == the SKILL surface header; (4) the pre-commit gate runs on commit (`chore(release): vX.Y.Z`); (5) `git tag -a vX.Y.Z`; (6) `git push && git push --tags`; (7) if the surface change renamed/removed a COMMAND, re-sync the global hook in lockstep (see §9 / the `csift-turns-slice.sh` coupling). A pushed release is never amended/force-pushed — a follow-up fix is its own PATCH bump + tag.
### 7.1 The introspection ledger (`INTROSPECTION.json`): schema + procedures
**Why.** csift depends on hundreds of small facts about what Claude Code writes, and every Claude Code build can move one. The ledger pins each dependency as a CLAIM with the code that relies on it and the instrument that re-verifies it, so a release audit is a mechanical walk (one claim, one instrument, one receipt) and the README badge is a checked statement. The standing law behind it: every csift behavior is backed by OBSERVED Claude Code behavior, recorded as a claim in the SAME change that lands the behavior. "Based on a finding" is not a claim; a claim names what was measured, how, and with which counting rule.
**File shape (schema 1; plain JSON, 2-space indent; LLM-facing: dense and regular, never edited by hand-wave; the `about` field restates the no-bulk-edit rule).**
```
{ "schema": 1, "about": "...", "verified_claude_code": "2.1.258", "claims": [ <claim>, ... ] }
claim = {
"id": "REC-038", unique, never reused; "<TAG>-<NNN>" with TAG = the area's tag (below), NNN zero-padded
"area": "record-model", one of the 18 areas
"behavior": "...", what Claude Code DOES: field names, values, shapes, counts, the version it was seen at
"depends": "...", what csift does with that fact, and what breaks if the fact moves; for a deliberate
ABSENCE ("csift never asserts X") say why the absence is the correct reading
"code": [ { "path": "src/live/verdict.rs", "lines": "296-302", "snippet": "<verbatim excerpt>" }, ... ],
>= 1 site; snippet copied byte-for-byte (indentation included), 3-15 lines; lines 1-based inclusive;
a test file may be a SECOND site, never the only one
"instrument": "...", how a stranger re-verifies from scratch: the command or probe, the corpus scope, the counting rule
"located": { "claude_code": "2.1.258", "csift": "0.10.1" | null, "source": "measured <date>" | "dev session <date>" | "SPEC.md section N" | ... },
where the claim was first pinned; `source` is free text but repo-internal (a document section, a
measurement, a dev-session DATE - never a session id or an external path)
"first_seen_claude_code": "2.1.170" | null, the earliest build the behavior was observed at (a floor, not a proof of absence before)
"producer_trace": "complete" | "partial" | "none",
the PRODUCER leg (v0.10.3): "complete" = three hops read in the shipped binary as verbatim
excerpts at byte offsets - the TRIGGER (the entry that starts the path), the GATE (the
branch that selects this fact in this shape) and the WRITER (the template, field or append
that becomes the bytes); "partial" = a template or field found without its gate and trigger;
"none" = nothing read, only inferred
"specimen": "observed" | "none", the SPECIMEN leg: a corpus count with a counting rule, a raw record, or a live trial;
a self-written fixture is never a specimen
"attribution": "end-to-end" | "producer-only" | "specimen-only" | "partial-producer" | "by-elimination",
DERIVED from the legs (rule 7): complete+observed = end-to-end · complete+none = producer-only ·
partial|none + observed = specimen-only · partial+none = partial-producer · none+none =
by-elimination; "the only candidates are X, Y, Z" is not attribution, and by-elimination
can never hold
"open_legs": [ "..." ], one sentence per missing leg naming the instrument that closes it (required unless end-to-end;
an end-to-end claim carries NONE - an open leg is a leg that is open; a "TEXT: ..." line is a
claim-text correction a traced hop demands, consumed when the text is rewritten)
"residue": [ "..." ], optional: a non-gap note for a later audit (a version question answered by the historical
builds, a stronger specimen that would be nice) - never a missing hop, never an unread site
"producer_chain": [ { "role": "trigger" | "gate" | "writer", "name": "<function or template>", "offset": 158819054,
"excerpt": "<verbatim, <= 700 chars>", "note": "<what this hop does for the claim>" } ],
the traced hops (v0.10.3), present when producer_trace is complete or partial: the anchors a
future audit re-reads first (a moved offset with the same excerpt = the same producer; a
missing excerpt = the producer changed). More than three hops is normal when a fact has
several writers (a constructor plus the envelope plus the append)
"writer_literal_floor": { "literal": "...", "lowest_present": "2.1.208", "highest_absent": "2.1.207", "ladder": "..." },
optional (v0.10.3): the writer hop's minification-stable literal byte-searched across historical
native builds - the lowest build carrying it and the highest lacking it. A literal dates itself,
not the behavior (a rename moves it; an older wording may carry the same fact), so this is a
lead for the next audit, never a substitute for `first_seen_claude_code`
"enumeration": [ { "site": "<caller or occurrence>", "offset": 176355907, "excerpt": "<verbatim, <= 300 chars>",
"verdict": "<what this site does for the fact>" } ],
present when the claim is a NEGATIVE ("nothing else writes X", "the only emitter is Y",
"exactly five kinds"): the closure of a negative IS the enumeration - every occurrence of the
function or field in the whole binary, each read, each with a verdict. An enumeration that
stops early leaves the producer partial
"checks": [ { "claude_code": "2.1.258", "csift": "0.10.1", "date": "2026-09-02",
"verdict": "holds" | "refined" | "drifted" | "unverifiable-here",
"instrument": "<what actually RAN>", "observed": "<what it returned, with the numbers>",
"rule": "<the counting rule the numbers obey>", "note": "<what changed, what was fixed, the successor claim>" }, ... ]
one record per release audit, appended, never rewritten
}
```
Areas and tags: record-model `REC` · turn-boundary `TURN` · classification `CLS` · compaction `CMP` · elicitation `ELI` · subagents `SUB` · background-tasks `BG` · queue-and-telemetry `QT` · freshness-file-history `FH` · bash-shell `BASH` · plan `PLAN` · narration `NAR` · images `IMG` · windows `WIN` · path-encoding `PATH` · live-registry `REG` · live-tail `TAIL` · misc `MISC`. Add an area only when a dozen-plus claims need it; pick an unused tag.
**Verdict semantics.** `holds` = observed exactly as the claim states. `refined` = the behavior holds and the claim text gained precision (a number, a version floor, a newly seen shape); the behavior/depends edit rides the same commit. `drifted` = Claude Code no longer does what the claim says, or csift never actually matched it; a correctness task for the SAME release: fix csift and rewrite behavior/depends to the current truth (the check's `note` keeps the before/after), or retire the claim with a note naming its successor. `unverifiable-here` = the instrument cannot run on this host (a Windows-only shape audited on a mac, a corpus with zero specimens); `observed` names the reason and the claim keeps its last decided text. `date` is the day the instrument ran, not the release day.
**Gate rules (`tests/cli/ledger.rs`; the pre-commit hook runs it by name before the suite).** (1) `verified_claude_code` is a version triple and the README badge carries the same version. (2) Every claim has a unique id, non-empty area/behavior/depends/instrument, and at least one check AT `verified_claude_code` with a verdict from the closed set. (3) A decided verdict (`holds`/`refined`/`drifted`) carries a non-empty (instrument, observed) pair that is UNIQUE across the whole ledger - a bulk replacement leaves identical pairs and fails here; `unverifiable-here` names its reason in `observed`. (4) Every code site exists and its snippet is found verbatim (whitespace-normalized) in that file, so a refactor that moves code fails the gate instead of orphaning the claim. (5) The README carries the mutation-score badge. (6) Every claim cites at least one code site. (7) Every claim carries an `attribution` from the closed set and, unless end-to-end, a non-empty `open_legs`; an end-to-end claim carries no open leg at all; when the leg fields are present the attribution equals their derivation; a by-elimination claim never carries a `holds` verdict. (8) The README's ledger tally table (between the `ledger-tally` markers) equals the ledger's per-attribution counts; it is regenerated, never typed.
**Attribution law (why rule 7 exists).** A behavior is pinned only when BOTH legs are in the file: the producer (the writer in the shipped binary: a template string, a function, a branch condition) and a specimen (what it wrote, observed on disk or live). "The only candidates are X, Y or Z", "presumably", "it appears", a story that fits the shape, or csift's own code are not attribution: when a later build moves the real producer, such a claim lets the next audit conclude that nothing changed. So an audit that cannot close a leg writes it into `open_legs` and keeps the verdict at `refined` or `unverifiable-here`; upgrading `attribution` is itself a measured act, recorded in the check that did the tracing. **The closure rule:** when a fact's closure condition is an enumeration, the conclusion is to exhaust it - grep every call site in the whole binary, read each, list them (`enumeration`); a bounded sample, "the only candidate", or an argument from absence leaves the producer partial. When a hop REFUTES the claim's text, the text is rewritten in the same change (verdict `refined` or `drifted`), never softened into a note. When a leg is a version question ("which build first wrote this shape"), the instrument is the historical builds themselves: native builds are fetched by version from the distribution bucket the installer names, and a fact is bisected on a distinctive string literal (minified identifiers change between builds; templates and field names do not).
**Procedures.**
- *A new behavior lands* (a feature, a fix, a newly modeled record shape): its claim lands in the SAME commit - behavior with the observed shape, depends, at least one code site, an instrument a stranger can rerun, `located`/`first_seen`, and ONE check at the current `verified_claude_code` carrying the receipt of the measurement you actually made (the corpus census, the binary string count, the live trial). The v0.10.1 leaf `harness.meta.system` shipped this way (`QT-SYS-001`).
- *A refactor moves cited code*: the gate fails naming the claim ("snippet not found verbatim in <path> - the code moved - fix the site"); update `code[].path` and `lines`, and the snippet only when the code itself changed (a visibility keyword, a renamed binding). `checks[]` is never touched for a move.
- *A release audit* (release flow step 0): one verification subagent per area, fresh context, read-only on the repo. It reads each claim, RUNS the instrument against the current Claude Code binary and the live corpus, and writes that claim's check with the instrument that ran, what it observed (numbers), the counting rule, the verdict and a note; refined and drifted checks come with the text edits. One receipt per claim; the uniqueness rule catches copy-paste, the discipline catches the rest. When every claim carries a check at version X, bump `verified_claude_code` and the README badge to X in the same commit, then triage: every `drifted` claim is fixed or retired before the release commit. An audit is the moment to add claims for behaviors the previous round missed (the critic's gap list).
- *Closing an open leg*: run the instrument the leg names (trace the writer in the binary, or capture the specimen), append a check whose `observed` carries the receipt, delete that leg from `open_legs`, and raise `attribution` in the same edit; never raise it on wording alone.
- *Retiring a claim*: never delete silently. Keep the entry, add a check whose note names the successor claim (or the removed feature), and drop the code sites the gate can no longer find.
- *Privacy*: no session ids, no external or user paths, no person or product names, no reference to any implementation other than the shipped Claude Code binary; instruments cite synthetic fixture shapes and corpus COUNTS, never transcript content.
- *Reading it*: `jq -r '.claims[] | select(.area=="live-tail") | "\(.id) \(.behavior)"' INTROSPECTION.json`; claim ids are stable handles for commit messages, CHANGELOG bullets and SPEC ledger entries.
---
## 8. Conventions
- `PascalCase` types, `snake_case` items, one module per concern.
- **Structure limits are law (husky-enforced)**: <=600 lines per `.rs` file (200-400 is the sweet spot; smaller only when natural), <=20 `.rs` files per folder, <=16 subfolders per folder, <=5 subfolder levels below `src`/`tests`, per-function cognitive complexity <=25 (clippy.toml). Markdown is exempt. Split BEFORE the gate forces you to.
- Comments capture WHY / a non-obvious constraint, not what the code already says.
- **Commit messages follow Conventional Commits**: `type(scope): subject` — types in use: `feat`, `fix`, `perf`, `refactor`, `docs`, `test`, `chore`, `style`; `!` after the type/scope marks a BREAKING surface change (pairs with the §7 version policy: pre-1.0, breaking ⇒ MINOR). Subject in the imperative mood; the body says what changed and why in plain engineering terms; messages are pure ASCII.
- `Cargo.toml` lints: `unsafe_code = "deny"`, `missing_debug_implementations = "warn"`, clippy `all = warn` with a few pedantic allows. Deps pinned by `^`-range + `Cargo.lock`.
- Output discipline: errors -> stderr (`csift: error:` + full chain); data -> stdout; `--format json` is the machine contract (one object per line where it makes sense).
- ASCII-only source/docs unless a test fixture intentionally exercises multi-byte (then keep it neutral: emoji / accented Latin). Comments in `.rs` files are strictly ASCII: no em dash, use `-` (gate-enforced); deliberate output glyphs and fixtures in string literals are the only non-ASCII.
- README/promotional prose: em dashes rare (single digits for the whole page; commas, colons, parentheses do the work), sentence lengths mixed, bullet skeletons varied - not every bullet a bolded lead + balanced elaboration. One punchy summary line per document, not per section.
---
## 9. What NOT to do
- **Don't introduce BM25 / embeddings / semantic search.** Regex only.
- **Don't `unwrap`/`expect` in library paths**, and **don't silently truncate** - always report the drop count.
- **Don't parse a whole 200MB file** when a head/tail read answers the question; don't lose mmap+memchr+prefilter+tail+rayon.
- **Don't add a second `unsafe` allow** beyond the audited mmap site; don't switch `deny` to `forbid` (it would refuse to compile mmap).
- **Don't trust most-recent-mtime for `whoami`**; don't guess when `$CLAUDE_CODE_SESSION_ID` is absent.
- **Don't reverse a path encoding** (lossy); don't assume a workflow subagent's env `$CLAUDE_CODE_SESSION_ID` is its own id (it is the parent's).
- **Don't blindly trust this doc's field list** - real jsonl evolves; re-verify against `~/.claude/projects` and EXTEND the model tolerantly rather than tightening it into a crash.
- **Don't bump a dependency major** without an explicit reason.
- **Don't edit `.git/hooks/pre-commit` directly** - edit `.cargo-husky/hooks/pre-commit` and re-run `cargo test`.
- **Don't add semantic/numeric exit codes** - 0 / non-zero only, full chain to stderr; the single exception is `wait`'s documented timeout code 124, never to be extended.
- **Don't ship a SURFACE change (a flag, output, JSON field, or `--help` text) without bumping the version triple + tagging + a CHANGELOG.md entry** - `Cargo.toml` ≡ `SKILL.md` surface header ≡ `csift --version` must move together, every release is tagged `vX.Y.Z`, and the version's CHANGELOG.md digest entry rides the release commit (§7 versioning discipline). A changed binary under an unchanged version defeats the SKILL's staleness guard.
- **Don't move the README's "verified against Claude Code" badge without the ledger audit** - `INTROSPECTION.json` must carry a per-claim check at that version first (§7), written claim by claim with the instrument that ran; never bulk-replace the check records, never edit a check's `observed` to make the gate pass.
- **Don't edit `CLAUDE.md`** - it's a symlink; edit `AGENTS.md`.
```