# AGENTS.md - csift operating manual
Project-specific operating manual for any AI agent (Claude Code, Codex, Cursor) working in this repo. Read this first when a conversation opens. This file is sufficient for **repo mechanics + invariants** - directory map, dispatch wiring, test harness, the rules you must not break - so a fresh mid-size model can add/modify a subcommand or fix a bug respecting every invariant without spelunking the source. For the exact **behavioural contract** of a subcommand (precise flag semantics, output shape, edge cases), open the matching `SPEC.md` section 6 subsection; SPEC is the authority on *what* each command does, this file on *how* to work in the repo.
> **About this file.** `AGENTS.md` is the canonical, vendor-neutral filename. `CLAUDE.md` is a **symlink to `AGENTS.md`** - edit `AGENTS.md` only; the symlink follows. **Companion doc:** [`SPEC.md`](./SPEC.md) (~175KB / ~1240 lines) is the product/behaviour spec - open it whenever you change search/verbatim/recover/etc. behaviour. Concrete pointers: SPEC **section 4** = record semantics, **section 5** = the full label taxonomy (`search -t` role.class.sub), **section 6** = per-subcommand specifications (flags, output shape, edge cases), **section 7** = the performance contract, **section 8.2** = the JSON envelope v2; `## 6 Subcommand specifications` opens with the per-version CHANGE LEDGERS, ordered newest-first (the newest entry is authoritative; historical entries describe their own versions' surfaces). This file = how to work in the repo; SPEC.md = what to build. **The FIVE-DOCUMENT CONTRACT** (user-ruled, v0.5.1): `SKILL.md` = the LLM manual (density + weak-attention retrievability) · `--help` = the human (CLI-proficient) manual — INFORMATION-PARITY with SKILL, never thinner, but plain prose with stronger structure/layout (help IS surface: changing it bumps the version triple) · `README.md` = promotion · `SPEC.md` = design intent · this file = maintenance. When code and a doc disagree, **code wins** - re-verify, then fix the doc.
---
## 1. What csift is
**csift - "ripgrep for Claude Code session transcripts".** A fast Rust CLI that **lists** and **regex-searches** Claude Code session `.jsonl` files under `~/.claude/projects/`, plus recovers files, reconstructs compaction-clipped turns, maps subagents, and extracts pasted images.
- **Primary consumer is an LLM** - a Claude Code agent searching/recovering its own or a peer session. Output is clean, token-efficient, regex-driven. Default output is human/LLM-readable (session/turn/label/timestamp headers); `--format json` is the machine format (there is no bare `--json` flag). `search --raw` / `show --raw` emit VERBATIM jsonl lines (the escape hatch for unrendered fields).
- **Explicitly NO BM25 / embeddings / semantic search.** Pure regex only. Lexical tokenisation across scripts (CJK / multi-byte) is intractable for scoring; regex is the strength and the whole point. Never add semantic search.
- **Thirteen subcommands** (`Command` enum, `cli.rs`): `list`, `search`, `show`, `stats`, `agents`, `whoami`, `files`, `recover`, `plan`, `verbatim`, `image`, `status`, `wait` (+ ONE hidden `Turns` tombstone variant, v0.6.4 — never runs, always bails with the rename pointer to `verbatim`; keep it bare + hidden). `show` = record FETCH (by `--line`/`--uuid`/`--turn`, one transcript, rendered full or `--raw` verbatim bytes — search no longer fetches; `--turn N|A..B` fetches EVERY record of the named turn(s) in the SAME `·tN` numbering `search`'s `<tok>·tN` headers print (v0.7.0: `<tok>` = a STABLE first-8 id-prefix token, an `@` target as-is; the `sN` legend/ordinals are GONE), so `show --turn -3..` is the tail-peek/monitoring path — "read a session's recent turns from the live transcript"); `verbatim` = the compaction-fidelity specialist (reconstruct the verbatim user/assistant turns a compaction summary clipped — the command FORMERLY named `turns`, a zero-BC rename: `csift turns …` is now an unrecognized subcommand; the module file stays `src/turns.rs`, handler `turns::run_verbatim`, args `cli::VerbatimArgs`, variant `Command::Verbatim`); `stats` = one-scan per-session aggregates (tokens by model / tools / turns / span / compactions / a whole-file line-type census). (There is NO `pending` subcommand — the elicitation sidecar it once read is merged TRANSPARENTLY into every record-reading surface; see §3.10.) `status`/`wait` (v0.9.0) are the LIVE-TRUTH pair — point-in-time, explicitly non-reproducible, the one documented departure from the forensic contract: `status` joins the harness session registry (`<claude-home>/sessions/<pid>.json`, transition-writes only) + the transcript tail state machine (unreturned tool call = in flight) + a `ps`-based pid probe (reuse-guarded via procStart, which renders UTC vs `ps lstart` local) into one of six verdicts with named evidence; `wait` polls with STRICT post-start baseline semantics and exits 124 on timeout (the one exit-code exception).
- **Subagent spanning default:** `list`/`search`/`stats`/`files`/`recover`/`plan`/`image` span each session's subagent transcripts by default. EVERY span command answers the SAME two switches: `--no-subagents` (restrict) and `--subagents` (affirm — a no-op on the default-on commands, kept so the pair is uniform; the contradictory pair is a clap `conflicts_with` parse error). `verbatim` is the exception (its per-session budget MULTIPLIES, so it defaults to top-level thread only; `--subagents` opts IN — the field is named `include_subagents` but the FLAG is `--subagents` — and a target is REQUIRED). `agents` LISTS subagents as its targets, so it hard-rejects BOTH span flags with a pointed error (its `--subagents` twin is a hidden no-op that exists only to be rejected). EVERY default-on `*Args` exposes `want_subagents() -> bool` (= `self.subagents || !self.no_subagents`) feeding `SubagentScope::from` (`true` -> `WithSubagents`, `false` -> `TopLevelOnly`); `verbatim` returns `self.include_subagents && !self.no_subagents`. `SubagentScope` has exactly those two variants. (There is no `--subagents-only` mode.)
---
## 2. Targeting grammar (@-tokens) - shared resolver `path::resolve_session_files`
Every session-operating subcommand resolves its target(s) through ONE function:
`path::resolve_session_files(paths: &[PathBuf], scope: SubagentScope, caller: Caller) -> Result<Vec<PathBuf>>` (src/path/). It returns the concrete jsonl files to scan. Multi-target commands actually call the thin wrapper `path::resolve_targets_with_session_list` (src/path/), which first UNIONs in the `--sessions-from <FILE|->` id list (`extend_with_session_list`, src/path/ — bare or `@`-prefixed uuid/prefix/agent-id tokens, each appended as an `@<id>` positional; an EXPLICIT empty list resolves to an EMPTY scope, never all-projects). `whoami` is the only subcommand with no target (reads `$CLAUDE_CODE_SESSION_ID`).
Each positional token (for `search` the FIRST positional is PATTERN, sessions come as later `@<uuid>` positionals):
- **real cwd path** / `.` -> encode it, locate the matching `~/.claude/projects/<encoded>` dir.
- **encoded dir token** (`-Users-...`, or the Windows drive shape `C--Users-...`) -> used directly; a bare `--`-leading token is rejected as a mistyped flag by `parse_project_target` (src/cli/ — a UNC-encoded `--server-...` dir is targeted via `@--server-...`); a drive-shaped token that matches no projects dir falls through to real-path resolution (it can be a genuine relative path).
- **`~/.claude/projects/<encoded>` path** -> used as-is.
- **`@<uuid>`** -> that one top-level session (`is_uuid`).
- **`@<uuid-prefix>`** e.g. `@13d9645a` -> the leading hex of a uuid (4..=11 dashless chars); unique-resolved against enumerated sessions, else ambiguity error (`is_uuid_prefix`).
- **`@main`** -> the CALLING TOP-LEVEL session, read from `$CLAUDE_CODE_SESSION_ID` (`resolve_env_session`).
- **`@trap:<marker>`** -> the CALLING SUBAGENT (or main thread). CC withholds a subagent's OWN id from its Bash env, so a running subagent cannot name itself. The caller INVENTS a unique literal marker, embeds it in *this very* csift command; csift finds the transcript whose SHELL `tool_use` carries it (`resolve_trap` — `Bash`, or Windows' separate `PowerShell` tool, v0.7.4). STRICT marker grammar (`validate_trap_marker`, src/path/): EXACTLY 3 CamelCase words (no single-letter / ALLCAPS like `HTML`/`USB`) + exactly 4 non-trivial trailing digits, hand-invented one-shot + CONTEXT-INDEPENDENT, shaped like `@trap:JollyShinyBrook4283` - a RESERVED example csift hard-refuses (don't copy it; a pasted example self-collides, so invent your own). Routes on EVERY target-taking subcommand via the shared `resolve_session_files`, AND on `whoami` (`whoami @trap:<marker>` -> the subagent's full UPSTREAM ancestry chain self -> ... -> top-level root, the walk-UP mirror of `agents`' walk-DOWN; env-independent; `path::resolve_trap_who` reuses `agents::topology_for_session` + `parent_agent_id`). Malformed/ambiguous -> hard bail with guidance. **TIMING (subagent verified live 2026-07-12; main-lane mechanism re-measured 2026-08-29):** a SUBAGENT's transcript flushes per content block, so the launching tool_use is on disk at dispatch and a first try resolves. The MAIN conversation's record is an ASYNC FLUSH of the completed assistant message, landing ~1-3.4s AFTER dispatch - a RACE, not a wait (the 2026-07-12 receipt, a 3s in-command sleep seeing 0 on disk, sat exactly at the window's edge and had no resolving power; a 263s command was later observed with its unpaired tool_use on disk 39s in). csift finishes inside that window, so a top-level FIRST use normally misses and a re-run of the SAME marker resolves - the no-match error routes both paths, `@main` FIRST (the main thread's direct answer, no race), then the literal-marker check, then retry-as-lane-confirmation (a NEW, SEPARATE invocation: a second attempt inside ONE script runs in that same unlanded window, whose width is invisible from inside; a FRESH marker restarts the race). A @trap that RESOLVES to the main transcript prints a stderr lane note (stdout unchanged, exit 0). The same flush window bites a UserPromptSubmit hook asking for the PREVIOUS prompt: at that instant the CURRENT prompt's record may or may not be on disk yet (both observed live), so a hook-side "previous prompt" query excludes hits younger than now-3s (the measured window is ~1-3.4s) and takes the newest survivor.
- **`@<agent-id>`** -> a subagent + its TOPOLOGICAL subtree (unless `--no-subagents`). Accepts EITHER a bare hex (built-in/workflow, len >=12, `is_bare_subagent_hex`) OR a name-embedded **teammate** id like `aVSRepro-68a2a1661c9390c1` (`is_teammate_agent_id`, path.rs). The shared gate is `path::is_subagent_id` (= `is_bare_subagent_hex || is_teammate_agent_id`); use it (NOT `is_bare_subagent_hex`) for any new `@<id>` / pin routing. EVERY id `csift agents` prints round-trips here regardless of shape — a teammate id used to fall through to the project-dir branch and fail; the `is_subagent_id` gate (src/path/) via `pins_single_session` (src/path/) is what fixed it, and the teammate arm tolerates dashes in the NAME (`aP1-engine-9cf2f06d6235ca64` is real data — the head accepts `[A-Za-z0-9-]` with an explicit `!is_uuid` guard so an `a`-led uuid can't slip in). Downstream `resolve_agent_subtree` matches a subagent by EXACT `session_id_from_path`, so it already handled non-hex ids; only the entry gate needed widening.
- **`*.jsonl` file** -> one transcript; a path containing a `subagents` component is treated as a subagent (+ subtree), else a top-level session.
- **bare uuid (no `@`)** is NOT special - it falls to the path branch and fails as "no project dir named <uuid>". Always prefix with `@`.
- **unrecognized `@`-shape** (a 1-3-char hex prefix, a dashed fragment, any non-id token not leading with `-`) -> HARD ERROR naming the @-grammar (the `resolve_session_files` catch-all arm, v0.6.1) - it never falls through to path resolution; only a `-`-leading token OR the Windows drive shape (`<letter>--...`, v0.7.3) is treated as an encoded project-dir name (`@-Users-...` and `@C--Users-...` both work). A 1-3-char hex token gets the dedicated "too short for a session-uuid prefix - needs 4-11" message.
Resolution is **fail-closed/fail-loud**: an id that was pinned but matches no file bails (never a silent empty). `0 paths` => every project under the root. `Caller` is a subcommand discriminator threaded through for FUTURE subcommand-aware remediation text - but it is currently **INERT** inside `resolve_session_files` (`let _ = caller;`, src/path/); pass the right variant for forward-compat, but no remediation text varies by it today. `SubagentScope` (src/path/) has exactly two variants: `WithSubagents` (default) / `TopLevelOnly`; `SubagentScope::from(bool)` maps `true->WithSubagents`, `false->TopLevelOnly`. (The former `SubagentsOnly` variant + `files`' `--subagents-only` flag were both removed in the crate-wide span-flag cleanup.)
---
## 3. The Claude Code jsonl domain model (load-bearing - verified against real data)
### 3.1 Data location + path encoding
```
~/.claude/projects/<ENCODED_CWD>/<session-uuid>.jsonl # a session transcript
~/.claude/projects/<ENCODED>/<uuid>/subagents/agent-<hex>.jsonl # (A) built-in Task/Agent subagent
~/.claude/projects/<ENCODED>/<uuid>/subagents/workflows/wf_<id>/agent-<hex>.jsonl # (B) workflow / OMC subagent (dominant)
~/.claude/projects/<ENCODED>/<uuid>/subagents/workflows/wf_<id>/journal.jsonl # (C) workflow EVENT log - NOT a transcript (excluded)
~/.claude/projects/<ENCODED>/<uuid>/subagents/**/*.meta.json # {agentType, description?} companions
~/.claude/projects/<ENCODED>/<uuid>/tool-results/<id>.txt # externalised tool output
```
**Encoding** (deterministic forward, lossy reverse; CC-exact since v0.7.3, extracted from the 2.1.228 binary): the cwd is NFC-normalized, then every UTF-16 CODE UNIT outside `[A-Za-z0-9]` becomes a single `-` (an astral char = two surrogate units = TWO dashes). NO consecutive-dash collapsing; `.`,`/`,`_`,space all -> `-`. E.g. `/Users/u/Projects/w_app` -> `-Users-u-Projects-w-app`; a `/.claude/` segment -> `--claude-`; WINDOWS `C:\Users\x` -> `C--Users-x` (letter-led!) and UNC `\\server\share` -> `--server-share`. Reverse is lossy (`-` could be `/`,`_`,`.`) so we NEVER reverse. Accept either a real fs path (encode + locate) or a direct encoded dir. **>200-char cap:** CC caps the encoded name at `MAX_SANITIZED_LENGTH = 200` (src/path/); a longer cwd is stored `<first-200>-<hash>`. The hash is NOT reconstructible (CLI uses Bun.hash, SDK djb2 - different digests), so a long path is resolved by **prefix-scanning** the projects root for `<first-200>-` (src/path/, `find_dir_by_prefix`); among collisions, disambiguate by the in-record `cwd` (authoritative).
### 3.2 Record model (one JSON object per line)
Top-level fields used: `type`, `uuid`, `parentUuid`, `timestamp` (ISO8601 UTC), `sessionId`, `cwd`, `version`, `gitBranch`, `isSidechain`, `userType`, `message`, + `subtype`/`content` on system records, `isCompactSummary` on compaction summaries. Many more exist and are IGNORED by the scanning surfaces (`attachment`, `file-history-snapshot`, `queue-operation`, `isMeta`, `sourceToolAssistantUUID`, `slug`, `promptId`, ...) - `toolUseResult` is NOT ignored: it is kept UNPARSED (`Box<RawValue>`, section 4 perf) and deep-read by `files`/`recover` (create-vs-edit, content anchors, the section 3.11 freshness signals) — with ONE opt-in exception: a `type:"attachment"` record whose payload is `{"type":"hook_additional_context","content":[...]}` (the context a SessionStart/UserPromptSubmit/... hook injected) classifies `harness.meta.hook` and is scanned by `search --additional-context` (default OFF — a default scan never parses attachment lines; the candidate needle is `&&`-gated like the D7 boundary keep) and rendered by an explicit `show --line`/`--uuid` address flag-free (the refetch law). `Record::hook_additional_context_text` (model.rs) is the one extractor — `content` is a string ARRAY joined with `\n` (bare string tolerated).
`type` values: `user`, `assistant`, `system`, plus metadata-only `last-prompt`/`ai-title`/`agent-name`/`mode`/`permission-mode`/`attachment`/`file-history-snapshot`/`queue-operation` (these often have **no `timestamp`** - skip in time logic, never crash).
**Block types** (`Block` enum, model.rs): `{type:text,text}`, `{type:thinking,thinking,signature?}`, `{type:tool_use,id,name,input}`, `{type:tool_result,tool_use_id,content,is_error?}`, `{type:image,source}`. `tool_result.content` may be a string OR an array of `{type:text,text}`/`{type:image}`. The enum has a `#[serde(other)] Unknown` arm - new/unknown block types parse, never crash.
### 3.3 Turn boundary = `opens_turn` (4 cases), NOT just `is_genuine_user`; classification is a SEPARATE axis
A `type:"user"` record's `message.content` is EITHER a string (genuine text, older format) OR an array of blocks. **CRUCIAL: a "user" record is NOT always a human turn** - `tool_result` blocks ride on `role:user` records too. Real ratio in one session: ~393 genuine vs ~1619 tool_result-carriers. `model::Record::is_genuine_user` (src/model/) returns true iff: `type:"user"` AND `role=="user"` AND `isCompactSummary` falsey AND `isMeta` falsey AND (content is a string OR blocks contain a `text` block and NO `tool_result` block) AND the string is NOT a synthetic marker AND **NOT `is_peer_message`** (the GOLD §1 + FINDING-2 fix - an inbound `<teammate-message>` OR `<agent-message>` is `type:user`/`role:user`/string and matches no synthetic marker, so it used to slip through as the human; 106 peer messages mislabeled in one real session). **FINDING-1:** peer/notification tags are detected ONLY at a SECTION BOUNDARY (`is_section_boundary` - content-start, just after the relay preamble `Another Claude session sent a message:`, or right after a prior section's close tag), so a genuine message that merely QUOTES `<teammate-message>`/`<task-notification>` mid-prose (common in csift's OWN dev sessions) stays `user.message`. A tool_result-carrier is NOT genuine.
**But `is_genuine_user` is NOT the turn delimiter.** The single boundary predicate every surface keys on is `model::Record::opens_turn` (src/model/):
```
opens_turn() = is_genuine_user() OR is_auq_answer_boundary() OR is_plan_rejection_boundary() OR is_peer_message_record()
```
A turn opens on ANY of four cases (using ONLY `is_genuine_user` drops three of them):
1. **genuine human message** (`is_genuine_user`).
2. **answered AskUserQuestion** (`is_auq_answer_boundary`, src/model/) - the user's ANSWER is their message; rides on a non-errored `tool_result` carrier (a cancelled / `is_error:true` / `Cancelled...` / `<tool_use_error>` AUQ is NOT a boundary).
3. **tool-use rejection-with-message** (`is_plan_rejection_boundary`, src/model/) - a rejection carrying a typed `To tell you how to proceed, the user said:\n` tail; a bare rejection (no typed tail) is NOT a boundary.
4. **inbound peer message** (`is_peer_message_record`) - a `<teammate-message>` OR `<agent-message>` (FINDING-2 folds the agent-message peer form in) from another session, at a section boundary: a real delivered message, so it STAYS a turn-opener (segmentation / turn-COUNT stays byte-stable where peers already opened turns - a non-isMeta teammate/agent message; an isMeta `<agent-message>` newly opens one), but `is_genuine_user` excludes it so it never counts as the human. Body render comes from `inbound_comm_preview` (`verbatim`/`list`) / `record_text_sections` (`search`), never the raw XML.
**Opening a turn ≠ the `-t` label.** `opens_turn` segments the transcript; `Record::classify` (the role.class.sub engine, §3.3a) is a SEPARATE axis that assigns the searchable label(s). A record that opens a turn classifies as `user.message` (genuine), `user.answer` / `user.rejection` (the AUQ/rejection duals), `agent.communication.inbox` (a teammate-message), or `harness.notification.<kind>` (a `<task-notification>`) - the opener body still renders via `reconstructed_user_text`, but its `-t` bucket follows `classify`, not `opens_turn`.
**EXCLUDED from opening a turn** (machine pseudo-turns that LOOK human - never delimiters, dropped/folded as turn MEMBERS): `isMeta` records; the two interrupt markers (`[Request interrupted by user]` / `...for tool use]`); `<local-command-stdout>...`; slash-command wrappers — `is_slash_command_wrapper` detects BOTH tag orders, a leading `<command-name>...` OR the current-CC `<command-message>...` (the new `COMMAND_MESSAGE_PREFIX`; verified 14 new-order vs 35 old-order sessions), and `is_genuine_user` now excludes BOTH so a new-order wrapper no longer masquerades as the human or opens a turn (a v0.5 correctness fix — turn NUMBERING may shift on transcripts carrying new-order wrappers); compaction summaries (`isCompactSummary`). **Slash-wrapper classify:** a WITH-args wrapper carries BOTH labels `[user.message, harness.command.invocation]` with `user.message` FIRST (richest-view law), so the UNFILTERED render is the extracted `/name args` (`slash_command_name`), never the wrapper XML; a NO-args wrapper → `harness.command.invocation` ONLY; an explicit `-t harness.command.invocation` still renders the raw wrapper. (SPEC §4.2.3 is the authority.)
**`<task-notification>...` automation pulse** (src/model/) passes every `is_genuine_user` gate so it DOES open a turn, but is a machine trigger, not operator prose - so `classify` buckets it under **`harness.notification.<kind>`** (NOT `user`; the GOLD §1 reparent), `<kind>` = one of FIVE `AutomationKind` slugs `background-command`/`workflow`/`subagent`/`monitor`/`task` (the role-`agent`-colliding `agent` slug renders as the `subagent` leaf). Render it via `Record::automation_label()` -> `[<kind> <task-id> <status>] <summary>` (parsed from the summary, src/model/; status falls back to `<event>` then `completed`) - NEVER dump the raw `<task-id>`/`<output-file>`/`<status>` XML. `monitor` is the Monitor/ScheduleWakeup cadence pulse (from_summary src/model//1676; for it the status slot renders the `<event>`, NOT a fabricated `completed`, src/model/ - directly relevant to monitoring/cron transcripts). A pulse carrying a `<result>` is a background-agent REPORT, so it ALSO carries `agent.communication.inbox` (child ⇨ self, G1). **Standing caution: these section-3 enumerations can LAG the enum - verify the live set against `AutomationKind::slug` (src/model/) before trusting a fixed list.**
**Render + group via the shared helpers, never hand-roll:** `Record::reconstructed_user_text(plan_index)` (src/model/) yields the normalized opener body for ALL three cases (genuine text / full AUQ Q+options+answer unit / rejection text + optional `[plan: <path>]` pointer when `plan_index` resolves the rejected id; `None` = does not open a turn). Delimit with `group_turn_indices_deduped` (src/model/, production - drops esc-cancel/edit-resend superseded-draft openers via shared `parentUuid`) or the bare `group_turn_indices` (test-only bool-fixture base). Both `search` exchange reconstruction and `files` mutation attribution route through these so they never drift. C-18 (v0.8.2): the superseded-draft collapse is DISCLOSED (search footer `N superseded draft(s) collapsed` + JSON summary `superseded_drafts`) and an explicitly ADDRESSED draft still fetches via `show --line`/`--uuid` (rendered annotated, outside turn numbering, JSON `superseded_draft:true` + null `turn_index`) - the collapse is turn hygiene, never a silent drop.
### 3.3a The `role.class.sub` classification (model.rs `Role`/`Class`/`classify`; SPEC §5 is the authority)
`Record::classify(ctx) -> Vec<Class>` (src/model/) is the engine `search -t` keys on. It is **multi-label** (one physical record can carry >1 leaf), pure, tolerant (an unmodeled record → empty `Vec`, never a crash). Three ROLES, **26 leaf classes** (`Class::ALL`, src/model/ - the single source of truth; a drift-guard test pins the count):
```
user (the human) user.message · user.answer · user.rejection
agent (the assistant) agent.message · agent.thinking · agent.tool.use · agent.tool.result
communication agent.communication.{inbox, sent, signal}
harness (CC machinery) harness.notification.{workflow, monitor, subagent, background-command, task}
harness.compaction.{summary, boundary}
harness.command.{invocation, stdout}
harness.interrupt.{user, tool}
harness.schedule.{wakeup, continuation}
harness.meta.{hook, loop, attachment}
```
- **`-t` selector = a dotted path matched by dot-SEGMENT prefix** (cli.rs `category_selectors`/`label_selected`, derived from `Class::ALL`): `-t agent` ⇒ the whole agent role, `-t agent.tool` ⇒ use+result, `-t agent.communication` ⇒ all three comm leaves, a full leaf path ⇒ just it. NO `-t` ⇒ every label. The **old flat values** `thinking`/`tool`/`tool-response` HARD-error (0 back-compat, like `--full`) — since v0.6.3 the error names the successor path (`agent.thinking`/`agent.tool`/`agent.tool.result`) ahead of the 25-value list; `user`/`agent` keep working as ROLE selectors.
- **Gated meta leaves**: `harness.meta.hook` is scanned only under `search --additional-context` (or the superset `--attachments`); `harness.meta.attachment` (any OTHER attachment payload; matchable text = the VERBATIM payload JSON, a byte substring of the raw line so the prefilter laws hold with no synth markers) only under `--attachments` / `--count-by attachment` - a default scan never parses attachment lines; an explicit `show --line`/`--uuid` address renders any of them flag-free (the refetch law). Both keeps are `&&`-gated candidate needles like the D7 boundary keep (`needs_hook_context` / `needs_attachments`, scan.rs).
- **Multi-label + Q4 dedup**: a record selected by ≥2 of its labels emits ONCE, richest view (AUQ answer → `user.answer` not `agent.tool.result`; a `SendMessage`/spawn → the `agent.communication.*` view not `agent.tool.use`; a `<result>` pulse / subagent return → the comm view). JSON carries the matched leaf as `label` + the full set as `labels[]`.
- **Direction (`from ⇨ to`)** on every `agent.communication.*` hit (`Record::direction`, src/model/): teammate-message `teammate_id ⇨ self`; `SendMessage` `self ⇨ input.to`; spawn `self ⇨ child`; subagent opener `parent ⇨ self`; subagent/`<result>` return `child ⇨ self`. The owner's own id renders as the literal `self`.
- **Render markers**: a ROLE glyph leads each hit line - **`◂` user · `▸` agent · `⚙` harness** (gear = machinery); `⇨` (hollow arrow) = the comm direction; `▹` = ONLY the `agent.tool.use ▹ agent.tool.result` pairing (joined by `tool_use_id`; an unreturned use → `(no result — pending)`, an orphan result → `(use not in scope)`).
- **Notable mappings**: `redacted_thinking` block → `agent.thinking` (opaque, renders `[redacted thinking]`); a pending elicitation sidecar marker (AUQ/ExitPlanMode/MCP) → `agent.tool.use`; an `isMeta` record that matches no harness marker is **EXCLUDED** (empty - never `user.message`); a `<task-notification>` → `harness.notification.<kind>` (§3.3).
- **`search`-surfacing note**: `classify` runs on every record. `search`'s §7 stage-1 prefilter (`line_is_transcript_candidate`) keeps `role:user`/`role:assistant` lines AND — since D7 — the rare `compact_boundary` `type:"system"` record — but the boundary keep is itself **`-t`-GATED**: a `needs_compact_boundary` bool (`label_selected(&args.categories, "harness.compaction.boundary")`) is derived ONCE before the scan, and the extra `memmem` is `&&`-gated behind it, so a `-t user`/`-t agent.*`/any non-boundary query pays ZERO (the `memmem` is never reached); only no-`-t` (match-all) or a selector reaching `harness.compaction.boundary` (`harness` / `harness.compaction` / `harness.compaction.boundary`) keeps it. The role keeps ALWAYS run (a conservative superset). **R13 needle law:** the role keeps are the SHARED serialization-tolerant matchers `parse::line_has_role_marker` / `line_has_user_role_marker` (JSON whitespace around the colon is the SAME record — the old exact compact bytes `"role":"user"` silently dropped a reserialized line, e.g. python `json.dumps` defaults, one layer BEFORE any malformed counter could see it: no preview, no count, no match, zero disclosure; found independently by R12's own fixture accident and R13's witness). Every OTHER prefilter needle in the tree is serialization-safe by construction (a bare value substring like `tool_use`/`plan_mode`/`compact_boundary`, or a key-only `"media_type"`/`"filePath"` form). When adding a NEW candidate needle: value-substring or key-only, or route it through a tolerant matcher — NEVER a raw compact `"key":"value"` byte pair for candidate SELECTION (a compact pair is fine for a redundant belt where a tolerant hook already covers the record). When it IS run, the `||` chain reaches the `memmem` only on lines that already failed both role checks, and boundaries are rare — so the §7 perf contract holds. So `search -t harness.compaction.boundary` IS surfaced: for a message-less system record `record_raw_text` falls back to the boundary's top-level `content` PLUS a readable `compactMetadata` excerpt (`[compaction boundary: trigger=… preTokens=… postTokens=… durationMs=…]`) as the match + excerpt, so compaction points are enumerable + inspectable. Its sibling `harness.compaction.summary` (a `type:"user"` record) is searchable too — so every leaf is now search-reachable (the boundary was the last exception).
### 3.4 AskUserQuestion
A `tool_use` block with `name=="AskUserQuestion"`. **HARD-WON: a PENDING/unanswered AskUserQuestion is NEVER flushed to jsonl** - only answered ones appear, with the answer arriving as a later user/tool_result record. So an option-picker freeze looks identical to a stall on disk. The user's answer to an AskUserQuestion classifies as **`user.answer`** (dual-labeled with `agent.tool.result` since it rides on the answering tool_result carrier; deduped to the richer `user.answer`, §3.3a).
### 3.5 Compaction shape
The summary is a `type:"user"` record with `isCompactSummary:true` + `isVisibleInTranscriptOnly:true` carrying **string** content (NOT a `type:"summary"` record). A separate `type:"system"` `subtype:"compact_boundary"` record carries `compactMetadata:{trigger,preTokens,postTokens,durationMs}` PLUS a top-level `logicalParentUuid` (the true predecessor record the compaction re-links to; the boundary's own `parentUuid` is null) - surfaced in the boundary's rendered excerpt as `[logicalParent=<uuid>]`. A compaction summary MUST be excluded from "genuine user". `verbatim` reconstructs the verbatim exchange a summary clipped.
### 3.6 Externalised output
Large tool outputs are moved to a sibling `tool-results/<id>.txt`, leaving an inline `<persisted-output>` pointer (absolute path + preview). `search --resolve-persisted` replaces the pointer with full file content BEFORE matching: prefers structured `toolUseResult.persistedOutputPath` (src/model/), falls back to scraping `Full output saved to: <ABS_PATH>`. Read failure is non-fatal - keeps inline + appends `[csift: could not resolve persisted output ...]`.
### 3.7 Subagent on-disk shapes, flat nesting, topology
Three on-disk shapes: (A) built-in Task/Agent under `subagents/agent-<hex>.jsonl`; (B) workflow/OMC under `subagents/workflows/wf_*/agent-<hex>.jsonl`; (C) `journal.jsonl` event log = NOT a transcript (read only for completion status, never listed/searched). **Kind = on-disk location, NOT `agentType`** — `BuiltinTask` vs `Workflow` (subagent.rs) — **with ONE meta-driven exception: `Teammate`.** Canonical agent id = bare `<hex>` (= the record/journal `agentId`) for A/B. The parent uuid is the dir segment above `subagents/`.
**Teammate subagents (the `in_process_teammate` kind, the new Claude Code "teammates"/FleetView feature).** A teammate lands at the BUILT-IN location (`subagents/agent-<id>.jsonl`) so location alone can't classify it — the discriminator is its `meta.json` `taskKind:"in_process_teammate"` (`make_subagent` upgrades `BuiltinTask` -> `Teammate` when it sees that). Distinctive shape, all verified against real data: (1) the canonical id EMBEDS the teammate name — `aVSRepro-68a2a1661c9390c1` = `a<Name>-<16hex>`, NOT a bare hex (so `is_bare_subagent_hex` rejects it; `is_teammate_agent_id`/`is_subagent_id` accept it — see §2); (2) the meta carries `{name, taskKind, teamName, color, model, permissionMode, spawnDepth}` and the `agentType` is OVERLOADED with the teammate NAME (e.g. `"VSRepro"`), NOT the real subagent type; (3) the meta has **NO `toolUseId`** — the usual id-join to the spawning tool_use can't reach it; (4) it is spawned by an `Agent` tool_use whose `input.name` == the teammate name AND `input.subagent_type` == the REAL type (`oh-my-claudecode:qa-tester`); (5) the team lead talks to it via `<teammate-message teammate_id="team-lead">` blocks + the `SendMessage` tool (bidirectional, persistent — unlike a one-shot Task subagent). **Spawn linkage = a NAME-join, not an id-join** (`ParentSpawnIndex::by_name` / `spawn_id_for_name`, joined in `node_for`): the teammate's `meta.name` -> the `Agent` spawn's `input.name` recovers the spawn `tool_use_id`, the TRUE trigger ts, the parent agent, and the real `subagent_type` (which `node_for` then prefers over the meta's overloaded `agentType` ONLY for a teammate). The node also carries `name` + `team_name`. A teammate is a normal topology PARENT — children it spawns nest under it by the usual issuer link. `agents --shape teammate` filters to them. **Control hint (NOT a csift op — csift is read-only):** `agents` surfaces a pointer to the CORRECT control tool because a real session burned ~30 min trying to `TaskStop`/`pkill` a runaway teammate — feeding it the name, the `Name@team` form, AND the exact `aName-<hash>` agentId csift prints, all rejected (`No task found with ID`). The ids were CORRECT; the TOOL was wrong. A teammate is steered/terminated via `SendMessage` addressed BY NAME (`message:{type:"shutdown_request"}` terminates — verified from the live SendMessage schema), never `TaskStop` (it only resolves `run_in_background` `task_id`s) or `pkill` (in-process, shares the orchestrator PID). Surfaced as a text footer when ≥1 teammate is in scope (`agents.rs` `TEAMMATE_CONTROL_HINT_*` / `any_teammate`) and as each teammate node's JSON `control_hint`. Keep the wording FACT-led (in-process Agent subagent, addressed by name) so it survives a tool rename.
**Workflow RUN `status` is journal-verbatim (open set).** An `agents` `kind:"run"` row's `status` is read straight off the workflow `journal.jsonl` (`str_f("status")`, subagent.rs) — observed values include `completed` and `killed`; it is NOT a csift enum and is distinct from the per-agent csift-computed lifecycle status. Never tighten it into a closed set.
**Flat on disk, nesting reconstructed:** all subagent transcripts sit flat under one session; depth>1 nesting (an agent spawning an agent) is NOT encoded in the path. `build_topology(session_jsonl, with_files)` (src/subagent/) rebuilds the tree: build a GLOBAL spawn index (`build_global_spawn_index`, src/subagent/) by forward-scanning the main transcript PLUS every subagent transcript for `Task`/`Agent`/`Workflow` `tool_use` spawn records, keyed by `spawn_tool_use_id`; each child's `spawn_tool_use_id` (from its head record / meta) joins to its issuer -> `parent_agent_id`; `assign_depths` walks the id->parent chain (cycle-guarded). `SubagentNode` carries `agent_id`, `kind`, `parent_session_id`, `parent_agent_id`, `spawn_tool_use_id`, `spawn_tool`, `workflow_id`, `agent_type`, `description`, `trigger_utc` (true spawn = parent tool_use ts; falls back to child-head `started_utc`), `completed_utc`, `returned_message` (+`_source`), `status`, `files_changed`, `depth`, `children`. **Subagent env quirk:** a workflow `agent()` subagent's `$CLAUDE_CODE_SESSION_ID` holds the PARENT session uuid, not its own - and current CC hands an Agent-tool subagent the PARENT uuid too (verified live 2026-07-12; the subagent's own id is withheld from its env; older builds handed a built-in Task subagent its own id) - never assume env == self; `@trap` is the env-independent self-resolution.
**r5 id shape - EVERY emitted row carries `is_subagent` + `parent_session_id` (REQUIRED, not optional).** A subagent transcript's `session_id` is a bare `<hex>` - it is NOT a re-feedable `@<uuid>` target (`csift verbatim <hex>` fails). So `list`/`search`/`files`/`verbatim`/`recover` rows are a TRIO, not a lone id: `session_id` (display) + `is_subagent` (id-domain discriminator) + `parent_session_id` (the always-re-feedable owning uuid; == `session_id` for a top-level row). Since v0.6.5 every search HIT (and sibling) object inside an exchange row carries the trio too — bare `.hits[]` flattening keeps real ids (jq cannot fail loud on a missing key, so the data matches the natural access pattern). Emitting only the hex = unusable rows. **Derive all three with `crate::subagent` helpers - NEVER read the path stem yourself** (the stem is `agent-<hex>` -> reading it gives the wrong `agent-`-prefixed id): `session_id_from_path` (src/subagent/ - strips the `agent-` prefix to the bare-hex canonical `agentId`), `is_subagent_path` (src/subagent/ - true iff a `subagents/` path component), `parent_session_id_from_path` (src/subagent/ - the dir component just before `subagents/`; `None` for a top-level file, so callers `.unwrap_or_else(|| session_id.clone())`). To re-feed a subagent match downstream, use `parent_session_id`, never the bare hex. **envelope v2 (SPEC §8.2) is built ONLY through `text::envelope_header`/`envelope_scope_header`/`envelope_summary`** — every `--format json` stream is exactly ONE `{"kind":"header","command":…}` line + kind-tagged rows + ONE `{"kind":"summary"}` line; `kind` belongs to the envelope exclusively (a transcript's shape is `shape`, a boundary's classifier is `cause`), and the jsonl-line key is `line` everywhere. The row TRIO keys themselves are still hand-built in each module's `json!` projector — when adding a spanning JSON surface, emit the envelope through the text.rs builders and replicate all three trio keys, deriving values via the `crate::subagent` helpers above.
### 3.8 whoami detection
CC exports `$CLAUDE_CODE_SESSION_ID` into the Bash tool env == the session's own jsonl basename. Definitive, per-session, version-independent, zero false positives. Use it and nothing else. When absent/empty, **DO NOT GUESS** (concurrent sessions; most-recent-mtime is a false-positive trap) - error with guidance to pass `@<uuid>`. whoami saying "ambiguous" is acceptable. LANE HONESTY (v0.8.2): the env names the TOP-LEVEL session in EVERY lane, so the env form's `is_subagent`/`parent_session_id`/`depth` are NULL (unknowable, never fabricated), text carries a `lane unknown` line, stderr names the resolution path; every `@main` resolution prints an unconditional stderr lane note (csift cannot know the caller's lane). (`CODEX_COMPANION_SESSION_ID` mirrors it but is Codex-specific.)
### 3.9 Pending tool_use / "frozen lane" (a permission ESCALATION leaves NO jsonl trace)
A pending tool-approval lives ONLY in CC process memory - it is in NEITHER transcript NOR any control file. On disk a blocked lane looks like: the assistant's `tool_use` record (`stop_reason:"tool_use"`, the command in `message.content[].input.command`) is the LAST record, FROZEN, with NO following `tool_result` for its `tool_use_id` (the block carries only `{type,id,name,input,caller}` - no `permission`/`isEscalated`/`requires_approval`); minutes later the `tool_result` is appended once a human answers. So THREE states share ONE on-disk signature: ① escalation-blocked (waiting for a human to approve) ② awaiting-execution (a slow tool) ③ wedged/dead. **Worse, the OLD status logic mis-reported a frozen lane as `completed`** - the tail walk-back for an end-of-turn text found the assistant text that PRECEDES the frozen `tool_use` (the L629->L630 shape) and called it a clean finish. `lifecycle` now detects the frozen lane from the NEWEST meaningful record (an unreturned `tool_use` ⇒ blocked, NEVER `completed` - status forced `Running`) and classifies it: a Bash command CC's `dangerous-rm` classifier ([`bash_danger`]) would HOIST even under bypass ⇒ **escalation-blocked** (the one state jsonl can POSITIVELY confirm); otherwise **awaiting-execution** (slow OR wedged - jsonl can't tell these apart, so don't pretend; the caller weighs elapsed-since-`pending_since_utc`). Surfaced on `agents` nodes as `pending_tool_use_id`/`pending_tool_name`/`pending_classification`/`pending_since_utc`. On EVERY agent node `completed_utc/_local` (+`duration`) are non-null ONLY when `status==completed` (a frozen lane is never "done" — the JSON now matches the text tree's suppression), and `last_activity_utc/_local` carries the tail newest-record instant regardless of status (== `pending_since_*` when frozen); `--order-by completion` windows on that terminal instant, so frozen lanes window on their freeze instant. `bash_danger` is a 1:1 port of CC's `Ywa`/`egp`/`Zhp` (the dangerous-rm path is pure static regex - no LLM, no fs); it is PURELY LEXICAL (flags `rm $VAR/{*,$,/,",',end}` WITHOUT checking if `$VAR` is empty - mirror CC exactly, never "improve" it). Higher-fidelity live signal would need a pre-installed CC `Notification`/`PreToolUse` hook writing a sidecar marker; the heuristic is the zero-setup retroactive path.
**Windows shell reality (extracted from the CC 2.1.228 binary, v0.7.4):** on Windows CC ships a SEPARATE first-class tool literally named **`PowerShell`** (tool registry `["Bash","BashOutput","KillShell","PowerShell",…]`; same `input.command` field; "Executes a given PowerShell command with optional timeout…"). Selection: `CLAUDE_CODE_USE_POWERSHELL_TOOL` explicit override → else PowerShell is FORCED ON when no Git-for-Windows bash is found (detection: `CLAUDE_CODE_GIT_BASH_PATH` override → `C:\Program Files\Git\bin\bash.exe` → the `(x86)` sibling) → else a feature gate decides. The Windows `Bash` tool runs the REAL Git-for-Windows (MSYS2) bash — POSIX syntax, so csift's lexical layers stay valid on those records. csift handling: `@trap` matches BOTH shell tools; the bash-LEXICAL layers (bash_danger escalation, bash_mutations attribution in files/recover) deliberately do NOT run on `PowerShell` records — a pending PowerShell lane classifies `awaiting-execution`, and PS shell-side file mutations are invisible to the heuristic (structured Read/Write/Edit attribution is unaffected); `--count-by tool` reports `PowerShell` as its own key. ALSO NOTED (bash_danger.rs header): 2.1.228's dangerous-rm has EVOLVED past the ported 2.1.x generation (fixpoint `$(…)` stripping; a tree-sitter pass bails to explicit approval at >64 command substitutions) — a port refresh is a recorded follow-up.
### 3.10 Elicitation sidecar - the TRANSPARENT merge (AskUserQuestion / ExitPlanMode / MCP)
Three elicitations BLOCK a session on a human yet leave NO usable live trace in the native jsonl: **AskUserQuestion** + **ExitPlanMode** (CC buffers the WHOLE assistant turn until answered - the tool_use is committed to the persisted message array only AFTER the interactive tool resolves; nothing on disk during pending - see §3.4) and **MCP Elicitation** (the inner `elicitation/create` request is an in-memory MCP-client callback, never a transcript record). A CC hook (SKILL.md recipe) records each to an append-only **SIDECAR** `<session-sidecar-dir>/elicitations.jsonl` (the `<uuid>/` dir beside `subagents/`, via `crate::subagent::sidecar_dir_for_session` / `crate::elicitation::sidecar_path`) - NEVER the native transcript (no pollution). Each line carries `csift:"elicitation-marker-v1"` + `csiftPhase`∈{`pending`,`resolved`} + `csiftKind` + `csiftKey`; a `pending` line is shaped like the NATIVE record CC will eventually write (an AUQ/ExitPlanMode `tool_use` on a `type:"assistant"` record; an MCP `type:"system"` record), a `resolved` line is a lightweight `type:"csift-elicitation-resolved"` close marker. **Keyed by the TOP-LEVEL session** (the hook's `session_id` is always the top-level/leader uuid, never a subagent's). The tolerant `csift*` fields parse onto `model::Record` (`is_elicitation_marker()` gates them).
**The merge (NO subcommand - this REPLACED the former `pending` subcommand).** `crate::elicitation::unresolved_pending(session)` groups the sidecar by `csiftKey`; a key with a `pending` and NO `resolved` is UNRESOLVED → its pending record is emitted (exactly the one MISSING from the native transcript - once resolved CC wrote the real record, so the pending is paired off and DROPPED ⇒ no duplicates, the auto-dedup is the whole point). **GHOST-PENDING guard (v0.6.3):** CC fires NO PostToolUse for a REJECTED AUQ/ExitPlanMode, so the hook can never write `resolved` on that path — `unresolved_pending` therefore ALSO drops an AUQ/EPM pending whose `csiftKey` appears on a native record as an actual `tool_use` block id / `tool_result` `tool_use_id` (`native_closes`, elicitation.rs — STRUCTURAL check, a key quoted in prose does not count; cost paid only when ≥1 such key is sidecar-unresolved). The native transcript outranks the sidecar; MCP markers (no native form, key may be a server NAME) are exempt and stay sidecar-paired only. **search / show / verbatim / list** read the sidecar when they read a TOP-LEVEL session (subagent transcripts have none — `show` merges too, via `fetch_records`; a merged record is addressable by `--uuid`) and merge the unresolved-pending records as if native: `classify` labels a pending marker **`agent.tool.use`**, so `search` matches all three kinds under `-t agent.tool.use` (an AUQ/ExitPlanMode `tool_use` via the `Block::ToolUse` arm; the MCP `system` record - which has no `tool_use` block - via a guarded §3.10 arm in `collect_record_hits` that matches its top-level `content` string and tags it `agent.tool.use` named by `csiftKind`, so `search MCPPROBE` and `-t agent.tool.use` both find it; that arm is gated to a marker with NO tool_use block so AUQ/ExitPlanMode never double-emit), `verbatim` appends each as its own pending turn unit (ranked most-recent), `list` annotates the row. A merged record has NO physical line - it renders `(elicitation sidecar)` in place of `Lnnnn` (never a fabricated L), carries JSON `source:"elicitation-sidecar"` + null `line`/`line_no`, and every surface that merged ≥1 record prints the EXACT note **`with elicitation sidecar`** (JSON `with_elicitation_sidecar:true`). Malformed sidecar lines are skipped + COUNTED (folded into the surface's `skipped_lines`, never silent) — and so is a SENTINEL-bearing marker whose `csiftPhase` the current schema cannot read (schema skew: a pre-release fossil under old field names `phase`/`kind`/`key` — provably ours, uninterpretable ⇒ counted, never merged, never invisible; R12, the `_` arm of `pair_unresolved`); a missing sidecar dir/file ⇒ no merge (not an error); near-free when nothing is pending (typically 0 records). **files / recover** carry no file ops for these records, so the merge would produce no output there - they deliberately do NOT read the sidecar (a no-op for uniformity). **Targeting rejection:** `resolve_session_files`' `*.jsonl` branch `bail!`s when the target `is_sidecar_path` (basename `elicitations.jsonl` OR a content-sniff: every line is a `csift`-marked record) - the sidecar is read automatically, never searched directly.
### 3.11 The Bash shell cwd + file freshness (verified against CC 2.1.237; SPEC 4.9 is the authority)
**cwd:** CC spawns a FRESH shell per Bash call at a TRACKED cwd and stamps that value on EVERY record as the top-level `cwd` field (subagents included, compaction-stable). The tracker advances via a `&& pwd -P >|`-chained read-back (non-zero exit / backgrounded commands never advance it); a cd into a project SUBDIR persists silently across calls; a reset fires only outside originalCwd + the `/add-dir` set ("Shell cwd was reset to <path>" in the result). **ANCHOR LAW: read `cwd` off the TOOL_USE record, never the result record** (~0.24% async lag). So bash-operand resolution needs NO cross-command state: `bash_mutations::cwd` joins each operand against the record's own cwd through the in-command cd checkpoints, with an explicit `Resolution` class per row (`absolute`/`cwd-joined`/`cd-tracked`/`unresolved` - verbatim, never guessed). Class-marker pseudo-paths (`git:`/`fmt:`/`interp:`/`pkg:`/`extract:`, `is_class_marker`) are flags, never resolved or path-matched.
**Freshness:** Write/Edit gate on mtime vs the last Read with a content-hash absolution for FULL reads only; bash never invalidates readFileState (bash `cat` SEEDS it); rejections always land `is_error:true`. Three stream signals are consumed by `recover` (v0.8.0): `toolUseResult.staleReadFileStateHint` on Bash results (CC names the modified files, paths relative to the shell cwd -> resolved via the CARRYING record's cwd -> hard `hint_modified` boundary; `parse_stale_read_hint`, recover/carriers.rs), `toolUseResult.staleRecovered:true` on successful Edits (-> `stale_recovered` annotation), and the `edited_text_file` attachment (16KB budget; an EMPTY snippet = the over-budget degraded form). A change producing NONE of these leaves no transcript trace - absence of signal is never evidence of absence, hence recover's per-window opaque accounting (`ScanResult::opaque`, the M/K/P disclosure + suggested-search).
---
## 4. Invariants + design decisions (with rationale)
- **No `unwrap`/`expect` in library/hot paths.** Propagate via `anyhow::Result` + `?`. Tests may `unwrap`. Only `main.rs` turns an error into an exit code. Rationale: a panic in a transcript-recovery tool destroys the recovery; surface the full chain instead.
- **No silent truncation / no silent failure.** Any cap (`--max-count`) MUST report the drop count; a skipped malformed line is counted, never hidden — since v0.6.6 this includes NON-candidate lines: every byte-prefilter rejection path runs the O(1) `parse::line_shape_malformed` shape check (non-blank but not `{…}`-framed ⇒ counted; catches free-text garbage + crash-truncation at zero §7 cost). Residue: a `{…}`-framed invalid INTERIOR is only counted on a parse candidate - EXCEPT on `stats` (v0.8.1): its non-candidate type probe (`line_type_probe`, stats.rs) fully validates every line for the line-type census, so the named census authority's count is exact. **SCOPE (R12): whole-file census vs window census.** The count is a WHOLE-FILE census only on the full-scan commands (`search`/`stats`/`show`/`files`/`recover`/`verbatim`/`image`); the head/tail readers — `list` rows and `agents` lifecycle — census the LINES THEY READ: `parse::head_records*` returns `(skipped, consumed_end)` and the paired tail scan FLOORS at that offset, so the two windows are DISJOINT and every malformed line in them is booked exactly ONCE (pre-v0.6.8 an all-garbage file reported exactly 2× — both scans, finding no anchor, walked the whole file and double-booked every line; the tail still walks below the floor for missing anchors, it just never re-counts). A mid-file tear is OUTSIDE those windows BY DESIGN (§7 — full coverage measured ~4× the unscoped `list` runtime), so `list`/`agents` `skipped_lines: 0` is NOT a whole-file verdict: the text note scope-qualifies (`among the head/tail lines read — full census: csift stats`) and `stats` is the named census authority. When pairing a head and a tail scan over ONE file, ALWAYS pass the head's `consumed_end` as the tail `floor` — summing two unfloored scans reopens the R12 double-book. When adding a scan with a byte prefilter, route the non-candidate arm through `parse::non_candidate_verdict` — a bare `LineVerdict::Ignore` there reopens the R10 hole. `resolve_session_files` bails when a pinned id matches nothing rather than returning empty. Rationale: a quietly-shortened search result is a correctness bug for an LLM consumer that trusts completeness. **Context-safety output caps.** `list` carries a DEFAULT cap of `session::DEFAULT_LIST_CAP = 50` rows when listing ALL projects (no target / no `--sessions-from`) — the flood guard against a ~69k-line all-projects dump; a scoped query is UNCAPPED, and `--max-count N` overrides. `stats` gained an opt-in `--max-count N` (default unlimited). Both keep the MOST-RECENTLY-active rows (`session::last_activity`) and report the drop (text footer + JSON summary `dropped_by_cap`), never silent — and `list`'s scope banner / JSON header `sessions_in_scope` are computed PRE-cap (v0.6.3: the row flood-guard never shrinks the scope numbers; only the rows are capped). `show` carries a DEFAULT cap of `show::DEFAULT_SHOW_CAP = 200` record units (open ranges like `--turn ..` address a whole transcript) — keep-FIRST; the drop reports the EXACT continuation command (`+N more record unit(s) beyond the 200-unit cap · continue: csift show @<id> --line A..B (or --max-count 0 = uncapped)`; JSON `dropped_by_cap`/`refetch_remainder`). **`--max-count 0` = UNCAPPED uniformly** on `list`/`stats`/`search`/`show` (a bare `Some(0)` used to be an absurd literal zero-cap). `files` (a compact `--by summary` rollup) and `agents` (a tree scoped to its target) were left scope-bounded, so neither took a default cap.
- **Tolerant parsing.** Real jsonl carries far more fields than documented and some records lack `timestamp`. Deserialize only what's used, ignore the rest, never crash on a new field/block type (the `#[serde(other)] Unknown` arm). EXTEND the model tolerantly when CC evolves; never tighten it into a crash.
- **JSON output = the `serde_json::json!` macro ONLY, never `#[derive(Serialize)]`.** There is ZERO `Serialize` derive in `src/` (verify: `grep -rn "derive(Serialize" src/` is empty). A fresh model's instinct - "add `#[derive(Serialize)]` to the row struct" - is WRONG and breaks the house style. Every JSON line is hand-built with `json!({...})` in a per-module `*_json` projector (`hit_json`/`preview_json`/`node_json`/...) then `serde_json::to_string(&value)?` + `println!`. **One JSON object per line** (JSONL, not a pretty array). Render structs stay plain (`#[derive(Debug, Clone)]`); the projector maps struct -> `json!` explicitly, so the wire shape is decoupled from field layout. **Envelope (search + verbatim):** the FIRST line is ALWAYS the shared `text::envelope_scope_header(command, top, sub)` record `{kind:"header", command, sessions_in_scope, top_level_sessions, subagent_sessions}` (text.rs) — emitted UNCONDITIONALLY, NOT gated on `sub > 0`; `list`/`files`/`recover` emit it too. `search` ALSO prints a TRAILING summary object `{matched, sessions, transcript_ids, transcript_ids_truncated, dropped_by_cap, skipped_lines, with_elicitation_sidecar, excerpts_truncated}` as the last line (search.rs `render_json`). `transcript_ids` (+ `transcript_ids_truncated`, first-100 cap) = the distinct MATCHING-TRANSCRIPT ids — DELIBERATELY named apart from `-l`, which emits the OWNING-session `parent_session_id`s; the two answer different "which sessions?" questions. On a ZERO-match result the summary gains `definitive_absence:true` + `active_filters` + `excluded_by_label` (§ empty-result self-diagnosis, below). `excerpts_truncated` = ≥1 emitted excerpt was clipped to the cap (machine echo of the text reader-caution; always false under `--no-truncate` — and under `show`'s `--line`/`--uuid`/`--turn` — since those lift the cap to `usize::MAX` so `Hit::truncated` is never set). Reproduce BOTH when adding a JSON surface that spans/caps. **v0.5 row-kind set** (`kind` belongs to the envelope alone): `search --count-by <axis>` emits `census` rows `{axis, key, records}` + a `{axis, matched_records, distinct_keys, excluded_records, dropped_by_cap, skipped_lines}` summary; **`agents --format json` is FLAT** — `{kind:"session"}` count row → `{kind:"run"}` rows → each subagent a `{kind:"agent"}` row in tree PRE-ORDER (nesting is TEXT-only, rebuilt from `parent_agent_id`/`depth`; NO nested `workflow_runs`/`children` row field, and `--agent` is no longer a bare-node exception); a node's transcript shape is `shape` (not `kind`). `files` summary gained `sessions` (distinct owning sessions; deliberately NO `dropped_by_cap` — files has no cap); `show` summary is `{records, dropped_by_cap, refetch_remainder, non_record_lines, skipped_lines, with_elicitation_sidecar}`. **`pairing` is a closed machine enum** `paired`|`pending`|`orphan`|`null`. **Timestamp PAIR LAW:** every machine instant is a `_utc`+`_local` PAIR — a record's OWN instant is `ts_utc`+`ts_local`; a NAMED instant is `<name>_utc`+`<name>_local` (`first_*`/`last_*`/`trigger_*`/`started_*`/`completed_*`/`last_activity_*`/`pending_since_*`; on an agents node `completed_*` is STATUS-GATED — non-null only when `status==completed` — and `last_activity_*` carries the tail instant on every timestamped lane). The text form derives from these but drops the UTC copy (§4 timez helpers).
- **Shared `text::` + `timez::` helpers are REQUIRED, not optional.** Do NOT hand-roll truncation, the scope banner, the malformed-line note, or a range parser - a divergent fourth copy (`agents::one_line` dropping the elision count) is exactly the silent-truncation contract bug this consolidation killed. REUSE:
| Need | Helper | Notes |
| --- | --- | --- |
| excerpt + explicit `... (+N chars)` | `text::truncate_excerpt(s, max)` | CHAR-counted (codepoint-safe); cap = **200** for `list`/`agents` scannable previews, **400** for `search`/`recover` context excerpts |
| flatten multiline + truncate | `text::collapse_and_truncate(s, max)` | the `agents` returned-message preview |
| malformed-line note fragment | `text::malformed_note(n)` | bare `N malformed line(s) skipped`; caller frames it |
| scope span wording | `text::scope_span_fragment(top, sub)` | the one phrasing |
| text scope banner (suppress when `sub==0`) | `text::emit_scope_banner(top, sub)` | `list`/`files`/`search`/`recover` |
| JSON scope header line (ALWAYS the envelope's line 1) | `text::envelope_scope_header(command, top, sub)` | `{kind:"header", command, sessions_in_scope, top_level_sessions, subagent_sessions}` (text.rs); emitted unconditionally |
| range spec: PARSE, then RESOLVE against a domain length | `text::parse_range_spec(s, label, one_based) -> RangeSpec` then `RangeSpec::resolve(len, one_based) -> (usize, usize)` | THE one range grammar (`--line` · `--turn` on EVERY command that windows on turns — `search`/`show`/`stats`/`files`/`recover`/`verbatim`/`image` · `--file-lines`): `N` (single) · `A..B` (closed) · `N..` (to end) · `..N` (from start) · `..` (all) · negative `-k` = k-th from the END (`-3..` = last 3, `-1` = the last). Types: `text::Endpoint` (At / FromEnd / Open) + `text::RangeSpec {start, end}`. A statically-detectable reversal (`9..3`, both explicit) errors at PARSE; a len-dependent reversal resolves to an EMPTY range. Each consumer resolves the spec against its OWN per-file domain length (turn count / line count) — `search`/`stats`/`files`/`recover`/`verbatim`/`image` resolve `--turn` per-transcript; `show --line` / recover `--file-lines` against the file's line count. The dash form `A-B` still hard-errors with the correct spelling. The range-valued clap args carry `allow_hyphen_values = true` so the space form `--turn -3..` parses (not mistaken for a flag). |
| timestamp — the ONE canonical local text form `YYYY-MM-DD HH:MM:SS <TZAB>(UTC±offset)` | `timez::format_timestamp(raw)` (seconds) | system-local via jiff, infallible; NO raw-UTC copy (that lives only in JSON `ts_utc`) — shares `tz_marker` with the ms variant |
| compact local instant `…HH:MM:SS.mmm <TZAB>(UTC±offset)` (no UTC copy) | `timez::format_local_compact(raw)` (ms) | for token-lean text; same renderer + `tz_marker` as `format_timestamp` |
| JSON `ts_local` ISO+offset | `timez::local_iso(raw)` | `None` if absent/unparseable |
| the detected local zone | `timez::local_tz()` | `TimeZone::system()` |
- **Performance is a contract** (SPEC section 7). `list`/`search` stay fast on 200MB+ files via: **mmap** (`parse::mmap_bytes`) + **`memchr` SIMD newline scan** (`scan_lines_bytes`, `memchr_iter`) + a cheap **byte/regex prefilter** with full `serde_json::from_slice` ONLY on candidate lines (`parse_candidates_parallel`) + **tail reads that SEEK from EOF backward** newest-first (`tail_records`, never parse the whole file to find the last user/agent msg; `list` additionally byte-prefilters head/tail lines via `head_records_prefiltered`/`tail_records_prefiltered` so megabyte metadata lines are never parsed) + **`rayon`** fan-out across files AND within one giant file (`scan_lines_parallel` — `search`/`verbatim`/`files`/`recover` all use it) + the **§7f whole-file gate** (see the landmine below) + **`mimalloc`** as the global allocator (macOS libmalloc contends across rayon workers) + heavy raw fields kept UNPARSED (`Record.tool_use_result`/`attachment` are `Box<RawValue>`; the hot paths read small fields via `Record::tur_probe`, deep consumers parse on demand via `tool_use_result_value`/`attachment_value` — never revert these to eager `Value`) + **`subagent::JournalCache`** (each workflow `journal.jsonl` read+parsed ONCE per topology build, not once per agent). Don't regress these: don't `BufReader`-copy a whole file, don't full-parse every line, don't lose the prefilter. **Prefilter silent-drop landmine (where "no silent truncation" actually bites).** `search`'s literal prefilter (`required_literal` + the `Prefilter` enum, search.rs) is the ONE place a naive optimisation silently drops matches. It scans RAW JSON line bytes, where string content is JSON-encoded (`"` -> `\"`, every control char `< 0x20` + DEL -> `\uXXXX`/`\n`/...), AND several render paths rewrite whitespace before matching (`normalize_line` collapses runs; multi-part texts join with `' '`/`'\n'` seams). So a prefilter literal is emitted ONLY when the pattern has NO regex metachar, NO JSON-escaped char (`json_escapes_in_string`: `"` / control / DEL), and NO whitespace (a rendered `hello world` can be raw `hello\nworld`). An eligible literal gets a byte prefilter in BOTH case modes: case-sensitive -> `memmem`; smart-case-insensitive -> a `(?i)`-escaped-literal bytes regex (`Prefilter::CaselessLiteral`, Teddy-class caseless scan). Non-ASCII (`>= 0x80`) survives verbatim as UTF-8, so it stays prefilter-eligible. **Synthesized-text escape hatch (`Matcher::synth`, REQUIRED for correctness):** a small closed set of render paths fabricates matchable text that is NOT a verbatim substring of the raw line (automation labels' kind slug/status, the AUQ scaffold, a rejection's `[plan: ...]` pointer resolved from a DIFFERENT record, the compact-boundary `trigger=...` excerpt, `--resolve-persisted` external content); each is detectable by a raw marker its carrier ALWAYS bears, checked via per-needle `memmem` finders (NOT Aho-Corasick — the quote-leading `"answers"` needle degenerates AC's start-byte prefilter on JSON). Markers split two tiers (`Matcher::synth_verifiable`/`synth_conservative`): VERIFIABLE markers (`<task-notification>`, `"answers"`, the AUQ answer markers, `-t`-gated `compact_boundary`) re-render JUST the marker lines through the SHARED engines (`record_text_sections`/`auq_exchange`/`record_raw_text`) and regex-check the synthesized texts — so a marker-heavy main session still gates when none of them can match; CONSERVATIVE markers (`To tell you how to proceed` — its `[plan: …]` pointer resolves through ANOTHER record — and the flag-gated persisted pointers — external file content) force the full scan. (No `AskUserQuestion` needle: the question-side `tool_use`'s matchable text is the verbatim `name` + re-serialized `input`, and the name bytes sit in the raw line — a bare needle would disable the gate for every file whose injected context merely mentions the tool.) **Whole-file gate (§7f):** when a PARALLEL per-line pre-scan (same rayon chunking as the full scan; a relaxed AtomicBool short-circuits it on the first may-match line) proves a whole file can't match (and no `--line`/`--uuid` addressing), `search` skips building records for it entirely — the gated file's candidate lines were each syntax-validated (`parse::validate_line_syntax`, no Record build) so the malformed-line count stays exact (a TESTED contract). NEVER extract a literal more aggressively (e.g. HIR-based literal extraction from a regex), NEVER prefilter decoded content, and NEVER add a synthesized render path without adding its raw marker to `synth_marker_automaton` — when a literal is unsafe, fall back to running the regex on the raw bytes (still pre-JSON, just no cheap short-circuit).
- **`unsafe_code = "deny"` crate-wide, ONE exception.** `deny` (not `forbid`) is deliberate: it bans every `unsafe` block EXCEPT the single audited mmap site in `src/parse/` (`#[allow(unsafe_code)] unsafe { Mmap::map(&file) }`), which is irreducibly unsafe and SPEC-mandated. `forbid` cannot be locally overridden so it would refuse to compile mmap. Any OTHER `unsafe` without an explicit allow is a hard error. Do NOT add a second allow.
- **No crate-level `#![allow(dead_code)]`.** The only `#[allow(dead_code)]` is targeted on `model::Record`/`Block` fields deserialized for tolerance but not yet read. A crate-wide allow would mask real dead code.
- **Exit codes** (main.rs): `Ok(()) -> ExitCode::SUCCESS` (0); any `Err -> eprintln!("csift: error: {err:#}")` (full anyhow chain, alternate `:#`) + `ExitCode::FAILURE` (non-zero, no custom numeric codes) — with ONE documented exception: `wait`'s timeout exits **124** (the GNU timeout convention; a monitor's timeout is a normal outcome scripts branch on, rendered via `std::process::exit` after the output flushes). Do not extend 124 to any other command or outcome. De facto (v0.6.4, informational): a clap USAGE error exits 2 (clap's own convention), a csift `bail!` exits 1 — never build on the split, the contract is 0-vs-non-zero. NOT errors (exit 0): `verbatim --slice` out-of-range prints nothing; a ZERO-match `search` still exits 0 (a DEFINITIVE absence, NOT an error) and prints its NORMAL empty-result output on STDOUT (`no matching exchanges` in text; the `kind:"header"` + `kind:"summary"` envelope lines in json) — only the `csift: 0 matches …` self-diagnosis rides STDERR (see §5). HARD non-zero: `show` address-miss (an explicit line/uuid/turn resolving to no record → `bail!("no such record(s): …")`); `recover` partial/no-history (`bail!` pointing at `--salvage`/`--patches`/`--coverage`); `image --id` no-match or ambiguous `#N` (`bail!`).
---
## 5. Module map (`src/` — directory modules after the 2026-08 modularization)
Every former >600-line single-file module is a DIRECTORY module with a PATH-NAMED root: `src/<m>.rs` beside `src/<m>/` (the root carries the module doc, shared imports, `mod` decls + `pub(crate) use` re-exports — every `crate::<m>::item` path still resolves), children are focused submodules (~200-500 lines, each opening `use super::*`), and a module's unit tests live in `src/<m>/tests.rs` (shared fixture helpers) + `src/<m>/tests/<feature>.rs` (feature-named files that mirror the source children — never numbered slices). **`mod.rs` is BANNED crate-wide** (the legacy layout): enforced twice, by clippy `mod_module_files` and by the pre-commit structure gate. Dispatch conventions are unchanged (`Command::Foo(args) => foo::run_foo(&args)`; variant->module still not 1:1 — `list` lives in `session/`). HARD structure limits (husky-enforced, §7): <=600 lines per `.rs`, <=20 `.rs` per folder (a paired module root `name.rs`+`name/` counts with its directory, not against the parent), <=16 subfolders per folder (raised from 15 at v0.9.0: src/ grows one dir per command family by design and sits AT the cap - the next command module must consolidate an existing dir first), <=5 levels below `src`/`tests`; per-function cognitive complexity <=25 (clippy.toml); no `mod.rs`.
```
main.rs # binary entrypoint: parse_argv -> install --claude-home override -> dispatch -> ExitCode
cli/ # clap surface: argv (the P0 normalize_argv machinery), root (Cli + Command),
# selectors (-t/-T label machinery), formats, one *_args file per command family
path/ # home (CC-exact cwd encoding + config home), project_dirs (>200 prefix scan),
# scope, trap (@trap self-id), targets (--sessions-from), resolver (the
# @-grammar: collect_targets/scan_top_level/resolve_prefix_uniqueness), ids
model/ # markers, record (serde model), predicates (is_genuine_user &c.), exchange
# (AUQ/opens_turn/reconstructed text), mutation (FileOp), grouping (turns),
# automation (pulse model), taxonomy (Role/Class::ALL), peer, classify_support,
# classify (the role.class.sub engine)
parse/ # lines (mmap + tolerant role matchers [the ONE unsafe mmap site], lazy parse),
# parallel (rayon scans + malformed accounting), readers (RevLines, head/tail)
session/ # `list`: rows, run (DEFAULT_LIST_CAP flood guard), summarize, render
search/ # types (Hit/Exchange), matcher (prefilter + synth markers, SPEC 7d/7f), census
# (-c/-l/--count-by + zero-match diagnosis), run (run_search), scan
# (search_one_file + fetch_records — show's engine), turns_match, hits (the
# emission engine), record_text, render (headers/tokens/envelope)
show/ # addr (line/uuid/turn specs + caps), run (raw/rendered fetch), render,
# branch (--branch-points fork facts)
stats.rs # one-scan per-session aggregates
subagent/ # ids (the r5 trio helpers), types, discover, meta (+JournalCache), lifecycle
# (frozen-lane detection), spawn (ParentSpawnIndex/global index), topology
agents/ # run (filters/windows/teammate hint), render (text tree), json (flat rows)
files/ # types, run, mutations (structured + bash heuristics), rollup, render
bash_mutations/ # entry, heredoc, mask, commands, outputs, redirect, cwd (CwdAt/Resolution
# tracking + resolve), interp (write-idiom analysis), classes (fmt:/pkg:/
# extract: markers) - the lexical bash parser
bash_danger.rs # faithful 1:1 port of CC's dangerous-rm classifier
recover/ # types, run (batch driver), backups (--list-backups store listing),
# scan (+opaque accounting), events (extraction),
# carriers (toolUseResult/attachment + freshness signals), buffer (SparseBuffer),
# replay, diff (LCS/unified), render (disclosure + merge), restore, report, timeline
plan.rs + plan/ # plan forward/reverse resolution (reuses whoami's detection + guidance);
# audit (--audit: edits joined against corpus bindings)
live.rs + live/ # `status` + `wait` (the live-truth pair): registry (sessions/<pid>.json
# + ps pid probe), tail (bounded tail state machine), children
# (subagent tails + incremental journal), verdict (six-verdict join),
# conditions (--until grammar), status, wait (poll loop), render
turns/ # `verbatim`: config, units, run, build, richness, select, planning
# (plan_session), render, scope, json
image/ # refs (base64 probes), selection (#N), run, convert (transcode), render
elicitation.rs # transparent elicitation-sidecar merge (no subcommand)
time_window.rs # --since/--until parsing (shared by all WHEN consumers)
timez.rs # REQUIRED shared time helpers
text.rs # REQUIRED shared helpers: excerpts, banners, envelope, the range grammar
whoami.rs # `whoami`: CLAUDE_CODE_SESSION_ID detection, false-positive-safe
```
**E2E tests** live in ONE integration binary `tests/cli/` (main.rs declares the modules): `harness.rs` + `harness/` (the `Home` isolation harness in `home.rs` + the shared fixture builders in `fixtures.rs` — all helpers live HERE, feature files carry no cross-file dependencies). Per-SUBCOMMAND modules own that command's tests (`search/` split by aspect — basics/filters/census/headers/taxonomy/classify/output/scope; `agents/`, `verbatim/`, `recover/`, `files/`, `targeting/` as feature dirs; `list`/`stats`/`show`/`image`/`plan_whoami` as single files), and CROSS-COMMAND sweeps get their own root files: `argv.rs` (flag ordering, exit conventions), `contracts.rs` (range grammar, timestamps, honest empties), `spanning.rs` (span-switch/scope-banner/cap uniformity), `elicitation.rs` (EVERY sidecar-merge test across surfaces). A new e2e test goes in the module whose FEATURE it exercises, never a grab-bag file. The golden baseline stays at `tests/turns_pre_feature_baseline.txt`.
**`search` flags from the verified audit** (cli.rs `SearchArgs`, search.rs): `-c/--count-only` prints ONLY the integer total `exchanges.len() + dropped_by_cap`; `-l/--sessions-with-matches` prints ONLY the distinct OWNING session ids (uncapped, stderr-noted on a `--max-count` drop — the pipe into `--sessions-from -`); the JSON summary's `transcript_ids` stays the per-transcript detail set (sorted, cap-100 + explicit `transcript_ids_truncated`) — DELIBERATELY named apart from `-l` (which emits the OWNING-session `parent_session_id`s), so the two "which sessions?" answers never collide. `--count-by <AXIS>` is a THIRD terminal mode (like `-c`/`-l`): a per-KEY CENSUS of the matched records along ONE closed axis (`label`|`tool`|`turn`|`session`|`pairing`|`model`|`attachment`|`version`|`result`; an empty pattern = a whole-scope census; the `attachment` axis IMPLIES the `--attachments` gate; `result` buckets tool results `ok`|`error` - pairing answers "did a result come back", result answers "was it good"; an errored result also renders an inline `[error]` and hit JSON carries `is_error`). On `label` each record counts under every leaf it carries THAT SURVIVES the active `-t`/`-T` (v0.6.3: census keys pass the same `LabelFilter::selected` predicate that admits the record's views — a dual-labeled record never leaks its filtered-out twin into the keys; no filter ⇒ the full label set, and a leaf's count = how many records `-t <leaf>` would surface, via `search::label_census` — also the zero-match probe engine, which passes `LabelFilter::all()` since it exists to name what the DROPPED filter excluded); every OTHER axis counts each record ONCE and EXCLUDES records outside its domain (no tool/pairing/model), reporting the excluded count (`search::axis_census`). A multi-SECTION record (text+tool_use, a batched notification) emits one hit per section but is ONE census record (`record_groups`, search.rs). `set_pairing` rides the tool BLOCK through the comm views (a SendMessage/spawn tool_use → sent/signal, a subagent-return tool_result → inbox), so a frozen SendMessage is `pending` under ANY selector; record-text comm units (no tool_use_id) stay outside the axis. `turn` sorts ascending (histogram); the rest count-desc except `label` (richest-first). JSON emits `census` rows `{axis, key, records}` + a `{axis, matched_records, distinct_keys, excluded_records, …}` summary. `-T/--label-not` excludes labels (same selector grammar as `-t`, `LabelFilter` include-minus-exclude, richest-SURVIVING-view dedup, statically-empty combos bail). `--raw` emits matched records' VERBATIM jsonl lines (per-file mmap backfill onto `Hit.raw`; stdout pure, notes on stderr; sidecar hits omitted + noted). Every JSON hit carries `refetch` (the ready-to-run `csift show` command addressed at the hit's own `session_id`); in TEXT output a SUBAGENT hit line ALSO prints a `↳ csift show @<agent-id> --line N` continuation — the exact refetch with the CORRECT id, closing the text-mode ID-law hole where a parent-uuid + subagent-line would silently fetch the WRONG record (a line number is per-FILE); top-level hits stay terse (safe via the header's id-prefix token — a valid `@` target since v0.7.0). `--siblings` is a ZERO-ARG flag with a FIXED policy (`sibling_cap`): message classes uncapped, thinking≤2 / tool.use≤3 / tool.result≤3 / harness≤2 per leaf; the capped remainder renders an explicit `(+N more · csift show @<id> --line A..B)` pointer and rides JSON as `siblings_hidden` + `turn_lines`. Record FETCHING moved to `csift show` (search has no `--line`/`--uuid`; `AddressSet` + `fetch_records` remain in search.rs as show's engine). `--resolve-persisted` rewrites tool-result text pre-match (failures soft-noted; under `--raw` it affects matching only). PATTERN guardrails: leading `@` = hard error; uuid-shaped + no session target = stderr note.
**Empty-result self-diagnosis (the keystone, `search`).** A ZERO-match search is a DEFINITIVE absence, NOT an error — exit 0, with stdout carrying the NORMAL empty-result output (`no matching exchanges` in text; the `kind:"header"` + `kind:"summary"` envelope lines in json). The diagnosis rides STDERR: a line framing the result as `0 matches — a definitive absence (exit 0), NOT an error`, the active filters (`active_filters_str`), and — WHEN a `-t`/`-T` label filter was active — an ACTIVE PROBE: a re-scan of the SAME pattern + scope with the label filter DROPPED that NAMES the label(s) the pattern DOES occur under (e.g. `⚠ but "X" DOES occur — N record(s) under: agent.tool.use`). The probe cost is paid ONLY on a zero-hit + label-filtered query. JSON echoes it in the summary: `definitive_absence:true` + `active_filters` + `excluded_by_label` (`{records, by_label}`, null when no label filter). Structures: `search::EmptyDiagnosis` / `emit_empty_diagnosis` / `active_filters_str`. Rationale: the observed #1 slippage cause is a model reading an empty result as a SYNTAX error and abandoning csift — this makes absence machine-legible and self-correctable.
**argv normalization:** the entrypoint is `cli::parse_argv` (NOT `Cli::parse`): it runs `cli::normalize_argv` to reorder declared flags ahead of leading-`-` encoded-project positionals, fixing clap's `allow_hyphen_values` greedy-absorb bug (#3880). Flag discovery is zero-drift via `Cli::command()` introspection (action `Set`/`Append` take a value; `SetTrue`/`SetFalse`/`Count`/help/version do not). **v0.5 P0 fix:** the subcommand is now located by SCANNING PAST any declared ROOT flags (+ their value tokens; a `--flag=value` inline form spans one token) rather than assuming `argv[1]` is the subcommand — so a pre-subcommand GLOBAL flag no longer disables normalization. Previously `csift --claude-home DIR list @x --max-count 3` mis-parsed (the trailing flag got swallowed by the PATH positional with a misleading "not a project target" error); now "flag order is free" and "`--claude-home` in any position" hold IN COMBINATION. So `csift list <ENCODED> --format json` and `csift --claude-home DIR list @x --max-count 3` both work.
---
## 6. How to add or modify a subcommand
1. **Args struct** in `cli.rs`: add `pub struct FooArgs` (`#[derive(Args, Debug)]`). Positionals that target sessions use `#[arg(value_parser = parse_project_target)]` so a `--`-typo errors cleanly. Put example-rich text in `#[command(after_help = "...")]`/doc comments (the `--help` is the LLM's manual - keep it usable from `--help` alone). If the subcommand spans subagents, add the span flag(s) + a `pub fn want_subagents(&self) -> bool` mirroring the existing ones (src/cli//1048/1220/...).
2. **Command variant**: add `Foo(FooArgs)` to `enum Command` (src/cli/).
3. **Dispatch**: add one arm to `run()` in `main.rs`. The real arm shape is `Command::Foo(args) => foo::run_foo(&args)` (handler fn convention is `run_<subcommand>`: `run_list`/`run_search`/`run_agents`/...). **The dispatch MODULE need not be named after the `Command` variant** - the precedent is `Command::List(args) => session::run_list(&args)` (main.rs): `list` lives in `session.rs` (`pub fn run_list`, src/session/); there is NO `src/list.rs`. Variant->module is NOT 1:1; pick whatever module already owns the concern. Every handler returns `anyhow::Result<()>`, so `?`/`bail!` funnel to the single FAILURE arm; never `eprintln!`+exit yourself.
4. **Module**: host the handler in a `src/*.rs` (a new `src/foo.rs`, or fold into an existing module as `list` did into `session.rs`); declare `mod foo;` in `main.rs` if new (keep alphabetical-ish). The handler `run_foo` then resolves targets via the shared resolver:
`let scope = SubagentScope::from(args.want_subagents()); let files = path::resolve_session_files(&args.paths, scope, path::Caller::Other)?;`
- **Resolve via `&args.paths`** - that is the convention for `list`/`agents`/`files`/`recover`/`verbatim`/`image`/`plan` (src/session/, src/agents/, src/files/, src/recover//373, src/turns/, src/image/, plan.rs/263). The ONE exception is `search`: its first positional is `PATTERN`, so `SearchArgs` adds the SOLE `targets()` helper (src/cli/ - the only `fn targets` in the tree) that returns the path positionals only (`self.paths.clone()`, excluding the pattern); `search` resolves `&args.targets()` (src/search/). Do NOT add a `targets()` method to a non-`search` struct (it has none -> compile error); use `args.paths`.
- **Scope resolution.** Every default-on span command (`list`/`search`/`stats`/`files`/`recover`/`image`/`plan`) exposes `want_subagents() -> bool` (= `self.subagents || !self.no_subagents`) and resolves `SubagentScope::from(args.want_subagents())` (`true->WithSubagents`, `false->TopLevelOnly`). `verbatim` is the opt-in exception (`self.include_subagents && !self.no_subagents`; flag spelled `--subagents`). `SubagentScope` has only those two variants; there is no subagents-only mode.
- `agents` passes `false.into()` because it LISTS subagents rather than spanning them.
- The `path::Caller` arg is currently **INERT** - `resolve_session_files` does `let _ = caller;` (src/path/, "reserved for future subcommand-aware guidance"). Pass `path::Caller::Other` (or the matching variant) to stay future-proof, but do NOT expect it to change remediation text today; the doc's "Caller tunes messages" is forward-looking, not live.
Read files via `parse::mmap_bytes` + the head/tail/parallel scanners; lazy-parse only candidate lines; **derive each row's id trio with the `crate::subagent` helpers** (`session_id_from_path` / `is_subagent_path` / `parent_session_id_from_path` - never read the stem, see section 3.7); render via `text::` excerpt helpers + `timez::` timestamps. Honor: no `unwrap`, report any cap's drop count, support `--format json` via a `json!` projector (section 4 JSON invariant - no `derive(Serialize)`).
5. **Tests**: add an end-to-end case to the matching FEATURE module under `tests/cli/` (one integration binary driving the built csift against synthetic transcripts; per-subcommand dirs and the cross-command root files are mapped in §5 — put a new test in the module whose feature it exercises, never a grab-bag file). **Isolation mechanism (load-bearing - reuse it, don't hand-roll a fixture scheme):** the e2e harness never touches the real `~/.claude`. The `Home` struct (`tests/cli/harness/home.rs`) makes a TempDir, creates `.claude/projects` under it, and `Home::write(rel, contents)` drops a fixture jsonl at `$HOME/.claude/projects/<rel>` (e.g. `<ENC>/<uuid>.jsonl`). Its `run`/`run_with_env`/`run_full` invoke the binary via `env!("CARGO_BIN_EXE_csift")` (the exact build cargo produced) with `.env("HOME", &self.root)` and `.env_remove("CLAUDE_CODE_SESSION_ID").env_remove("CODEX_COMPANION_SESSION_ID")` for deterministic `whoami` (a test that needs a session id sets it back via `run_with_env`). This works because `claude_home()` (src/path/) falls back to `$HOME/.claude` when no `--claude-home`/`$CLAUDE_CONFIG_DIR` is set. Start from the `populated_home()` builder (`tests/cli/harness/fixtures.rs`) - the canonical fixture, used in ~198 cases (`populated_home()`/`Home::new()` combined) - rather than inventing a new tree.
Unit tests local to a module go in `src/<m>/tests.rs` (shared fixture helpers + `mod` decls, wired by a plain `#[cfg(test)] mod tests;` at the bottom of `src/<m>.rs` - **no `#[path = "..."]` attribute needed**) with feature-named children under `src/<m>/tests/` that mirror the source submodules; child files never depend on each other. A SMALL module may keep an inline `#[cfg(test)] mod tests { ... }` while it fits the 600-line cap. Run `cargo test`.
---
## 7. Commands + quality gate
```bash
cargo build # debug (GATE: must succeed)
cargo build --release # optimised (thin-LTO, 1 cgu) for real scans
cargo run -- <subcommand> ... # run a subcommand
cargo fmt --all -- --check # format gate
cargo clippy --all-targets -- -D warnings # lint gate (warnings-as-errors)
cargo test # unit + integration tests (also installs the hook on first run)
```
**Pointing csift at a fixture tree (not the real `~/.claude`).** csift resolves its data root (`claude_home()`, src/path/) with precedence `--claude-home <DIR>` > `$CLAUDE_CONFIG_DIR` > the OS home's `.claude` (`$HOME` on Unix, `%USERPROFILE%` on Windows — the split mirrors CC's own `os.homedir()`; a stray Git-Bash `HOME` on Windows is deliberately ignored, v0.7.1), then reads `<root>/projects/<encoded>/*.jsonl` (`projects_root`). `--claude-home` is a clap **global** flag (`global = true`, src/cli/; installed in `main` before dispatch, main.rs), so it is honored by EVERY subcommand and `normalize_argv` lets it sit before OR after the subcommand: `csift --claude-home /tmp/fix list` and `csift list --claude-home /tmp/fix` both work. Use `--claude-home <dir>` (or export `$CLAUDE_CONFIG_DIR`) to run a manual repro against a synthetic projects dir instead of your live transcripts. All three positions are exercised by the `custom_claude_home_via_env_var_and_flag` e2e test. **Test-isolation trap:** `--claude-home` installs via `set_claude_home_override`, which writes a SET-ONCE process-global `OnceLock<PathBuf>` (`CLAUDE_HOME_OVERRIDE`, src/path/; `let _ = ...set(dir)` silently ignores later sets, src/path/). So an IN-PROCESS unit test CANNOT relocate the data root via the override - the first set wins for the whole test binary (all tests share one process). For path-resolution unit tests call the PURE `resolve_claude_home(flag, env, home)` (src/path/, factored out precisely so precedence is testable WITHOUT touching the OnceLock); for e2e use the `Home` harness, which spawns a fresh subprocess per test with `$HOME` set (`tests/cli/harness/home.rs` `run_full`). Don't write a flaky test that calls the global `claude_home()`.
**Pre-commit gate (cargo-husky).** On the first `cargo test`/`cargo build` after checkout, cargo-husky (dev-dep, `user-hooks` feature) installs `.git/hooks/pre-commit` from `.cargo-husky/hooks/pre-commit`. It runs, in order: the STRUCTURE GATE (hard limits: <=600 lines per `.rs` file [Markdown exempt], <=20 `.rs` files per folder, <=16 subfolders per folder, <=5 subfolder levels below `src`/`tests`) -> `cargo fmt --all -- --check` -> `cargo clippy --all-targets -- -D warnings` (which also enforces per-function cognitive complexity <=25 via clippy.toml + the `[lints.clippy] cognitive_complexity` warn) -> `cargo test`. A failure blocks the commit. No CI service runs - this hook is the ENTIRE gate. Edit the **source** hook (`.cargo-husky/hooks/pre-commit`), never the installed `.git/hooks` copy, then re-run `cargo test` to reinstall. Genuine-WIP bypass: `git commit --no-verify` (sparingly). Test layout: `tests/cli/` (the ONE e2e binary, feature modules + `harness/`), `tests/turns_pre_feature_baseline.txt` (golden), and per-module unit tests under `src/<m>/tests/`.
**Versioning + release discipline (LOAD-BEARING — the SKILL's staleness guard depends on it).** The version is TRIPLE-LOCKED and the three MUST move together in the SAME commit: `Cargo.toml` `version` ≡ `SKILL.md` `Surface: **vX.Y.Z**` header ≡ `csift --version`. This is not cosmetic: `SKILL.md` line 3 tells the LLM consumer "Surface vX must == `csift --version`; if an invocation you were confident about errors, your knowledge is stale — re-read this file", so it uses version==surface as its ONLY staleness detector. Shipping a changed surface under an unchanged version silently defeats that guard and puts two DIFFERENT binaries in the wild claiming the same version — a correctness bug for the primary (LLM) consumer, not a nicety.
- **What is a "surface change" (⇒ REQUIRES a version bump):** any change to a command, a flag, a flag's semantics, a default value, a JSON field name/shape, a row `kind`, output text a consumer keys on, OR the `--help` text (SKILL calls `csift <cmd> --help` "the authoritative flag manual", so help IS surface). Pure internal refactors, tests, comments, and non-user-facing docs are NOT surface changes and do not require a bump.
- **Which number (0.x policy — csift is pre-1.0, so per SemVer the MINOR carries breaking changes):** bump the **MINOR** (`0.Y.0`) for a BREAKING surface change — a removed/renamed command or flag, a renamed/removed JSON field, a changed default or exit semantics (e.g. v0.3→v0.4 = `turns`→`verbatim`, `session_ids`→`transcript_ids`, removed `--subagent`). Bump the **PATCH** (`0.4.Z`) for a NON-breaking surface change — an added flag, a corrected help string, a fixed output, a bug fix (e.g. v0.4.0→v0.4.1 = help/doc drift corrections). At 1.0+, breaking ⇒ MAJOR.
- **Tag discipline:** every released version gets an ANNOTATED git tag `vX.Y.Z` on its release commit (`git tag -a vX.Y.Z -m "…"`), and tags are pushed (`git push --tags`). The release commit is `chore(release): vX.Y.Z`. Never leave a shipped version untagged.
- **Changelog discipline:** `CHANGELOG.md` (repo root) carries ONE digest entry per released version, newest first (`## [X.Y.Z] - YYYY-MM-DD`), written IN the release commit — never after the fact. The SPEC §6 per-version ledger stays the authoritative long-form record; the changelog is its scannable digest, and the two must agree. Entries are contemporaneous: name flags/fields by their names AT that version (no forward references to later renames).
- **Release flow (do ALL, in order):** (1) bump the triple (Cargo.toml + SKILL header) AND add the version's CHANGELOG.md entry in one commit; (2) `cargo build --release && cargo install --path .`; (3) confirm `csift --version` == the SKILL surface header; (4) the pre-commit gate runs on commit (`chore(release): vX.Y.Z`); (5) `git tag -a vX.Y.Z`; (6) `git push && git push --tags`; (7) if the surface change renamed/removed a COMMAND, re-sync the global hook in lockstep (see §9 / the `csift-turns-slice.sh` coupling). A pushed release is never amended/force-pushed — a follow-up fix is its own PATCH bump + tag.
---
## 8. Conventions
- `PascalCase` types, `snake_case` items, one module per concern.
- **Structure limits are law (husky-enforced)**: <=600 lines per `.rs` file (200-400 is the sweet spot; smaller only when natural), <=20 `.rs` files per folder, <=16 subfolders per folder, <=5 subfolder levels below `src`/`tests`, per-function cognitive complexity <=25 (clippy.toml). Markdown is exempt. Split BEFORE the gate forces you to.
- Comments capture WHY / a non-obvious constraint, not what the code already says.
- **Commit messages follow Conventional Commits**: `type(scope): subject` — types in use: `feat`, `fix`, `perf`, `refactor`, `docs`, `test`, `chore`, `style`; `!` after the type/scope marks a BREAKING surface change (pairs with the §7 version policy: pre-1.0, breaking ⇒ MINOR). Subject in the imperative mood; the body says what changed and why in plain engineering terms; messages are pure ASCII.
- `Cargo.toml` lints: `unsafe_code = "deny"`, `missing_debug_implementations = "warn"`, clippy `all = warn` with a few pedantic allows. Deps pinned by `^`-range + `Cargo.lock`.
- Output discipline: errors -> stderr (`csift: error:` + full chain); data -> stdout; `--format json` is the machine contract (one object per line where it makes sense).
- ASCII-only source/docs unless a test fixture intentionally exercises multi-byte (then keep it neutral: emoji / accented Latin). Comments in `.rs` files are strictly ASCII: no em dash, use `-` (gate-enforced); deliberate output glyphs and fixtures in string literals are the only non-ASCII.
- README/promotional prose: em dashes rare (single digits for the whole page; commas, colons, parentheses do the work), sentence lengths mixed, bullet skeletons varied - not every bullet a bolded lead + balanced elaboration. One punchy summary line per document, not per section.
---
## 9. What NOT to do
- **Don't introduce BM25 / embeddings / semantic search.** Regex only.
- **Don't `unwrap`/`expect` in library paths**, and **don't silently truncate** - always report the drop count.
- **Don't parse a whole 200MB file** when a head/tail read answers the question; don't lose mmap+memchr+prefilter+tail+rayon.
- **Don't add a second `unsafe` allow** beyond the audited mmap site; don't switch `deny` to `forbid` (it would refuse to compile mmap).
- **Don't trust most-recent-mtime for `whoami`**; don't guess when `$CLAUDE_CODE_SESSION_ID` is absent.
- **Don't reverse a path encoding** (lossy); don't assume a workflow subagent's env `$CLAUDE_CODE_SESSION_ID` is its own id (it is the parent's).
- **Don't blindly trust this doc's field list** - real jsonl evolves; re-verify against `~/.claude/projects` and EXTEND the model tolerantly rather than tightening it into a crash.
- **Don't bump a dependency major** without an explicit reason.
- **Don't edit `.git/hooks/pre-commit` directly** - edit `.cargo-husky/hooks/pre-commit` and re-run `cargo test`.
- **Don't add semantic/numeric exit codes** - 0 / non-zero only, full chain to stderr; the single exception is `wait`'s documented timeout code 124, never to be extended.
- **Don't ship a SURFACE change (a flag, output, JSON field, or `--help` text) without bumping the version triple + tagging + a CHANGELOG.md entry** - `Cargo.toml` ≡ `SKILL.md` surface header ≡ `csift --version` must move together, every release is tagged `vX.Y.Z`, and the version's CHANGELOG.md digest entry rides the release commit (§7 versioning discipline). A changed binary under an unchanged version defeats the SKILL's staleness guard.
- **Don't edit `CLAUDE.md`** - it's a symlink; edit `AGENTS.md`.
```