Module segment 

Deterministic transcript segmentation for the compile_transcript MCP tool (V2b-2): splits a raw agent-session transcript into turns and, within each turn, into code/log/body sub-segments — pure, zero-regex, zero-clock, so the same transcript + policy always segments byte-identically. See segment::segment_transcript. Deterministic transcript segmentation for the compile_transcript MCP tool (V2b-2, see the crate’s PLAN.md, section V2b).

segment_transcript turns a raw agent-session transcript — plain text with role markers, or JSONL — into an ordered list of TranscriptSegments, each wrapping an ordinary [super::ContextFragment] plus the audit metadata (turn, role, kind, byte range) the compile_transcript tool reports alongside the compiled context. The resulting fragments feed the existing, unmodified super::ContextCompiler pipeline — this module only decides how to cut the transcript up, never what to keep or drop.

Zero regex, zero clock, single linear scan per stage — same determinism contract as [super::chunk]: the same transcript + the same SegmentationPolicy always segment byte-identically (see segmentation_twice_is_byte_identical in the test suite).

Pipeline

1. Format detection ([detect_and_segment]): jsonl when every non-empty line parses as a {role, content} JSON object, plain otherwise. A caller-forced format that does not parse is a hard error — never a silent fallback to the other format — surfaced as crate::error::MemoryError::SegmentationError (a FORMAT failure, distinct from a budget/cap breach; see below).
2. Turns: jsonl — one line, one turn, role taken directly from the parsed JSON. plain — a CLOSED table of markers ("System:", "User:", "Human:", "Assistant:", "AI:", "Tool:", "### User", "### Assistant"), first match at the start of a line opens a new turn; a transcript with no marker at all is one turn with role: None.
3. Sub-segmentation (plain turns only — a jsonl turn’s content is a JSON-decoded string, not a byte-aligned slice of the transcript, so it is never re-scanned; the underlying content.contains("```") / value-density rules in [super::classify] still see it, unaffected): fenced code blocks ([super::chunk::fence_segments]) become atomic code segments; runs of at least 8 consecutive log-like lines (a volatile timestamp/pid prefix — [super::log_normalize::mask_volatile_prefix] — or a raw-text repeat) become log segments; everything else is body.
4. Normalization: an unsplittable fence over crate::limits::MAX_FRAGMENT_BYTES is a hard error (never silently truncated); an oversized body segment is re-split with [super::chunk_text]; segments under SegmentationPolicy::min_segment_bytes merge into an adjacent segment of the same turn and kind; more than crate::limits::MAX_FRAGMENTS segments after merging is a hard, actionable error (“raise min_segment_bytes”) — never a silent drop.

A genuine budget/cap breach (transcript over crate::limits::MAX_TRANSCRIPT_BYTES, an unsplittable oversized fence, or too many fragments after merging) surfaces as crate::error::MemoryError::ContextOverLimit; a FORMAT/parsing failure (a forced jsonl line that does not parse) surfaces as the distinct crate::error::MemoryError::SegmentationError (issue #1516, m2 — kept separate precisely so a caller filtering on the error message cannot confuse a malformed-input error for a size breach, even though both map to the same INVALID_PARAMS-category MCP code). A path-sourced transcript can additionally fail with crate::error::MemoryError::IngestDisabled/crate::error::MemoryError::IngestOutsideRoots/ crate::error::MemoryError::IngestPath, via [super::ingest::resolve_transcript_path].

Structs

SegmentationOutcome

The full result of segment_transcript: the detected format, the segments, and how much normalization merging did.

SegmentationPolicy

Tuning knobs for segment_transcript. Default is the recommended profile.

TranscriptSegment

One segmented piece of the transcript: an ordinary ContextFragment (ready to feed super::ContextCompiler) plus the audit metadata the compile_transcript tool reports in its segmentation.segments list.

Enums

SegmentFormat

Which transcript format to assume, or detect automatically.

SegmentKind

What kind of content a sub-segment carries — decides whether it was cut out as an atomic fence, a detected log run, or ordinary prose/dialogue left for [super::classify]’s rule table to judge.

Functions

segment_transcript

Segment text under policy — see the module docs for the full pipeline. Pure: no I/O, no clock, no randomness; the same text + policy always produce byte-identical output.