Expand description
Template collapse for the governor’s R4 rule.
super::lossless_collapse only folds byte-identical lines, so real logs,
lint output and exports, whose lines differ in a timestamp, a sequence
number or an id, never shrink. Here every line is reduced to a template:
its variable fields become a placeholder. Variable fields are maximal ASCII
alphanumeric words that are
- a number (
2026,750, the03of2026-03-14), except a short enumeration suffix of a name (worker-2,shard_7: at mostMAX_NAME_SUFFIX_DIGITSdigits after<letter>-/<letter>_), - a number with a short unit or separator letter (
750ms,14T10,411Z,0x1f: starts with a digit, at most three letters), - a hex id of at least
MIN_HEX_IDcharacters with a digit (fc56b260), or a UUID.
Decimals, IP addresses, durations and dates follow from these rules.
Words such as lint codes (W501, E201), hosts, paths and messages are
not variable, so two lines with different text never share a template.
Lines sharing one template at least MIN_GROUP times are shown once,
where the first one was, as [N× like] <first line> … last: <changed part of the last line>. Everything else stays verbatim and in order. This is
lossy for the reader; the caller keeps the full output in the archive and
names it. The result is a pure function of the input (#498): no clock, no
hash-map order reaches the output.
Safety classes:
- Input with a diff hunk is refused whole: every diff line matters.
- Protected lines are never grouped: test-runner lines, Python traceback blocks (header, indented frames, exception line) and indented continuation lines right after a severity line.
- JSON records (lines opening with
{, or a[line that parses as JSON) are never grouped: in an export the numbers are the data. - Severity lines (error, warning, failure, panic, exception words) stay
verbatim while all of them together fit in
SEVERITY_VERBATIM_BYTES. Past that, each distinct normalized message is kept once with its count, grouped only with lines of the same severity template, never with plain lines. A rare error therefore stays visible: its text differs from the surrounding retries, so its template has fewer thanMIN_GROUPmembers and the line stays verbatim.
Structs§
- Template
Collapsed - The collapse result: the rendered body and its counts.
Constants§
- MAX_
INPUT_ BYTES - Larger input is left alone (bounded runtime; the archive keeps 10 MB).
- MAX_
NAME_ SUFFIX_ DIGITS - Digits after
<letter>-or<letter>_up to this length name a member of a small set (worker-2) and stay literal. - MIN_
GROUP - Fewest lines sharing a template that are collapsed. A group line costs about two lines (first line plus the changed tail of the last), so eight members save at least 75 % of them; smaller sets are usually distinct events the reader wants to see one by one.
- MIN_
HEX_ ID - Shortest hex word (with a digit) treated as an id.
- SEVERITY_
VERBATIM_ BYTES - All severity lines stay verbatim while their total size stays below this (about 1,000 tokens); past it each distinct message is kept once.