Skip to main content

quarantine_corrupt_lines

Function quarantine_corrupt_lines 

Source
pub fn quarantine_corrupt_lines(
    paths: &RunPaths,
    backup_ts: &str,
) -> Result<Option<Quarantine>>
Expand description

Heal a poisoned events.jsonl by excising its corrupt physical lines.

P2 made the supervisor skip a corrupt JSONL line in memory and keep tailing, but the bytes stayed on disk forever — so every fresh strict reader (read_all_events / a future rebuild_projections) still hard-errors on them, and the skip diagnostic is unreachable to a strict replay (the corrupt line aborts the read before it). This is the durable repair: under the run’s RunLock, the original log is renamed to events.jsonl.corrupt-<ts>.bak and a recovered events.jsonl is written in its place containing every line except the corrupt ones.

“Corrupt” means exactly what the strict readers reject: a newline-terminated, non-empty line that does not parse as a full Event envelope. Empty lines and a torn (newline-less) final line are retained verbatim — the readers already tolerate both, so excising them would be a behavior change, not a repair.

Returns Ok(None) when the log is missing or already clean (no rename, no rewrite — the common case is cheap: one read, no corrupt line found). Returns Ok(Some(_)) with the backup path and removed offsets when at least one line was excised. Caller is expected to surface the outcome (e.g. a supervisor.event_log_quarantined diagnostic) and, for a live tail, restart its read cursor at offset 0 since every byte offset shifts.

backup_ts is supplied by the caller (kept out of core so the rename is deterministic in tests); a filename-safe basic-ISO stamp like 20260628T120000Z is the intended form.

§Operator recovery

The excised bytes are never destroyed — they survive verbatim in the events.jsonl.corrupt-<ts>.bak sibling (named by the emitted supervisor.event_log_quarantined { backup_path } diagnostic). To recover a line the automated repair dropped: open the .bak, inspect the line(s) at the reported removed_byte_offsets, hand-fix any salvageable JSON, and — if you want the record back — stop the run’s supervisor, append the corrected line to the live events.jsonl (or replace the file wholesale from a fixed copy of the backup), then restart the supervisor. The healed log is the source of truth; projections rebuild from it.