Skip to main content

Module protect

Module protect 

Source
Expand description

Explicit, deterministic preservation of user-marked spans across every compressor (read, shell, proxy, prose).

Two complementary, fully deterministic mechanisms (#498 — pure functions of their inputs, no model, no clock, no global state):

  1. Universal markers <lc_safe>…</lc_safe>: any compressor wraps its work in compress_preserving; the content between the markers passes through verbatim and the markers themselves are stripped from the output.
  2. protect token list (ctx_read convenience): line_is_protected lets the line-based lossy filters (entropy / information-bottleneck) force-keep every line that contains one of the given tokens.

Security: callers MUST run secret redaction before protect (redact → protect → compress). Protect never re-introduces redacted secrets because it only ever passes through bytes that already survived redaction.

Constants§

SAFE_CLOSE
Closing marker for a verbatim-preserved span.
SAFE_OPEN
Opening marker for a verbatim-preserved span.

Functions§

compress_preserving
Compress only the unprotected regions of input with f; everything between <lc_safe> and </lc_safe> passes through byte-for-byte and the markers are stripped from the output.
has_markers
Cheap pre-check: does the input contain at least one protect marker? Lets hot paths skip the span-splitting machinery entirely when nothing is marked.
line_is_protected
True if line must survive a lossy line filter because it contains one of the explicit protect tokens. Empty tokens are ignored so an empty list (or a list of empty strings) reproduces today’s behaviour exactly.
protect_fragment
Stable cache-key fragment for a protect token list, or "" when the list is empty (so unprotected reads keep their current cache key).