Skip to main content

Module slicer

Module slicer 

Source
Expand description

The contract every slicer meets, and the machinery that makes it hold.

§Why the guarantee is in the trait rather than in each implementation

Law II says a step receives only the context it declared, and the cost model rests on that: s_max is what keeps total cost linear in the number of steps instead of quadratic. A slicer that could exceed its cap would break a bound the compiler has already promised a reviewer — and it would break it silently, four slicers deep, in whichever one was written last.

So Slicer::condense is a provided method. An implementation writes slice and never sees the cap enforcement, the CAS write, or the condensate assembly; condense does all three around it. Adding a fifth slicer therefore cannot forget any of them, which is the only way a guarantee spread across implementations stays true.

§Over the cap is an error, never a truncation

SliceError::OverCap is deliberately not recoverable by chopping the tail off. A slicer that produced too much has misjudged its own output, and silently cutting the excess would be exactly the naive truncation the whole layer exists to avoid — the last thing dropped would be the end of the file, which is where the interesting part usually is. Failing tells the author their slicer is wrong; truncating tells the model a lie.

§Counting tokens without a tokeniser

Caps are in tokens; nothing here can tokenise. TokenCount is the seam. The default, Estimate, is deliberately pessimistic — it assumes text costs more tokens per byte than it usually does — because the failure modes are asymmetric: over-estimating wastes a little window, under-estimating breaks Law II at admission time, after the money is spent. A caller with a real tokeniser injects it and gets exact behaviour.

§Determinism

Slicing is a pure function of (raw, cap). No clock, no randomness, no iteration over a hashed collection. Replay (US-1006) re-slices recorded raw output and compares digests, so a slicer that varied between runs would make a replayed run fail to match for reasons that have nothing to do with the model.

Structs§

ByteCeiling
Counts a token per byte.
CommandOutcome
What running something produced, beyond its output.
Estimate
The default token estimate: pessimistic, and deterministic.
MemoryStore
A store that keeps everything in memory.
Raw
Raw tool output, before slicing.
Slice
What a slicer produced, before it becomes a Condensate.

Enums§

SliceError
Slicing failed.
StoreError
The content-addressed store could not be used.

Traits§

Slicer
Layer 4: turns raw output into a size-capped, typed summary.
Store
Where raw output is kept so that nothing is lost by slicing.
TokenCount
Turns text into a token count.