Skip to main content

Module pools

Module pools 

Source
Expand description

The pool’s shadow log (0.7.7): what each detached check (check_async.rs) showed its model, and how other orders would have ranked the same lessons, one line per check in pools.jsonl beside the store’s outcomes.

The stop pool fills its budget in one fixed order (whole.rs select_filled: the shortlist, then BM25 × weight × trust, then newest first). Whether another order finds more of what the judge applies can only be measured on lessons the model was shown, so each check logs, per eligible lesson in its fill order (Pooled::ids), its role (prefix, boundary draw, tail sample, not shown), its exact chance to be shown (pi: 1 in the prefix, pt + (1 − pt)·π_b past it, whole.rs explore) and the scores of the candidate orders the pre-registered switch test compares (internal evaluation “learn-retrieval”, not published):

  • bm25q: today’s word match (conscience.rs raw), before weight and trust; w, trust, act: the memory weight, the authors’ trust, the activation behind the weight (memory.rs).
  • bm25f: BM25 of question, situations and consequence against the whole work, idf over the store’s lessons at the stop (E3-fb).
  • cue: the best BM25 of the lesson’s retrieval cues (cues.rs) against the work’s tail, idf over every cue of the store (E5); null without cues.
  • mv: BM25 of situations and cues against the design moves of the session’s latest check that finished before this stop (predict.rs moves; E6); null without such moves.
  • q: the lesson’s online value from its earlier verdicts, Q ← Q + 0.2(r − Q), changed +1, ignored −0.5, prior 0, placebos skipped (E2).
  • cos: the cosine of the lesson’s vector to the work’s tail (vectors.rs scored); null without vectors.
  • rrf: reciprocal rank fusion of bm25f, cue, mv (bm25f where mv is null) and q over the rows the model was shown (E7): Σ 1/(60 + rank), ranks 1-based, ties at their mean, a missing score last.

Logged only. None of these reaches the pool, its order or the prompt (the shadow-purity test pins it); the order changes only when the switch test passes. Local: never synced (borg-mesh SYNCED), never shared, kept whole until that test reads it (its size is in borg status). Nothing with learning off. The miss causes in borg status and borg web fold from it (misses).

Structs§

Check
What the check adds after its call (hook.rs settle).
Misses
The miss causes (borg status, borg web): of the first applies at grade ≥ 2 the checks logged (not pending, not in play), those the fill order ranked low, at a fill position past half the prefix or shown only by a draw (boundary or tail), each with its mechanical cause (cause).
Pooled
One check’s line of pools.jsonl: per-check fields, then columns aligned with ids (every eligible lesson in fill order: the prefix’s order, then the lessons it dropped as the fill met them), floats at 4 decimals, null where a score does not apply.
Rows
What the pool step knows of its rows (hook.rs pool): the eligible lessons in fill order, their roles and chances, today’s word match, and the exploration’s sizes.

Constants§

BOUNDARY
CAUSES
The mechanical causes, in the order cause tries them: out of scope for where the session works; no word of the work matched it but the vectors found it; no word matched; its words matched in the pool’s top half but weight and trust put it in the bottom half (demoted); its words matched less than half the pool’s (weak words); otherwise (the shortlist, ties, newest first).
DELIVERED
IN_PLAY
IN_SCOPE
NOT_SHOWN
Pooled::role values.
PENDING
PREFIX
Q_CHANGED
Q_IGNORED
Q_STEP
The online value’s step and rewards (q). Method: the offline scorer’s E2, fixed before the switch test was registered: a verdict moves Q a fifth of the way; a change is worth +1, an ignored question −0.5; unclear and not arisen leave it.
RRF_K
RRF’s rank offset. Method: the standard constant (Cormack, Clarke and Büttcher, “Reciprocal Rank Fusion outperforms Condorcet and individual Rank Learning Methods”, 2009), as the offline scorer.
SHORTLIST
Pooled::flags bits: in the shortlist (BM25’s top vectors::SHORTLIST plus vector hits, computed even when the pool fits), added by the vectors, in scope for where the session works, delivered in the session already (hook.rs had_in), pending a verdict this check judges, in play (applied by an earlier check of the session, or delivered).
TAIL
VECTOR
VERSION
Protocol: the version of a pools.jsonl line’s format (Pooled::v).

Functions§

cause
Why row i of p sat low in the fill order (CAUSES), from its logged scores; the rank shares are within the check’s rows, 1-based ranks over their number, ties at their mean (the offline miss report).
log
Appends the check’s line (pooled) to pools.jsonl in home. Nothing with learning off; fails quietly (a log, never in the way of the check).
misses
The miss causes of the store home (Misses): the cache, then the log’s lines past its offset folded in and the cache saved again; a log shorter than the cache’s offset (replaced) is folded anew.
path
The log beside a store’s outcomes.
pooled
The check’s line: rows scored as the module doc says.
ranks
1-based ranks of v, best (largest) first, ties at their mean rank, a missing value below every present one (tied with the other missing ones), as the offline scorer ranks.