Expand description
lineprior: domain-agnostic action priors built from historical action
sequences. Given a state, it answers “what actions have historically
worked well from here?” – as a prior for another system to weigh, not
as an oracle. See AGENTS.md for the full design rationale.
This library never writes to stdout/stderr and never panics on
malformed user input; all failure paths return Error.
Structs§
- Build
Config - Tuning knobs for
crate::build::build_prior_book. - Build
Output - Result of
build_prior_book_from_reader: the built book plus any non-fatal warnings collected along the way.bookmay legitimately be empty (no observations, or everything got filtered out) – callers that want “empty means an error” should checkbook.entries.is_empty()themselves, the same waycrate::report::summarize’s callers do. - Build
Stats - What a build actually did to the input, independent of the resulting
PriorBook– lets a caller check whether their own pre-filtering (e.g. a domain-specific ply/depth cutoff) combined withBuildConfig’s thresholds behaved as expected, without re-deriving these numbers by hand from the input. - Calibration
Bin - Ranking quality for one equal-width confidence bin of the #1 candidate,
among evaluated test observations whose top1 confidence fell in
[min_confidence, max_confidence)(the last bin is closed on both ends). Always exactlyEvalConfig::calibration_binsentries, in ascending bin order, regardless of whether a given bin saw any observations. - Eval
Config - Tuning knobs for
evaluate. - Eval
Output - Result of
evaluate: the report plus warnings from the train pass. Warnings are collected from the train pass only – when both readers point at the same file (the only way the CLI uses this), a malformed line is malformed identically in both passes, so a second collection would just duplicate the first. The test pass still skips (non-strict) or aborts (strict) on invalid records; it just doesn’t re-report them. - Eval
Report - Ranking-quality report produced by
evaluate. - Gate
Calibration Bin - Ranking quality for one equal-width
probability_positivebin, over out-of-fold predictions collected duringGateModel::fit’s nested cross-validation pass (see its doc comment) – not the single-level CV used to pick the deployed model’s lambda, which would be optimistically biased if reused here directly. Mirrorseval.rs’sCalibrationBinshape: a well-calibrated model should showempirical_positive_ratetracking bin probability roughly 1:1. - Gate
FitOutput - Result of
GateModel::fit: the fitted model plus its own diagnostics. - Gate
FitReport - Diagnostics from a
GateModel::fitcall, for deciding whether the fitted model is trustworthy enough to act on before any acquisition logic is layered on top (deferred to a later round). - Gate
Model - A fitted gate-outcome surrogate. Opaque by design –
GateModel::fitandGateModel::predictare the API; the standardized-space coefficients are an implementation detail, not something callers should need to read directly. - Gate
Model Config - Tuning knobs for
GateModel::fit. - Gate
Observation - One historical training candidate that has already been through a real
gate run, used as training data for
GateModel::fit. - Gate
OofPrediction - One nested-CV out-of-fold audit row:
predicted_elo/probability_positivewere produced by a model that never saw this candidate’s own row or itsgroup_idduring fitting or lambda selection (seeGateModel::fit_with_validation’s doc comment) – exactly the populationGateFitReport::weighted_rmse/calibrationare computed from, exposed per-candidate for real-data validation (comparing against an external baseline, auditing calibration and interval coverage by hand) before building anything on top of Round A. Not deduplicated bycandidate_id: a caller whose data happens to repeat one is still owed every row, not a silently-collapsed one. - Gate
Prediction GateModel::predict’s output for oneGateQuery.- Gate
Query - A not-yet-gated candidate to score with
GateModel::predict. - Gate
Validation Output - Result of
GateModel::fit_with_validation: everythingGateFitOutputcarries, plus the nested-CV out-of-fold audit table and the confidence level its intervals represent. A superset, not a fork –reportis the exact sameGateFitReportfitreturns, so the two functions can never disagree about the aggregate metrics. - Observation
- One recorded step: from
state,actionwas taken, withoutcome. - Pareto
Entry - One point on the coverage/MRR tradeoff curve.
- Parse
Outcome - Everything produced by a parse pass: the valid observations plus any non-fatal issues collected in non-strict mode.
- Prior
Action - One candidate action ranked for a given state.
- Prior
Book - In-memory prior book: state -> ranked candidate actions.
- Prior
Entry - One line of prior-book output: a state and its ranked actions.
- Sequence
Prior Score - Result of
PriorBook::score_sequence: an aggregate read on how much historical precedent supports a whole caller-supplied candidate path. - State
Entropy - Entropy of one state’s prior distribution, in bits. Low entropy means one action dominates and the prior is likely useful; high entropy means many actions compete and a fallback search may be safer.
- Step
Score - Per-step result of
PriorBook::score_sequence: whether/how well this step of a caller-supplied candidate path is backed by historical data. - Summary
Report - Aggregate statistics over an entire prior book, for the CLI
summarycommand. - Threshold
Sweep Entry - Selective-prediction metrics at one confidence threshold: if the caller
only acted when the #1 candidate’s confidence was
>= min_confidence, how often would they have predicted at all, and how good were those predictions? Always exactlyEvalConfig::thresholds.len()entries, in the requested order. - TopK
HitRate - Hit rate for one requested
k. AVec(not a map) so JSON output has a deterministic order matching the ordertop_kwas requested in. - Tune
Candidate Result - One evaluated grid candidate’s outcome. Note the field is
objective_value(a number), notobjective– that name is reserved at theTuneOutputlevel for whichTuneObjectivewas chosen. - Tune
Constraints - Candidates failing a constraint stay in
TuneOutput::all_results(so the caller can see why they were excluded) but are neverselect_best- eligible.Noneon any field means “no floor/ceiling on this metric.” - Tune
Metrics - The subset of
EvalReportthat matters for comparing candidates. - Tune
Output - Full result of a
tunerun. - Warning
- A non-fatal issue skipped in non-strict mode. Carries enough detail to report to the user without aborting the whole run.
Enums§
- Confidence
Mode - How
PriorAction::confidenceis computed. See [crate::score::confidence] and [crate::score::wilson_lower_bound] for the underlying formulas. - Error
- Errors produced while parsing, validating, or building a prior book.
- Missing
Timestamp Policy - What to do with an observation that has no
observed_at_unix_secondswhenBuildConfig::time_decay_half_life_daysis set. Inert when time decay is disabled. - Outcome
- Result of taking
actionfromstate. - Tune
Objective - Which held-out metric
tuneranks candidates by. See each variant’s doc comment;CoveredMrris the recommended default – optimizingMrralone tends to pick configs that abstain (report no candidate) except when very confident, while optimizing coverage alone tolerates a sloppy prior.CoveredMrrpenalizes both. - Tune
Param - One
--param key=v1,v2,...sweep. Each variant already carries its values pre-parsed to the right type, so applying it to aBuildConfigcan never mismatch a key against the wrong value type – unlike a generic(key, values: Vec<AnyValue>)pair, this is checked by the compiler, not at grid-expansion time.
Constants§
- DEFAULT_
CONFIDENCE_ K - We use k=20 so confidence grows slowly for low-sample actions. This prevents one-off successes from dominating the prior.
- DEFAULT_
CONFIDENCE_ Z - z=1.96 is the two-sided-95% value conventionally used (Evan Miller / Reddit-ranking style) as a one-sided Wilson lower bound – slightly more conservative than a strict one-sided 95% bound (which would use ~1.64), which is the right direction for a “how much should I trust this” score.
- DEFAULT_
DRAW_ VALUE - A draw is a genuine partial outcome in adversarial games (chess, shogi), not a loss – we default to crediting it as half a win rather than scoring it identically to a failure.
- DEFAULT_
SOURCE_ WEIGHT - Multiplier applied to an observation whose
sourceisNoneor not a key inBuildConfig::source_weights.1.0means “trust unlabeled/unknown sources same as any other” – the backward-compatible default.
Functions§
- build_
candidate_ result - Builds one
TuneCandidateResultfrom an already-evaluated candidate. Does not callevaluate()itself – callers (e.g. the CLI’s grid loop) own opening/parsing the input and only reach this onOk. - build_
config_ fingerprint - Deterministic fingerprint of a
BuildConfig, stable within a given lineprior version (it hashes a JSON encoding, and serde_json’s exact byte layout for floats is not itself guaranteed forever-stable across serde_json versions – unlikeeval’s sequence-id hash, which only ever hashes raw string bytes and so carries a stronger cross-version guarantee). Good enough to detect a stale cached prior book within one project’s lifetime; not meant as a long-term archival checksum. - build_
prior_ book - Aggregates observations into a
PriorBook, applying filters, smoothing, normalization, and ranking per AGENTS.md’s scoring model. - build_
prior_ book_ from_ reader - Parses and aggregates a JSONL observation stream in one pass, without
ever collecting a
Vec<Observation>– peak memory stays bounded by the number of unique(state, action)pairs, not the number of lines read. Prefer this overparse_jsonl+build_prior_bookfor anything beyond small, already-in-memory inputs. - covered_
fraction - Observation-weighted fraction of test observations whose state had at
least one candidate.
1.0 - fallback_rate, notEvalReport::coverage(that field is state-weighted and deliberately doesn’t complement to 1 withfallback_rate– see its doc comment). Defaults to0.0(no coverage) on the degeneratefallback_rate: Nonecase (zero test observations), matching the “can’t confirm coverage” reading. - default_
gate_ lambda_ grid 1e-4through100.0, log-spaced-ish, chosen to bias toward strong regularization for a p-approx-n regime – seeGateModelConfig::lambda_grid.- evaluate
- Builds a prior from a sequence-held-out train split and reports how well it ranks the actual action taken on the held-out test split.
- expand_
grid - Deterministic Cartesian product over
params, in the given param and value order, applied on top ofbase.config_ids are"cfg_001","cfg_002", … in generation order. No params -> a single candidate (baseitself, as"cfg_001"). - load_
prior_ book - Reads a prior book back from the JSONL format emitted by
build. Querying itself isPriorBook::query– an unseen state returns no candidates rather than an error or an invented action. - load_
prior_ book_ with_ config - Like
load_prior_book, but also checks the file’s embeddedBuildConfigfingerprint (if any) againstexpected_config’s fingerprint. ReturnsError::BuildConfigMismatchif the file has a header and it doesn’t match – otherwise a stale cached book could silently be reused as if itsconfidence/priorvalues were computed under the caller’s current config. A file with no header (saved by plainsave_prior_book, or by a version of lineprior that predates this) is accepted unconditionally – there’s nothing to compare against. - meets_
constraints - Whether
reportclears every floor/ceiling set inconstraints. A missing metric (None) is treated as failing a floor and failing a ceiling check is skipped only when the ceiling itself is unset – i.e. “no data” never counts as “constraint satisfied.” - objective_
value - The value
tuneranks a candidate by. A metric field ofNone(valid config, but its book covers nothing to measure – e.g.min_confidencefiltered every candidate away) scores0.0here rather than being treated as an error; only a configevaluate()itself rejects (anErr) is excluded before this is ever called (see [crate::tune]’s module doc). - pareto_
front - Non-dominated set over
(mrr, covered_fraction), both maximized: a candidate is excluded if another candidate is at least as good on both dimensions and strictly better on at least one. Unlikeselect_best, this ignoresmeets_constraints– it shows the whole tradeoff space so a caller can override “best” with their own pick. O(n^2); fine at the grid sizestunetargets (tens to low hundreds of candidates). - parse_
jsonl - Streams JSONL observations from
reader, one line at a time so memory stays bounded regardless of input size. - save_
prior_ book - Writes a prior book as JSONL: order-0 entries in the same deterministic
order as
PriorBook::entries_sorted, followed by any context entries inPriorBook::context_entries_sorted’s order. The inverse ofload_prior_book. - save_
prior_ book_ with_ config - Like
save_prior_book, but also writes a leading header line withconfig’s fingerprint, so a laterload_prior_book_with_configcall can detect whether the book was built under different config values than the caller currently expects. - select_
best - Highest
objective_valueamongmeets_constraints == trueentries inresults, ties broken by earliestconfig_id(grid generation order).Noneifresultsis empty or nothing satisfies its constraints. - state_
entropy - Per-state entropy, sorted lexicographically by state for determinism.
- summarize
- Summarizes a whole prior book: how many states/actions it covers and how confident/decisive it tends to be.