Skip to main content

Module gate

oxibrain_cli::cmd

Module gate 

Source
Expand description

oxibrain eval --suite gate — three-arm comparison runner (ARCHITECTURE.md §17.2).

Loads eval/golden/ (manifest + episodes + questions), ingests the episodes as declarations on a fresh brain, then for each question runs arm (b) lexical and arm (c) hybrid and scores whether the answer appears in the rendered top-K statements.

Per ROADMAP §4, this is the controlled comparison whose outcome decides whether to proceed with M10 as written, fix extraction, or invoke D19’s pre-commitment (demote the graph). Arms (a) and the frontier tier are out of scope for the golden-only gate (LongMemEval removed from the plan, 2026-08-13).

The runner is deterministic and self-contained: no network, no LLM, no live model. Tokens/query is approximated by the number of ranked items + the character length of rendered statements.

Functions§

run
run_with_dir
Entry point used by the CLI dispatcher — supports the gate suite with a corpus-dir argument. The CLI passes the path to eval/golden/ (resolved relative to the manifest via the --corpus flag, defaulting to eval/golden from the workspace root).