Skip to main content

Module oracle

Module oracle 

Source
Expand description

Deterministic oracle system for validating RLM REPL trace outputs.

This module provides oracles that can verify FINAL() answers from RLM traces without requiring cloud LLM judges. This enables synthetic training data generation for the BitNet distilled navigation model.

§Architecture

  • Grep Oracle: Pattern-match verification (e.g., “find all async functions”)
  • Tree-sitter Oracle: Structural AST verification (function signatures, etc.)
  • Validator Pipeline: Routes queries to appropriate oracles and outputs golden traces

§Usage

use codetether_agent::rlm::oracle::{TraceValidator, OracleResult};

let validator = TraceValidator::new();
let result = validator.validate(&analysis_result, &source_file).await;

match result {
    OracleResult::Golden(trace) => save_to_jsonl(trace),
    OracleResult::Unverified => {} // No oracle available
    OracleResult::Failed(reason) => {} // Oracle disagrees
}

Structs§

AstPayload
AST/structural query payload.
AstResult
A single AST query result.
BatchValidationStats
Statistics from batch trace validation.
GeneratedQuery
A pre-generated query ready for use.
GrepMatch
A single grep match with line number and text.
GrepOracle
Grep-based oracle for validating pattern-match queries.
GrepPayload
Grep/pattern-match payload.
OracleTracePersistResult
Captures the outcome of writing a single oracle trace to the local spool and (optionally) uploading it to remote storage.
OracleTraceRecord
Flat, serialisable snapshot of an oracle verdict plus its full trace — the unit of persistence for the spool and S3.
OracleTraceStorage
Coordinates local-spool-first persistence with optional upload to a MinIO/S3 remote backend.
OracleTraceSyncStats
Counters from a single sync pass over the local spool.
QueryTemplate
A query template for oracle-friendly trace generation.
SemanticPayload
Semantic/free-form text payload (unverifiable).
SplitWriteStats
Statistics from split-write operations.
TraceStep
A single tool invocation recorded during an RLM REPL session.
TraceValidator
Main trace validator for RLM REPL outputs.
TreeSitterOracle
Tree-sitter based oracle for validating structural queries.
ValidatedTrace
Complete snapshot of a validated RLM REPL session.

Enums§

FinalAnswerFormat
Classification of an RLM FINAL() answer format.
FinalPayload
The top-level FINAL() payload envelope.
GrepVerification
Result of grep oracle verification.
OracleResult
Outcome of oracle validation for a single REPL session.
QueryType
Query type classification for routing to the appropriate oracle.
TemplateKind
The kind of oracle this template is designed for.
TreeSitterVerification
Result of tree-sitter oracle verification.
VerificationMethod
Identifies which oracle backend produced a verdict.

Functions§

default_spool_dir
Resolve the directory where oracle traces are buffered before upload to remote storage.