Skip to main content

Module locomo

Module locomo 

Source
Expand description

LoCoMo memory benchmark (#291).

Long-conversation memory, measured the way lean-ctx works: store every turn as a memory, then for each question recall the top-k memories and score the recalled context against the gold answers (token-F1 / exact-match / answer containment), plus the token cost of the recalled context versus dumping the whole transcript.

Pipeline: dataset (load) → runner (ingest + recall + score) → report (aggregate to publishable numbers). Run via the locomo_bench example: cargo run --example locomo_bench --features dev-tools.

Modules§

dataset
LoCoMo benchmark dataset schema + loader (#291).
report
Aggregate per-question results into publishable LoCoMo metrics (#291).
runner
Run a LoCoMo sample through lean-ctx memory: ingest every turn as a knowledge fact, then for each question recall the top-k memories and score the recalled context against the gold answers (#291).

Functions§

run
Run a suite end-to-end and aggregate a report.