Struct SteelDb 

pub struct SteelDb { /* private fields */ }

An indexed corpus you can ask questions of.

Read-only after construction and Send + Sync, so one instance can serve many threads.

Implementations

impl SteelDb

pub fn ingest<I, S>(docs: I) -> Result<Self, Error>

where I: IntoIterator<Item = S>, S: AsRef<str>,

Ingest. Discover a vocabulary from documents and index them.

Offline and deterministic: no credentials, no network, no model files. The same documents always give the same vocabulary.

Examples found in repository

examples/quickstart.rs (line 18)

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let docs = [
        "Morty Shade defeated Wallace Gale at Ecruteak City during the Indigo Invitational in 2025.",
        "Bea Strike defeated Iris Draco at Ecruteak City during the Indigo Invitational in 2025.",
        "Juan Tide defeated Cynthia Ward at Ecruteak City during the Indigo Invitational in 2026.",
        "A habitat survey recorded Aggron near Sootopolis City at an elevation of 1082 m.",
        "A habitat survey recorded Salamence near Sootopolis City at an elevation of 2369 m.",
        "A habitat survey recorded Milotic near Sootopolis City at an elevation of 340 m.",
        "Milotic is not permitted in Series 1 play for the 2025 season.",
        "Metagross is permitted in Series 4 play for the 2026 season.",
    ];

    let db = SteelDb::ingest(docs)?;
    println!("indexed {} situations", db.len());

    println!("\nyou can ask about:");
    for c in db.categories() {
        println!("  {:<14} {}", c.wildcard(), c.words.iter().take(4).cloned().collect::<Vec<_>>().join(", "));
    }

    // use a category the corpus actually produced — see the list printed above
    let q = "(and elevation/* (not state/negated))";
    match db.query(q) {
        Ok(a) => {
            println!("\n{q}  ->  {} situations", a.len());
            for (id, text) in db.resolve(&a).take(2) {
                // chars(), not a byte slice: `&text[..n]` panics mid-character
                println!("  [{id}] {}", text.chars().take(56).collect::<String>());
            }
        }
        Err(r) => println!("\n{r}"),
    }

    // a category this corpus does not have — refused, not answered with an empty list
    match db.query("survey/*") {
        Ok(_) => println!("\nunexpected: survey is not a category here"),
        Err(r) => println!("\n{r}"),
    }

    match db.query("gene/brca1") {
        Ok(_) => println!("\nunexpected: that should not exist"),
        Err(r) => println!("\n{r}"),
    }

    println!("\nbelief in a denied claim: {}", db.belief("state/negated"));
    Ok(())
}

pub fn ingest_with<I, S>(docs: I, opts: Options) -> Result<Self, Error>

where I: IntoIterator<Item = S>, S: AsRef<str>,

As SteelDb::ingest, with explicit discovery settings.

pub fn ingest_curated<I, S>( docs: I, curated: &Proposal, surfaces: &[String], ) -> Result<Self, Error>

where I: IntoIterator<Item = S>, S: AsRef<str>,

Index a corpus against a curated ontology. The third stage of neural discovery.

The full pipeline is three explicit steps, because each is a different kind of work:

use steeldb::{SteelDb, learn::Teacher, tagger_discover};

// 1. mechanical: the tagger reads typed spans, transport groups them into RAW clusters
let sample: Vec<String> = documents.iter().take(100).cloned().collect();
let raw = tagger_discover::discover(&sample, 12, 8)?;

// 2. judgment: merge synonyms, drop noise, name each surviving facet as a KIND
let curated = Teacher::ollama("qwen3:1.7b")?.curate(&raw).await?;

// 3. deterministic: gate the facets and index the FULL corpus, model-free
let db = SteelDb::ingest_curated(documents, &curated, &raw.surfaces())?;

Step 2 cannot be skipped. A raw cluster’s label is a value, not a facet: the reference’s own committed spec has clusters called ph and ge, and only curation turns those into weapon-platform and control-system with the raw terms demoted to examples. Indexing raw clusters directly puts ph in the index as a retrieval dimension.

The models run once, over the sample. What this produces is the ordinary artefact vocabulary, and the whole corpus is then projected by it with the model-free matcher — so SteelDb::query stays model-free, and after SteelDb::save later runs need no model at all.

surfaces is the observed entity spans from the raw spec, which becomes the gazetteer. It is kept separate from curation deliberately: the surfaces are what the tagger actually saw, so they survive whatever the curator chooses to merge or drop.

pub fn ingest_using<I, S>( docs: I, artifact_dir: impl AsRef<Path>, ) -> Result<Self, Error>

where I: IntoIterator<Item = S>, S: AsRef<str>,

Ingest, following an existing artefact set.

Reads the vocabulary from artifact_dir instead of rediscovering it, so the result is reproducible and no model is involved even if a model produced the vocabulary originally. This is the pairing that makes learn worth running: the expensive, non-deterministic step happens once, and every run afterwards is offline and identical.

pub fn save(&self, artifact_dir: impl AsRef<Path>) -> Result<(), Error>

Write this database’s vocabulary to an artefact directory.

Only derived vocabulary is written — categories, mention surfaces, relation verbs. No document text, so the directory is safe to commit alongside code.

pub fn save_with_training( &self, artifact_dir: impl AsRef<Path>, ) -> Result<usize, Error>

As SteelDb::save, and additionally write a finetuning set derived from the corpus.

The set is weak supervision: every span the discovered vocabulary can locate, labelled with the category that claims it. It is what you would hand to a tagger finetune so the model learns to find these spans in text it has not seen.

Unlike the vocabulary files, this one contains document text — a span label is meaningless without the words it points at. It is written to a training/ subdirectory which crate::artifact::Artifacts::save excludes with a .gitignore, and the manifest records that the subdirectory is unsafe to publish.

pub fn open(dir: impl AsRef<Path>) -> Result<Self, Error>

Index a directory of documents, discovering the vocabulary from what it finds.

pub fn query(&self, ikl: &str) -> Result<Answer, Refused>

Run a query, or refuse it.

Every tag is checked against the vocabulary before anything executes, so an unsupported query costs nothing and comes back with alternatives.

Examples found in repository

examples/quickstart.rs (line 28)

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let docs = [
        "Morty Shade defeated Wallace Gale at Ecruteak City during the Indigo Invitational in 2025.",
        "Bea Strike defeated Iris Draco at Ecruteak City during the Indigo Invitational in 2025.",
        "Juan Tide defeated Cynthia Ward at Ecruteak City during the Indigo Invitational in 2026.",
        "A habitat survey recorded Aggron near Sootopolis City at an elevation of 1082 m.",
        "A habitat survey recorded Salamence near Sootopolis City at an elevation of 2369 m.",
        "A habitat survey recorded Milotic near Sootopolis City at an elevation of 340 m.",
        "Milotic is not permitted in Series 1 play for the 2025 season.",
        "Metagross is permitted in Series 4 play for the 2026 season.",
    ];

    let db = SteelDb::ingest(docs)?;
    println!("indexed {} situations", db.len());

    println!("\nyou can ask about:");
    for c in db.categories() {
        println!("  {:<14} {}", c.wildcard(), c.words.iter().take(4).cloned().collect::<Vec<_>>().join(", "));
    }

    // use a category the corpus actually produced — see the list printed above
    let q = "(and elevation/* (not state/negated))";
    match db.query(q) {
        Ok(a) => {
            println!("\n{q}  ->  {} situations", a.len());
            for (id, text) in db.resolve(&a).take(2) {
                // chars(), not a byte slice: `&text[..n]` panics mid-character
                println!("  [{id}] {}", text.chars().take(56).collect::<String>());
            }
        }
        Err(r) => println!("\n{r}"),
    }

    // a category this corpus does not have — refused, not answered with an empty list
    match db.query("survey/*") {
        Ok(_) => println!("\nunexpected: survey is not a category here"),
        Err(r) => println!("\n{r}"),
    }

    match db.query("gene/brca1") {
        Ok(_) => println!("\nunexpected: that should not exist"),
        Err(r) => println!("\n{r}"),
    }

    println!("\nbelief in a denied claim: {}", db.belief("state/negated"));
    Ok(())
}

pub fn check(&self, ikl: &str) -> Result<(), Refused>

Check a query without running it. Cheap, and the same check query performs.

pub fn belief(&self, tag: &str) -> Interval

The evidential bound on a tag across the whole corpus.

Examples found in repository

examples/quickstart.rs (line 50)

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let docs = [
        "Morty Shade defeated Wallace Gale at Ecruteak City during the Indigo Invitational in 2025.",
        "Bea Strike defeated Iris Draco at Ecruteak City during the Indigo Invitational in 2025.",
        "Juan Tide defeated Cynthia Ward at Ecruteak City during the Indigo Invitational in 2026.",
        "A habitat survey recorded Aggron near Sootopolis City at an elevation of 1082 m.",
        "A habitat survey recorded Salamence near Sootopolis City at an elevation of 2369 m.",
        "A habitat survey recorded Milotic near Sootopolis City at an elevation of 340 m.",
        "Milotic is not permitted in Series 1 play for the 2025 season.",
        "Metagross is permitted in Series 4 play for the 2026 season.",
    ];

    let db = SteelDb::ingest(docs)?;
    println!("indexed {} situations", db.len());

    println!("\nyou can ask about:");
    for c in db.categories() {
        println!("  {:<14} {}", c.wildcard(), c.words.iter().take(4).cloned().collect::<Vec<_>>().join(", "));
    }

    // use a category the corpus actually produced — see the list printed above
    let q = "(and elevation/* (not state/negated))";
    match db.query(q) {
        Ok(a) => {
            println!("\n{q}  ->  {} situations", a.len());
            for (id, text) in db.resolve(&a).take(2) {
                // chars(), not a byte slice: `&text[..n]` panics mid-character
                println!("  [{id}] {}", text.chars().take(56).collect::<String>());
            }
        }
        Err(r) => println!("\n{r}"),
    }

    // a category this corpus does not have — refused, not answered with an empty list
    match db.query("survey/*") {
        Ok(_) => println!("\nunexpected: survey is not a category here"),
        Err(r) => println!("\n{r}"),
    }

    match db.query("gene/brca1") {
        Ok(_) => println!("\nunexpected: that should not exist"),
        Err(r) => println!("\n{r}"),
    }

    println!("\nbelief in a denied claim: {}", db.belief("state/negated"));
    Ok(())
}

pub fn s_path(&self, from: &str, to: &str, s: usize) -> Answer

Situations on a chain from from to to where each step shares at least s tags.

Raising s demands more agreement per step, which is what stops a walk drifting somewhere unrelated.

pub fn filtration(&self, max_s: usize) -> Vec<Level>

Both readings of the incidence matrix, swept over the overlap threshold.

pub fn categories(&self) -> Vec<Category<'_>>

The discovered categories and the words each claims.

Examples found in repository

examples/quickstart.rs (line 22)

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let docs = [
        "Morty Shade defeated Wallace Gale at Ecruteak City during the Indigo Invitational in 2025.",
        "Bea Strike defeated Iris Draco at Ecruteak City during the Indigo Invitational in 2025.",
        "Juan Tide defeated Cynthia Ward at Ecruteak City during the Indigo Invitational in 2026.",
        "A habitat survey recorded Aggron near Sootopolis City at an elevation of 1082 m.",
        "A habitat survey recorded Salamence near Sootopolis City at an elevation of 2369 m.",
        "A habitat survey recorded Milotic near Sootopolis City at an elevation of 340 m.",
        "Milotic is not permitted in Series 1 play for the 2025 season.",
        "Metagross is permitted in Series 4 play for the 2026 season.",
    ];

    let db = SteelDb::ingest(docs)?;
    println!("indexed {} situations", db.len());

    println!("\nyou can ask about:");
    for c in db.categories() {
        println!("  {:<14} {}", c.wildcard(), c.words.iter().take(4).cloned().collect::<Vec<_>>().join(", "));
    }

    // use a category the corpus actually produced — see the list printed above
    let q = "(and elevation/* (not state/negated))";
    match db.query(q) {
        Ok(a) => {
            println!("\n{q}  ->  {} situations", a.len());
            for (id, text) in db.resolve(&a).take(2) {
                // chars(), not a byte slice: `&text[..n]` panics mid-character
                println!("  [{id}] {}", text.chars().take(56).collect::<String>());
            }
        }
        Err(r) => println!("\n{r}"),
    }

    // a category this corpus does not have — refused, not answered with an empty list
    match db.query("survey/*") {
        Ok(_) => println!("\nunexpected: survey is not a category here"),
        Err(r) => println!("\n{r}"),
    }

    match db.query("gene/brca1") {
        Ok(_) => println!("\nunexpected: that should not exist"),
        Err(r) => println!("\n{r}"),
    }

    println!("\nbelief in a denied claim: {}", db.belief("state/negated"));
    Ok(())
}

pub fn askable(&self) -> Vec<String>

The wildcard for every discovered category — the set of things you can ask about.

pub fn tags(&self) -> BTreeMap<String, Vec<String>>

Every tag in the index, grouped by its category and sorted within each group.

Sorted because the engine is otherwise deterministic and a caller should not have to defend against index iteration order — two runs over the same documents return byte-identical output.

pub fn text(&self, situation: u32) -> Option<&str>

The document behind a situation id.

Without this an Answer is a list of integers. Every result is traceable back to the text that produced it, which is what makes an answer checkable rather than merely plausible.

pub fn resolve<'a>( &'a self, answer: &'a Answer, ) -> impl Iterator<Item = (u32, &'a str)> + 'a

The documents an answer refers to, in id order.

Examples found in repository

examples/quickstart.rs (line 31)

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let docs = [
        "Morty Shade defeated Wallace Gale at Ecruteak City during the Indigo Invitational in 2025.",
        "Bea Strike defeated Iris Draco at Ecruteak City during the Indigo Invitational in 2025.",
        "Juan Tide defeated Cynthia Ward at Ecruteak City during the Indigo Invitational in 2026.",
        "A habitat survey recorded Aggron near Sootopolis City at an elevation of 1082 m.",
        "A habitat survey recorded Salamence near Sootopolis City at an elevation of 2369 m.",
        "A habitat survey recorded Milotic near Sootopolis City at an elevation of 340 m.",
        "Milotic is not permitted in Series 1 play for the 2025 season.",
        "Metagross is permitted in Series 4 play for the 2026 season.",
    ];

    let db = SteelDb::ingest(docs)?;
    println!("indexed {} situations", db.len());

    println!("\nyou can ask about:");
    for c in db.categories() {
        println!("  {:<14} {}", c.wildcard(), c.words.iter().take(4).cloned().collect::<Vec<_>>().join(", "));
    }

    // use a category the corpus actually produced — see the list printed above
    let q = "(and elevation/* (not state/negated))";
    match db.query(q) {
        Ok(a) => {
            println!("\n{q}  ->  {} situations", a.len());
            for (id, text) in db.resolve(&a).take(2) {
                // chars(), not a byte slice: `&text[..n]` panics mid-character
                println!("  [{id}] {}", text.chars().take(56).collect::<String>());
            }
        }
        Err(r) => println!("\n{r}"),
    }

    // a category this corpus does not have — refused, not answered with an empty list
    match db.query("survey/*") {
        Ok(_) => println!("\nunexpected: survey is not a category here"),
        Err(r) => println!("\n{r}"),
    }

    match db.query("gene/brca1") {
        Ok(_) => println!("\nunexpected: that should not exist"),
        Err(r) => println!("\n{r}"),
    }

    println!("\nbelief in a denied claim: {}", db.belief("state/negated"));
    Ok(())
}

pub fn len(&self) -> usize

How many situations are indexed.

Examples found in repository

examples/quickstart.rs (line 19)

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let docs = [
        "Morty Shade defeated Wallace Gale at Ecruteak City during the Indigo Invitational in 2025.",
        "Bea Strike defeated Iris Draco at Ecruteak City during the Indigo Invitational in 2025.",
        "Juan Tide defeated Cynthia Ward at Ecruteak City during the Indigo Invitational in 2026.",
        "A habitat survey recorded Aggron near Sootopolis City at an elevation of 1082 m.",
        "A habitat survey recorded Salamence near Sootopolis City at an elevation of 2369 m.",
        "A habitat survey recorded Milotic near Sootopolis City at an elevation of 340 m.",
        "Milotic is not permitted in Series 1 play for the 2025 season.",
        "Metagross is permitted in Series 4 play for the 2026 season.",
    ];

    let db = SteelDb::ingest(docs)?;
    println!("indexed {} situations", db.len());

    println!("\nyou can ask about:");
    for c in db.categories() {
        println!("  {:<14} {}", c.wildcard(), c.words.iter().take(4).cloned().collect::<Vec<_>>().join(", "));
    }

    // use a category the corpus actually produced — see the list printed above
    let q = "(and elevation/* (not state/negated))";
    match db.query(q) {
        Ok(a) => {
            println!("\n{q}  ->  {} situations", a.len());
            for (id, text) in db.resolve(&a).take(2) {
                // chars(), not a byte slice: `&text[..n]` panics mid-character
                println!("  [{id}] {}", text.chars().take(56).collect::<String>());
            }
        }
        Err(r) => println!("\n{r}"),
    }

    // a category this corpus does not have — refused, not answered with an empty list
    match db.query("survey/*") {
        Ok(_) => println!("\nunexpected: survey is not a category here"),
        Err(r) => println!("\n{r}"),
    }

    match db.query("gene/brca1") {
        Ok(_) => println!("\nunexpected: that should not exist"),
        Err(r) => println!("\n{r}"),
    }

    println!("\nbelief in a denied claim: {}", db.belief("state/negated"));
    Ok(())
}

pub fn is_empty(&self) -> bool

pub fn documents(&self) -> &[String]

The documents as given, for tracing a result back to its source.

impl SteelDb

pub fn adopt(&mut self, proposal: &Proposal) -> Vec<Verdict>

Apply a proposal, keeping only what the MECE gate accepts.

The gate is the same one SteelDb::ingest uses, so a model-proposed category has to earn its place on the same terms as a locally-discovered one: enough coverage, and not a near-duplicate of something already present. Candidates are judged in order, so the second of two similar suggestions is rejected against the first.

Returns a verdict per candidate; the kept ones are queryable immediately.