Skip to main content

Module artifact

Module artifact 

Source
Expand description

Artefacts — what learn leaves behind, and what ingest and query follow.

learn is the only step that needs a model. Running it on every startup would make the system slow, non-deterministic, and dependent on a service being reachable. So it runs once and writes its result to a directory of small JSON files; every run after that reads those files and is offline and reproducible.

.hypersteeldb/
  manifest.json      what produced this set, when, and a digest of each part
  vocabulary.json    the categories and the words each claims
  gazetteer.json     multi-word mentions kept whole
  relations.json     relation verbs, with polarity
  motifs.json        latent motifs, so `motif/*` tokens survive a reload
use steeldb::SteelDb;

// once — discovers (or learns) and records the result
let db = SteelDb::ingest(["…documents…"])?;
db.save(".hypersteeldb")?;

// every time after — same vocabulary, no model, no network, byte-identical results
let db = SteelDb::ingest_using(["…documents…"], ".hypersteeldb")?;

§What is deliberately not in here

No document text. An artefact set is derived vocabulary: category names, the words that signal them, mention surfaces, relation verbs. It is not a copy of the corpus, and it is not an index. Two reasons that matters: an artefact set is safe to commit to a repository next to code, and shipping one does not ship the documents it was learned from. A test asserts the files contain no sentence from the corpus.

You still need the documents to ingest. The artefacts say how to read them, not what they said.

Structs§

Artifacts
The complete set. Small, human-readable, and diffable on purpose — a vocabulary change should show up in review like any other change.
CategoryRecord
One category and the words that put a document in it.
Registration
A registered entity mention: how it appears, and what it resolves to.

Enums§

ArtifactError
Why an artefact set could not be read or written.

Constants§

GAZETTEER
IGNORE_FILE
MANIFEST
File names inside an artefact directory.
MOTIFS
Latent motifs: the themes a situation can join without sharing a keyword (the paper’s sixth dimension).
RELATIONS
SCHEMA
Current on-disk layout version. Bumped when a field’s meaning changes, so an old set is refused rather than silently misread.
TRAINING_DIR
Subdirectory for the finetuning set. Separate because, unlike everything else here, it necessarily contains document text.
TRAINING_FILE
VOCABULARY