Expand description
SEMQ: deterministic symbolic encoding of float32 vectors.
Five values and one service, backed by a C core that owns every rule:
CodecConfig: the rule (quant,phaseororbit).Codec: encodes unit-norm float32 vectors into anEncoding.Encoding: ids, canonical rows, a manifest and two identities;write/read,concatanddiff.Diff: added, removed and changed ids with hamming distances.Floor: the envelope of variation observed in null diffs, bound to the config, id kind and reference it was measured against.BuildInfofrombuild_info.
Errors are the six variants of Error. Handles are freed on drop and
are Send + Sync; the views into an Encoding borrow it.
§Quickstart
use std::collections::BTreeMap;
use std::fs::File;
use semq::{Codec, CodecConfig, Encoding, Floor};
fn main() -> Result<(), Box<dyn std::error::Error>> {
// The rule: quant, 4 dimensions, 4 magnitude bins per sign.
let config = CodecConfig::quant(4, 4)?;
let codec = Codec::new(&config)?;
// Unit-norm float32 rows, one per id, in any order.
let ids = [3u64, 1, 2];
let vectors = [
0.5f32, 0.5, 0.5, 0.5, //
-0.5, -0.5, -0.5, -0.5, //
1.0, 0.0, 0.0, 0.0,
];
let manifest = BTreeMap::from([("encoder".to_string(), "my-model@1".to_string())]);
let reference = codec.encode(&ids, &vectors, Some(&manifest))?;
// Save, then load in another process: the identity is the same.
reference.write(File::create("reference.semq")?)?;
let loaded = Encoding::read(File::open("reference.semq")?)?;
assert_eq!(loaded.state_id(), reference.state_id());
// Measure the floor from a null rebuild of the reference, then
// compare a candidate against it.
let null = loaded.diff(&codec.encode(&ids, &vectors, Some(&manifest))?)?;
let floor = Floor::measure([&null])?;
let candidate = codec.encode(&ids, &vectors, Some(&manifest))?;
let diff = loaded.diff(&candidate)?;
println!(
"changed {} of {} shared rows; within floor: {}",
diff.changed().len(),
diff.n_unchanged() + diff.changed().len() as u64,
diff.within(&floor)?
);
Ok(())
}Structs§
- Build
Info - Versions and kernel selection of the linked core.
- Codec
- An immutable encoder for one
CodecConfig, safe to share between threads. Freed on drop. - Codec
Config - An immutable value: operator, dimension and the operator’s parameter.
- Diff
- The result of
Encoding::diff. - Diff
Report - A
Diffas plain data: the report schema without a serializer. - Encoding
- An immutable set of ids with one canonical row each, a manifest and two identities.
- Floor
- An envelope of observed variation, bound to the context it was measured
in: the config, the id kind and the
state_idof the reference every null diff was taken against. - Floor
Report - A
Flooras plain data: thesemq-floor/1schema without a serializer. - Gate
Options - Which checks
Diff::evaluateapplies beyond those ofDiff::within, and which dataFloor::measure_forrecords for them. Every check is off by default, so a new check never changes an existing verdict. - Ids
- A borrowed sequence of ids of one kind.
- Verdict
- The result of
Diff::evaluate: whether the candidate passed, every check that failed, and the rows the per-row check flagged.
Enums§
- Error
- An error from the SDK. Each core status maps to exactly one variant.
- Id
- One owned id of either kind.
- IdKind
- The two id kinds. One per Encoding, never mixed.
- Operator
- The three operators. Values are pinned in the canonical form.
- Reason
- A check that failed in a
Verdict. - Which
- Which integrity check failed on load: the check, not a cause.
Functions§
- build_
info - Query the linked core.