Expand description
Content-hash idempotency ledger for zeph knowledge ingest.
The ledger records every (source_uri, content_hash) pair that has been successfully
ingested, so unchanged inputs are skipped on subsequent runs (spec-067 FR-012, INV-5).
§Scope and limitations
This is a re-read / cost guard only — it prevents redundant embedding and LLM extraction for inputs whose content has not changed since the last run. It does not:
- Reconcile LLM extraction drift across model versions. If the model changes, re-ingest must be forced by clearing ledger rows for the affected sources.
- Delete stale Qdrant points when a file’s content changes. When
content_hashchanges for the same path, the old vectors remain in Qdrant (orphaned). Stale-point cleanup is a Phase-2 concern and is documented as a known MVP limitation in the testing playbook.
§Database
The ledger shares the agent’s existing SQLite pool (SemanticMemory::sqlite().pool()). It is
operator-triggered, one row per ingested file, and is not on the hot path — no dedicated file
is needed (mirrors the skill_trace_sessions pattern).
§Examples
use zeph_memory::graph::ingest::IngestLedger;
// In practice, pool comes from SemanticMemory::sqlite().pool().clone().
let ledger = IngestLedger::new(pool);
let hash = IngestLedger::content_hash(b"hello world");
let uri = "specs/README.md@abc123";
let batch = "01J0000000000000000000000";
ledger.mark_ingested(uri, &hash, batch, 0, 0).await?;
assert!(ledger.is_ingested(uri, &hash).await?);Structs§
- Ingest
Ledger - Content-hash idempotency ledger for
zeph knowledge ingest. - Ledger
Entry - A single row from the
knowledge_ingest_ledgertable.
Enums§
- Batch
IdResolution - Outcome of resolving a (possibly abbreviated) batch id prefix against the ledger.