Skip to main content

CREATE_LOG_INTEGRITY_TABLE

Constant CREATE_LOG_INTEGRITY_TABLE 

Source
pub const CREATE_LOG_INTEGRITY_TABLE: &str = r#"
    CREATE TABLE IF NOT EXISTS log_integrity (
        id INTEGER PRIMARY KEY CHECK (id = 1),
        rows_removed INTEGER NOT NULL DEFAULT 0 CHECK (rows_removed IN (0, 1))
    )
"#;
Expand description

Derived analytics output, keyed by concept and label (§5.4, D-041).

Deliberately outside the ledger. Three properties are load-bearing and each is the opposite of what the four normative tables above do.

No log trigger. Nothing in CREATE_TRIGGERS fires on this table, so an annotation never reaches transaction_log. That is Doctrine VII’s reasoning about embeddings applied to the other derived artifact: a community label is a function of an algorithm, a version of that algorithm, and a graph — not a statement about the world, and a ledger that records it is recording the analytics schedule as though it were history. A reconstruction that wants labels recomputes them, which is the only honest way to ask what a past graph’s communities were.

No delete guard. Doctrine V protects the hot ledger tables; this table is derivative state in Doctrine VI’s second category, so wiping it must stay a legal, ordinary operation — a rerun replaces the previous pass, and dropping the whole table costs nothing but the recomputation.

Upsert on (concept_id, label). One current value per label per concept. Storing a history of successive runs here would be the ledger again, by another name.

The foreign key is safe in a way links_current’s omitted ones are not: concepts are never physically deleted (D-022), and this table is rebuilt by re-running an algorithm that read concepts in the first place, so there is no insertion-order problem to solve. One row, one bit: has anything ever been deleted from transaction_log? (v16, 0.15.7, W14.5, D-249, review C-5.)

§Why this is a table and not a query

The bit is temporal::replay’s reach guard, and until v16 the guard computed it: MIN(seq_id) = 1 AND COUNT(*) = MAX(seq_id), exact because seq_id is INTEGER PRIMARY KEY AUTOINCREMENT and never reused. The MIN and MAX are index seeks; the COUNT(*) is a scan of the whole log, and it ran on every recorded-time read below the newest surviving stamp. Measured (examples/log_integrity_probe.rs): 0.134 ms at 2,000 rows and 32.6 ms at 500,000, against an id-bounded hydration that is flat at 0.14 ms whatever the log holds. Reading this row instead is 0.033 ms at every size.

There is no cheaper exact query. LOG_ARCHIVABLE removes superseded rows wherever they sit, so a gap can be anywhere in the sequence and only counting finds it. What there is instead is a fact the storage already knows at the moment it becomes true, and did not write down.

§Why a trigger and not the archive code

CREATE_TXLOG_MARK_GAP maintains it, so the bit is a property of the table rather than of the crate’s archive path. §4.2 admits that raw SQL against the same file can do what this API refuses; a bit maintained in Rust would be wrong after exactly that, and wrong in the direction that folds a gap silently. A trigger is wrong in neither direction, because there is no route to deleting a log row that does not pass through it.

§The seed is computed, not assumed

A database arriving at v16 may already have gaps, so SEED_LOG_INTEGRITY derives the initial value from the log’s own sqlite_sequence high-water mark. That test is exact where the guard’s old COUNT(*) = MAX(seq_id) was exact and in one state where it was not — a hot log archived down to nothing, which the old form called intact. See the_bit_agrees_with_the_count_it_replaced.