pub const CREATE_CONCEPTS_FTS: &str = r#"
CREATE VIRTUAL TABLE IF NOT EXISTS concepts_fts USING fts5(
title,
content,
content='concepts',
content_rowid='rowid'
);
"#;Expand description
The keyword half of hybrid search: an FTS5 index over concept text (§5.9).
External content. The table declares content='concepts', so the tokens
are indexed but the text itself is not duplicated — FTS5 reads it back from
concepts by rowid when it needs a column value. Two reasons beyond the
storage saving, and the second is the one that decided it:
- There is exactly one copy of the text, so the index cannot disagree with the concept about what the concept says. A standalone FTS table would be a second description of data the ledger already holds, which is the failure class D-030 and D-035 exist to prevent.
INSERT INTO concepts_fts(concepts_fts) VALUES('rebuild')reconstructs the whole index from the content table in one statement. D-036 requires every derivative table to be rebuildable from the ledger, and here that is the engine’s own operation rather than code of ours that has to be kept honest.
The cost is that external-content tables do not maintain themselves: an
UPDATE must retract the old terms before adding the new ones, using the
old column values. That is what trg_concepts_fts_update does, and getting
it wrong leaves an index that still matches text no concept contains.