Skip to main content

Module shadow

Module shadow 

Source
Expand description

Rebuilding links_current beside itself, in chunks (T1.2, D-082).

§What this is for

rebuild_current is one BEGIN IMMEDIATE … COMMIT holding the write lock for its whole duration — measured at 318 ms for 40,000 rows in links (D-077), and D-023 is why it cannot simply be split: the window between the DELETE and the INSERT is the entire of current belief, and a reader landing in it sees a graph with no edges and no error.

Building the replacement beside the live table removes that window. The live table stays live and trigger-maintained throughout, so readers and trg_links_single_open keep working, and the only moment anything is unavailable is the swap.

§Two things about this are easy to get wrong, and one of them is silent

CREATE TABLE … AS SELECT does not carry the schema. The obvious way to make a shadow copies the rows and nothing else: no primary key, no CHECK constraints, no indexes. The swap then succeeds, the rename succeeds, and the next INSERT INTO links fails inside trg_links_current_sync with ON CONFLICT clause does not match any PRIMARY KEY or UNIQUE constraint — because the conflict target no longer exists. Probed on libSQL 0.9.30; the projection had stopped being maintained and the only symptom was an error on an unrelated write. The shadow is therefore created from CREATE_LINKS_CURRENT_TABLE with the name substituted, so it cannot drift from the declared table.

The rename reparses the whole schema. ALTER TABLE … RENAME (SQLite ≥ 3.25) re-resolves every trigger body, and both links triggers name links_current — so the rename fails with error in trigger trg_links_current_sync: no such table: main.links_current while they exist. Probed. The order that works, also probed, is DROP TRIGGERDROP TABLERENAMECREATE INDEX → recreate triggers. PRAGMA legacy_alter_table=ON also works and is not used: it disables the reference fixups the modern rename exists to perform.

§Why the indexes are built inside the swap and not on the shadow

This is the one place the shape is dictated by SQLite rather than chosen. Index names are global, so the shadow cannot carry idx_lc_traversal_cover while the live table still holds that name — and SQLite has no ALTER INDEX … RENAME. Building them on the shadow under temporary names would leave links_current permanently indexed under names that do not appear in CREATE_INDICES, so the next migration would create a second copy of each.

DROP TABLE links_current frees the names, and they are reusable within the same transaction (probed). So the swap pays the index builds, and what the chunking buys is that the projection — the window function over all of links, which is the O(E log E) term — happens outside the lock. That is a smaller win than “the swap is microseconds”, which is what the naive reading of the shadow idea promises, and it is the real one.

Enums§

ShadowOutcome
What a ShadowStep produced.
ShadowStep
One step of a chunked rebuild, as sent to the actor.