Module chunk_rows 

Rows per chunk on the background write paths (§5.1.5, D-011, D-014, D-058).

The Write Actor holds the sole write connection, so a single large statement blocks every other writer for its duration. Chunking bounds that stall; the cost is that a bulk import is not atomic across chunks, which is why it is a separate command from HighPriCommand::WriteBulkAtomic rather than a tuning parameter on it.

Why these are four constants and not one

Through 0.5.5 this was a single CHUNK_ROWS = 1000 for all four bulk paths. The golden rule it was meant to serve is a bound on duration — a background chunk must commit fast enough that an interactive write queued behind it is not made to wait — and one row count cannot express one duration across paths whose measured per-row costs differ by 60× (D-058). At 1,000 rows the four paths took 3.5 ms, 24 ms, 89 ms and 143 ms: the same constant, four answers, three of them far outside the bound.

Each size below is derived from benches/budgets.rs’s chunk_scaling sweep against CHUNK_BUDGET, then verified by measuring that size directly. They are measurements of this machine, not universal constants — D-055’s reasoning about reference hardware applies here too, and re-deriving them on materially different storage is a cargo bench away.

Sized for the tail, not the median

The first derivation solved f + c·n = 3 ms exactly and produced sizes whose median commit was 2.93 ms and whose upper estimate was 2.96 — inside the bound as reported and outside it for any chunk slower than typical. A latency bound is a statement about the chunk an unlucky interactive write actually queues behind, so these solve for ≈2.5 ms instead, leaving the remainder as headroom for the tail. That costs a few percent of throughput on the two linear paths and nothing on the two superlinear ones.

As measured by chunk_budget, each at its own size: edges 2.39 ms, concepts 2.35 ms, annotations 2.36 ms, embeddings 2.06 ms, no upper estimate above 2.42.

Known limitation: these are empty-database figures

chunk_budget seeds concepts and starts with no links and no vectors, and D-059 established that per-row cost on the edge and embedding paths grows with the size of the structure being written, not with the chunk. The same 90-edge chunk takes 9.06 ms into an 8,000-edge table. So the bound is met as measured here and not met on a populated database.

That gap was published as 47.7 ms until 0.10.0 and attributed to the schema defect D-059 documents. The defect was fixed by the v5 → v6 rung and the figure was never updated. 9.08 ms is a 0.10.0 measurement, not D-059’s 8.0 ms carried forward: chunk_budget gained a seeded arm, because until it did, nothing in the bench suite wrote a chunk into a populated table and this number was unfalsifiable. It agrees with D-059 once the session is accounted for — the empty arm read 2.69 and 2.65 ms beside it against the 2.39 ms published above, so the ratio is 3.4× here and 3.35× there.

The residual is attributed as of 0.11.0 (D-142). It is not the missing index, which shipped in 0.5.6; it is the links_current write. Dropping the three links insert triggers one at a time puts effectively all of the growth in trg_links_current_sync — the single-open guard contributes none, the log trigger and the base insert ~0.35 ms of a 4.15 ms rise — and within that trigger, 89% of the growth is maintenance of idx_lc_traversal_cover and idx_lc_open_interval rather than the upsert itself, which costs 0.49 ms run directly against the same table. Page-cache size, foreign keys and the fixture’s key distribution were each tested and are each not the cause.

Knowing the cause does not by itself change the constant: the expensive index is D-042’s covering index for the traversal, so narrowing it moves cost onto the read path it exists to protect. Re-deriving these constants against the D-088 fixture matrix is the named successor.

These are ceilings as of 0.12.0, not sizes

D-143 re-derived all four against the D-088 matrix and the edge path came back 20 against a shipped 90 — and 20 would have been wrong at 80,000 edges for the same reason 90 is wrong at 8,000, because per-row cost there grows with links_current. The finding was that no row count can bound a duration on such a path.

So the chunk loop stopped trying to pick one ahead of time. Each chunk is timed by the actor and its measured hold chooses the next size; these constants are the largest size that will ever be asked for, and every derivation below still applies to them as such. A path may run well under its constant on a populated database and at exactly it on an empty one, and both are the bound being met rather than a size being missed.

Constants

ANNOTATIONS

Analytics annotations (write_analytics_annotations).

CONCEPTS

Concept upserts (write_concepts).

EDGES

Edge assertions (bulk_import).

EMBEDDINGS

Embedding vectors (upsert_embeddings).