Expand description
Cursor persistence for stream positions.
Stores the last-read stream ID for each peer in SQLite. This survives Redis restarts (we’re not relying on AOF!) and daemon restarts.
§Debounced Writes
To reduce SQLite write pressure, cursors are debounced:
set()updates the in-memory cache immediately and marks the cursor dirtyflush_dirty()persists all dirty cursors to disk in a batch- The coordinator calls
flush_dirty()periodically (every few seconds) - On shutdown,
flush_dirty()is called to ensure no data loss
This means a crash between set() and flush_dirty() could lose up to
one flush interval of cursor progress. On restart, we’d re-read some
events that were already applied (idempotent, safe).
§SQLite Busy Handling
SQLite can return SQLITE_BUSY/SQLITE_LOCKED when the database is contended. We handle this with:
- Automatic retry with exponential backoff
- Configurable max retries (default 5)
- Cache-first writes (cache is updated immediately, disk write retried)
§Why SQLite?
- Redis may be ephemeral (no AOF/RDB persistence)
- We need to survive both Redis and calling daemon restarts
- Cursors are small and low-write (updated every few seconds)
- SQLite WAL mode gives us durability with good performance
§Cursor Semantics
The cursor stores the last successfully applied stream ID.
On restart, we resume from cursor + 1 (exclusive read).
read event 1234 → apply to sync-engine → persist cursor 1234
(crash here = re-read 1234, idempotent)Structs§
- Cursor
Entry - Stream cursor entry
- Cursor
Store - Persistent cursor storage backed by SQLite.