Module cursor

Module cursor 

Source
Expand description

Cursor persistence for stream positions.

Stores the last-read stream ID for each peer in SQLite. This survives Redis restarts (we’re not relying on AOF!) and daemon restarts.

§Debounced Writes

To reduce SQLite write pressure, cursors are debounced:

  • set() updates the in-memory cache immediately and marks the cursor dirty
  • flush_dirty() persists all dirty cursors to disk in a batch
  • The coordinator calls flush_dirty() periodically (every few seconds)
  • On shutdown, flush_dirty() is called to ensure no data loss

This means a crash between set() and flush_dirty() could lose up to one flush interval of cursor progress. On restart, we’d re-read some events that were already applied (idempotent, safe).

§SQLite Busy Handling

SQLite can return SQLITE_BUSY/SQLITE_LOCKED when the database is contended. We handle this with:

  • Automatic retry with exponential backoff
  • Configurable max retries (default 5)
  • Cache-first writes (cache is updated immediately, disk write retried)

§Why SQLite?

  • Redis may be ephemeral (no AOF/RDB persistence)
  • We need to survive both Redis and calling daemon restarts
  • Cursors are small and low-write (updated every few seconds)
  • SQLite WAL mode gives us durability with good performance

§Cursor Semantics

The cursor stores the last successfully applied stream ID. On restart, we resume from cursor + 1 (exclusive read).

read event 1234 → apply to sync-engine → persist cursor 1234
                  (crash here = re-read 1234, idempotent)

Structs§

CursorEntry
Stream cursor entry
CursorStore
Persistent cursor storage backed by SQLite.