pub async fn compact_cursor(
pool: &SqlitePool,
did: &str,
feed_url: &str,
) -> Result<Option<String>>Expand description
Fold ids already covered by a high-water-mark into read_through, so the
exception set stops growing. Returns the new read_through when it advanced.
What was wrong. read_through was never COMPUTED — project_entry_into_cursor
only carried an existing value through, and it starts NULL, so in practice it
was always NULL. That left read_ids as the sole mechanism, growing one id
per article read, bounded only by max_entries_per_feed (2000) — while the
flusher caps the record at ReadState::MAX_IDS (1000) keeping the TAIL, with
no log line. Past 1000 read articles in one feed, the oldest read-state
silently stopped syncing, and those articles came back UNREAD in any other
atproto reader. The cap helper’s own comment assumed “the exception sets
are expected to stay well under the cap in normal use”; against a 2000-entry
per-feed ceiling that does not hold.
The rule. read_through means “every entry at or before this time is
read”. So it may advance only to a point with no unread entry at or before
it. That point is computed here as the newest entry timestamp STRICTLY OLDER
than the oldest unread entry — strictly, because entries can share a
timestamp, and a watermark equal to an unread entry’s time would assert that
entry is read.
Once the watermark moves, every read_ids entry at or before it is
redundant and is dropped — that is the compaction. unread_ids is filtered
the same way; by construction nothing unread sits at or below the new
watermark, so it empties, but the filter is written rather than assumed so it
stays correct if that invariant ever shifts.
Timestamps compare lexicographically because every writer normalises to UTC
...Z at seconds precision (feed::fmt_time, now_rfc3339) — the same
assumption poll_health and the retention window already make.