Expand description
SQLite persistence layer (via sqlx, runtime queries).
FeatherReader keeps the source of truth for what a user follows and their
read-position in the user’s own atproto PDS (as community.lexicon.rss.*
records). This module is the local per-DID cache + debounce
buffer: a single SQLite file that holds
feeds+entries— a shared cache of feed metadata and articles, keyed by feed URL / feed-native GUID and shared across every DID that follows the same feed (many users on one instance don’t multiply fetch load), andentry_state+read_cursor— per-DID read/star state and the per-feed read cursor that the (v1.1) batched flusher syncs up to the PDS.
All queries here are runtime queries (sqlx::query / sqlx::query_as),
not the compile-time query! macros — so the crate builds with no
DATABASE_URL and no offline metadata. Schema creation is idempotent
(CREATE TABLE IF NOT EXISTS) and runs inside init.
Errors propagate as anyhow::Result; nothing in the non-test paths panics.
Structs§
- Entry
- A cached article/item belonging to a
Feed. Shared cache (not per-DID). - Entry
State - Per-
(did, entry)read/star state — the fast in-session working copy that the batched flusher later syncs to the PDS as a per-feed read cursor. - Feed
- A cached syndication feed, shared across all DIDs that subscribe to its URL.
- NewEntry
- New-entry payload for
insert_entries(id is assigned by SQLite,fetched_atdefaults to “now” when not supplied). - NewFeed
- New-feed payload for
upsert_feed(id is assigned by SQLite). - Purge
Counts - The row counts purged by
purge_did_data, for a confirmable success message and for assertions in tests. - Read
Cursor - Per-
(did, feed_url)read cursor — the local mirror of the PDScommunity.lexicon.rss.readStaterecord plus flush bookkeeping.
Enums§
- Redeem
Error - Typed failure modes for
redeem_code. Distinct variants so the web layer can map each to the right user-facing message / HTTP status without string matching. Everything else (a real SQLite error) still propagates asanyhow::Errorout of theResult.
Constants§
- REDACTED_
DID - Sentinel written into
beta_access.granted_bywhen the granting DID deletes its data: the column isNOT NULL, so we redact rather than NULL it. Keeps the grantee’s seat valid while removing the departed DID’s back-reference.
Functions§
- bump_
feed_ errors - Record a poll FAILURE for a feed: bump its
consecutive_errorsby one and return the NEW count. The count drives the exponential poll backoff, so a persistently-failing feed spaces its retries out toward the ceiling instead of hammering the 5-minute floor forever. Reset to 0 byreset_feed_errorson any success/304. - clear_
cursor_ dirty - Clear the
dirtyflag on a cursor after a successful PDS flush — but ONLY if the row still carries the exactflushed_updated_atsnapshot we flushed. - count_
beta_ access - Count the beta seats currently granted — the numerator checked against the configured cap on redeem.
- count_
feeds - The number of distinct feeds in the shared cache. Backs the global feeds ceiling checked before a brand-new feed is inserted.
- count_
subscriptions_ for_ did - The number of feeds a
didcurrently subscribes to (itssub_refrows). Backs the per-DID subscription cap enforced at the add/import paths. - db_
size_ bytes - The used size of the SQLite database, in bytes, computed as
(page_count - freelist_count) * page_size. Backs the DB-size watermark that disables new polling. - did_
subscribes_ to_ entry - Whether
didcurrently subscribes to the feedfeed_idowns (i.e. asub_refrow exists). The authorization primitive behind every per-DID scoped read/mutation. - dirty_
cursors - The flusher’s hot query: every cursor with
dirty = 1for a DID — the ones whose read-state changed since the last batched PDS flush. - due_
feeds - The scheduler’s hot query: feeds whose
next_pollis due (<= as_of, or never polled), oldest-due first.as_ofis an RFC3339 timestamp. - ensure_
seed - Seed the admin-bootstrap DIDs: for each, insert a
beta_accessrow (granted_by = 'admin') if one does not already exist. Idempotent — an existing seat is left untouched. Returns how many new seats were created. - entries_
for_ feed - All entries for a feed, newest-published first — scoped to
did’s subscriptions. Returns an empty vec ifdiddoes not subscribe to the feed. - expire_
old_ codes - Sweep: flip every
activecode whoseexpires_atis in the past toexpired. Returns the number of codes expired. Called periodically by the scheduler. - feeds_
for_ did - The feeds a
didcurrently subscribes to, per itssub_refprojection. Used by the PDS-unreachable fallback inresolve_subscriptionsto render the sidebar from the caller’s OWN last-known subscriptions (fail closed) rather than every cached feed. - generate_
invite_ code - Generate a random, unguessable invite code of the form
FEATHER-XXXXXXXX. - get_
cursor - Fetch a single read cursor, if present.
- get_
feed_ by_ url - Fetch a feed by its URL, if present.
- get_
starred_ for_ did - Starred entries for a DID, newest-published first.
- get_
unread_ for_ did - Unread entries for a DID: entries with no
entry_staterow for that DID, or one whereread = 0. Newest-published first. This is the daily-driver list query, so it’s aLEFT JOIN(an entry with no state row is unread). - grant_
access - Grant a beta seat directly (admin / seed path — no code consumed). Idempotent
on
did(re-granting updates the row rather than erroring). - has_
beta_ access - Whether a DID currently holds a beta seat.
- init
- Open the per-DID SQLite cache described by
Config(itsdb_path), run schema creation, and return the pool. - init_
schema - Run the idempotent schema creation. Split out so callers/tests can (re)apply it against an already-open pool.
- init_
url - Open (creating if needed) the SQLite database at
db_url, run schema creation, and return a connection pool. - insert_
entries - Insert a batch of entries for
feed_id, deduping on(feed_id, guid), then trim the feed to at mostcrate::config-configuredmax_entries_per_feedrows (newest by published date) so one firehose feed can’t fill the disk. - mark_
cursor_ pds_ created - Mark a cursor’s PDS
readStaterecord as CREATED after the flush that first created it, so subsequent flushes emit anupdateinstead of anothercreate. Idempotent; a no-op if the row is gone. - mark_
feed_ read - Mark every entry of a feed read (or unread) for a DID in one statement —
backs the “mark-all-read (per feed)” action. Also projects the change into
the feed’s per-DID
ReadCursor(dirty=1) so the batched flusher syncs the new read-state to the PDS. - mark_
read - Mark a single entry read/unread for a DID, upserting the per-DID state row
and stamping
updated_at. Preserves any existingstarredbit. Also projects the change into the per-(did, feed_url)ReadCursorand marks itdirtyso the batched flusher pushes it to the PDS (see [project_entry_into_cursor]). - mark_
starred - Star/unstar a single entry for a DID (upsert, preserving
read). - mint_
code - Mint a new
activeinvite code owned bycreator_did, expiringttl_secsfrom now. Returns the generated code string. - prune_
old_ entries - Delete entries whose age exceeds the retention window — the shared cache’s
rolling window. “Age” is
COALESCE(published, fetched_at)so an UNDATED entry falls back to when it was fetched (never NULL) rather than being treated as infinitely old.entry_statecascades via itsON DELETE CASCADEFK. - purge_
did_ data - Delete all local rows owned by
didin a single transaction: the per-DID read/star state (entry_state), per-feed read cursors (read_cursor), the subscription projection (sub_ref), the closed-beta seat (beta_access), and any invite codes this DID created (invite_codes). The sharedfeeds/entriescache is intentionally left intact — it is deduped and not owned by any single DID. - reclaim
- Reclaim freed pages so the database file (and its used-page accounting) can actually shrink after a retention/prune sweep DELETEs rows.
- redeem_
code - Atomically redeem an invite code for
did, granting a beta seat. - replace_
sub_ refs - Replace the per-DID subscription projection (
sub_ref) fordidwith exactlyfeed_ids, in one transaction. - reset_
feed_ errors - Reset a feed’s
consecutive_errorsto 0 after a successful poll (or a 304). A no-op UPDATE if the row is missing. - subscribed_
feed_ ids - The feed ids a
didcurrently subscribes to (itssub_refrows). Bounded by the per-DID subscription cap, so callers can safely iterate it — e.g. the global “mark all read” path fans out over feeds (bounded) rather than over unread entries (unbounded). - upsert_
cursor - Insert or update a per-
(did, feed_url)read cursor, stampingupdated_at. The write path for local mark-read updates (and the seam a login-time PDS merge would use, once that is wired). - upsert_
feed - Insert a feed by URL, or update its metadata if the URL already exists. Returns the feed’s row id (existing or newly assigned).
Type Aliases§
- Pool
- The SQLite connection pool type the rest of the crate refers to as
Pool. A thin alias overSqlitePoolsocrate::AppStateand the web layer name one stable type; if the backend ever changes, this is the single place to swap it.