Module store

Expand description

SQLite persistence layer (via sqlx, runtime queries).

FeatherReader keeps the source of truth for what a user follows and their read-position in the user’s own atproto PDS (as community.lexicon.rss.* records). This module is the local per-DID cache + debounce buffer: a single SQLite file that holds

feeds + entries — a shared cache of feed metadata and articles, keyed by feed URL / feed-native GUID and shared across every DID that follows the same feed (many users on one instance don’t multiply fetch load), and
entry_state + read_cursor — per-DID read/star state and the per-feed read cursor that the (v1.1) batched flusher syncs up to the PDS.

All queries here are runtime queries (sqlx::query / sqlx::query_as), not the compile-time query! macros — so the crate builds with no DATABASE_URL and no offline metadata. Schema creation is idempotent (CREATE TABLE IF NOT EXISTS) and runs inside init.

Errors propagate as anyhow::Result; nothing in the non-test paths panics.

Structs§

Entry: A cached article/item belonging to a Feed. Shared cache (not per-DID).
EntryListRow: One row of a LIST view — deliberately without content_html.
EntryState: Per-(did, entry) read/star state — the fast in-session working copy that the batched flusher later syncs to the PDS as a per-feed read cursor.
FailingFeed: One failing feed, named, for the ADMIN view only.
Feed: A cached syndication feed, shared across all DIDs that subscribe to its URL.
NetworkStat: One relay’s observation of how many repos hold a collection (design/NETWORK-SPEC.md §4.3). A projection: droppable, rebuildable from the network, and never read by anything on the reading path.
NewEntry: New-entry payload for insert_entries (id is assigned by SQLite, fetched_at defaults to “now” when not supplied).
NewFeed: New-feed payload for upsert_feed (id is assigned by SQLite).
PollHealth: Aggregate poll health, for the public stats page.
PurgeCounts: The row counts purged by purge_did_data, for a confirmable success message and for assertions in tests.
ReadCursor: Per-(did, feed_url) read cursor — the local mirror of the PDS community.lexicon.rss.readState record plus flush bookkeeping.

Enums§

AutoVacuum: A database’s auto_vacuum mode.
ListView: Which list list_entries (and its siblings) is producing.
RedeemError: Typed failure modes for redeem_code. Distinct variants so the web layer can map each to the right user-facing message / HTTP status without string matching. Everything else (a real SQLite error) still propagates as anyhow::Error out of the Result.
StarredIdentities: The (url, guid) identity pairs of every cached starred entry for a DID.
VacuumMigration: What migrate_to_incremental_vacuum did.

Constants§

ADOPTION_STAT_KEY: The network_stat key the relay adoption probe writes under.
REDACTED_DID: Sentinel written into beta_access.granted_by when the granting DID deletes its data: the column is NOT NULL, so we redact rather than NULL it. Keeps the grantee’s seat valid while removing the departed DID’s back-reference.

Functions§

auto_vacuum_mode: Read the database’s auto_vacuum mode.
bump_feed_errors: Record a poll FAILURE for a feed: bump its consecutive_errors by one and return the NEW count. The count drives the exponential poll backoff, so a persistently-failing feed spaces its retries out toward the ceiling instead of hammering the 5-minute floor forever. Reset to 0 by reset_feed_errors on any success/304.
clear_cursor_dirty: Clear the dirty flag on a cursor after a successful PDS flush — but ONLY if the row still carries the exact flushed_updated_at snapshot we flushed.
clear_star_by_identity: Clear did’s star on any cached entry matching url or guid, ignoring the subscription projection. Returns the number of entry_state rows changed.
compact_cursor: Fold ids already covered by a high-water-mark into read_through, so the exception set stops growing. Returns the new read_through when it advanced.
count_active_codes: Count active, unexpired invite codes — the outstanding-but-unredeemed seats a bot has already promised. Added to count_beta_access this is the “seats committed” figure the bot mint path (POST /bot/claims) checks against the cap, so it doesn’t over-promise more claims than seats remain (the redeem-time cap in redeem_code is the hard backstop; this avoids telling a follower “you’re in” for a seat that will be full by the time they claim it).
count_beta_access: Count the beta seats currently granted — the numerator checked against the configured cap on redeem.
count_entries_for_view: How many entries the same scope + view would return, unpaged. Used for the “N entries” heading and to decide whether a next-page link is warranted — both of which used to read entries.len() off a fully materialized list.
count_feeds: The number of distinct feeds in the shared cache. Backs the global feeds ceiling checked before a brand-new feed is inserted.
count_subscriptions_for_did: The number of feeds a did currently subscribes to (its sub_ref rows). Backs the per-DID subscription cap enforced at the add/import paths.
db_size_bytes: The used size of the SQLite database, in bytes, computed as (page_count - freelist_count) * page_size. Backs the DB-size watermark that disables new polling.
did_subscribes_to_entry: Whether did currently subscribes to the feed feed_id owns (i.e. a sub_ref row exists). The authorization primitive behind every per-DID scoped read/mutation.
dirty_cursors
due_feeds: The scheduler’s hot query: feeds whose next_poll is due (<= as_of, or never polled), oldest-due first. as_of is an RFC3339 timestamp.
ensure_seed: Seed the admin-bootstrap DIDs: for each, insert a beta_access row (granted_by = 'admin') if one does not already exist. Idempotent — an existing seat is left untouched. Returns how many new seats were created.
expire_old_codes: Sweep: flip every active code whose expires_at is in the past to expired. Returns the number of codes expired. Called periodically by the scheduler.
failing_feeds: Every currently-failing feed with its recorded cause, worst first.
feeds_for_did: The feeds a did currently subscribes to, per its sub_ref projection. Used by the PDS-unreachable fallback in resolve_subscriptions to render the sidebar from the caller’s OWN last-known subscriptions (fail closed) rather than every cached feed.
find_active_code_for_did: The code of an outstanding (active, unexpired) invite minted FOR the follower intended_did, if one exists — the app-side idempotency lookup for POST /bot/claims. Some(code) means “return this existing code, do NOT mint a second”; None means “no live code for this DID — mint one”.
generate_invite_code: Generate a random, unguessable invite code of the form FEATHER-XXXXXXXX.
get_cursor: Fetch a single read cursor, if present.
get_feed_by_url: Fetch a feed by its URL, if present.
grant_access: Grant a beta seat directly (admin / seed path — no code consumed). Idempotent on did (re-granting updates the row rather than erroring).
has_beta_access: Whether a DID currently holds a beta seat.
init: Open the per-DID SQLite cache described by Config (its db_path), run schema creation, and return the pool.
init_schema: Run the idempotent schema creation. Split out so callers/tests can (re)apply it against an already-open pool.
init_url
insert_entries: Insert a batch of entries for feed_id, deduping on (feed_id, guid), then trim the feed to at most crate::config-configured max_entries_per_feed rows (newest by published date) so one firehose feed can’t fill the disk.
is_intended_active_conflict: Does this error chain represent the partial-unique-index conflict raised when a SECOND active claim is minted for a DID that already has one (idx_invite_codes_intended_active)? The web layer uses this to recover from a lost mint race (S4): on a conflict it re-reads the winner’s code instead of 500-ing. Matches on the sqlx Database error’s UNIQUE-constraint code (SQLite 2067 / primary 19) AND the offending COLUMN in the message (invite_codes.intended_did — SQLite names the column(s), not the index), so an unrelated constraint violation (e.g. the code PRIMARY KEY) is NOT swallowed.
latest_network_stat: The highest observation for key across every relay — the number to surface (design/NETWORK-SPEC.md §4.1: relays disagree; show the max). None when no probe has ever succeeded.
list_entries: One page of a list view, newest-published first, scoped to did’s subscriptions (sub_ref) and optionally narrowed to feed_ids.
list_entry_ids: The ordered entry ids for a scope + view — the same ordering list_entries renders, used for the reader’s prev/next links.
mark_cursor_pds_created: Mark a cursor’s PDS readState record as CREATED after the flush that first created it, so subsequent flushes emit an update instead of another create. Idempotent; a no-op if the row is gone.
mark_feed_due: Make a feed due for polling on the next tick.
mark_feed_read: Mark every entry of a feed read (or unread) for a DID in one statement — backs the “mark-all-read (per feed)” action. Also projects the change into the feed’s per-DID ReadCursor (dirty=1) so the batched flusher syncs the new read-state to the PDS.
mark_read: Mark a single entry read/unread for a DID, upserting the per-DID state row and stamping updated_at. Preserves any existing starred bit. Also projects the change into the per-(did, feed_url) ReadCursor and marks it dirty so the batched flusher pushes it to the PDS (see [project_entry_into_cursor]).
mark_starred: Star/unstar a single entry for a DID (upsert, preserving read).
migrate_to_incremental_vacuum: Move a populated database from auto_vacuum = NONE to INCREMENTAL.
mint_code: Mint a new active invite code owned by creator_did, expiring ttl_secs from now. Returns the generated code string. The browser/admin path leaves the bot idempotency key (intended_did) NULL; see mint_code_for_did for the bot path that records the target follower.
mint_code_for_did: Like mint_code but records the follower intended_did the code is minted FOR, so a later POST /bot/claims for the same DID can return the SAME code (see find_active_code_for_did) rather than minting a duplicate. This is the app-side idempotency backstop that survives a bot-host state loss.
parked_readstate_dids: The flusher’s hot query: every cursor with dirty = 1 for a DID — the ones whose read-state changed since the last batched PDS flush. How many DIDs hold read-state that cannot currently be flushed: dirty cursors with no OAuth session to send them with.
poll_health: Compute PollHealth as of now (RFC3339, seconds precision — the same format the scheduler writes, so the comparisons are lexicographic).
prune_old_entries: Delete entries whose age exceeds the retention window — the shared cache’s rolling window — except those a reader has starred or not yet read. “Age” is COALESCE(published, fetched_at) so an UNDATED entry falls back to when it was fetched (never NULL) rather than being treated as infinitely old. entry_state cascades via its ON DELETE CASCADE FK.
purge_did_data: Delete all local rows owned by did in a single transaction: the per-DID read/star state (entry_state), per-feed read cursors (read_cursor), the subscription projection (sub_ref), the closed-beta seat (beta_access), and any invite codes this DID created (invite_codes). The shared feeds/entries cache is intentionally left intact — it is deduped and not owned by any single DID.
reclaim: Reclaim freed pages so the database file (and its used-page accounting) can actually shrink after a retention/prune sweep DELETEs rows.
record_network_stat: Record one relay’s observation, keyed by (key, source) so each relay’s number is kept separately (non-archival relays legitimately disagree).
redeem_code: Atomically redeem an invite code for did, granting a beta seat.
replace_sub_refs: Replace the per-DID subscription projection (sub_ref) for did with exactly feed_ids, in one transaction.
reset_feed_errors: Reset a feed’s consecutive_errors to 0 after a successful poll (or a 304). A no-op UPDATE if the row is missing.
set_next_poll: Schedule a feed’s next poll delay from now.
starred_identities
subscribed_feed_ids: The feed ids a did currently subscribes to (its sub_ref rows).
unread_counts_by_feed: Unread counts per feed_id for a DID — the sidebar’s per-feed badges.
upsert_cursor: Insert or update a per-(did, feed_url) read cursor, stamping updated_at. The write path for local mark-read updates (and the seam a login-time PDS merge would use, once that is wired).
upsert_feed: Insert a feed by URL, or update its metadata if the URL already exists. Returns the feed’s row id (existing or newly assigned).

Type Aliases§

Pool: The SQLite connection pool type the rest of the crate refers to as Pool. A thin alias over SqlitePool so crate::AppState and the web layer name one stable type; if the backend ever changes, this is the single place to swap it.

Module store

Module store Copy item path

Structs§

Enums§

Constants§

Functions§

Type Aliases§

Module store