pub async fn poll_feed(
pool: &SqlitePool,
client: &Client,
feed: &Feed,
max_entries_per_feed: i64,
) -> Result<PollOutcome>Expand description
Fetch, parse, sanitize, normalize, and store a single feed.
Performs a conditional GET using the feed’s stored ETag / Last-Modified.
On 304 it returns PollOutcome::NotModified without touching entries. On
200 it parses with feed-rs, sanitizes every entry’s HTML with ammonia,
upserts the feed row (carrying the fresh validators) and inserts new entries
(deduped by GUID). Any fetch/parse error is logged and returned as
PollOutcome::Failed — it never panics and never propagates as Err for
a merely-broken feed, so one bad publisher can’t stall the scheduler.
Err is reserved for store failures (a broken local DB is a real error the
caller should see), not for feed misbehaviour.
max_entries_per_feed caps how many entries this feed retains after insert
(newest N by published date); <= 0 disables the per-feed trim.
Bodies are sanitized off the async runtime with a per-entry timeout
(SANITIZE_TIMEOUT; see normalize_entries). A poll in which a sanitize
timed out does not save the response’s ETag / Last-Modified — the
next poll must get a full 200 to store the bodies this one could not, and
a saved validator would answer it with 304s until the feed next changed.
Such a poll is reported as a FailureKind::Body failure, after storing
what it has, so the feed backs off and shows why in /stats: the body was
the feed’s. A poll that could not get a sanitize permit at all — every one
held by OTHER feeds’ abandoned sanitizes — stores nothing and returns
PollOutcome::Deferred, which counts against no one.