Skip to main content

Module fetch

Module fetch 

Source
Expand description

HTTP client with conditional GET. Owner: fetcher agent.

Frozen public interface — crate::core depends on these exact signatures. Implement the bodies; do not change the signatures without coordinating with the team lead.

§Requirements

  • Build a reqwest::Client with gzip, a sane redirect policy, the given timeout, and the provided User-Agent.
  • Honor CachePolicy:
    • NoCache: plain GET, never read or write the cache.
    • MaxAge(d): if a cache entry exists and is younger than d, return it without any network call (from_cache = true, not_modified = true). Otherwise revalidate.
    • CacheFirst: if a cache entry exists, return it without any network call regardless of age (from_cache = true, not_modified = true); only a cache miss falls through to a conditional GET. Used by item lookup so a rolled feed window cannot evict an item the caller already saw (ADR-0014).
    • Revalidate (default): send If-None-Match (etag) / If-Modified-Since (last_modified) from the cache entry if present. On 304, return the cached body with not_modified = true, from_cache = true. On 200, store the new body + validators (ETag, Last-Modified) in the cache and return it.
  • On a non-success, non-304 status, return RssError::Http.
  • final_url is the URL after following redirects (used to resolve relative links).

Structs§

HttpClient
Reusable HTTP client.
RawFeed
Raw bytes of a fetched (or cached) feed, plus the metadata parse/core need.