Trait Source

Source

pub trait Source: Send + Sync {
    // Required methods
    fn name(&self) -> &str;
    fn can_serve(&self, profile: &CapabilityProfile, ref_: &Ref) -> bool;
    fn fetch<'life0, 'life1, 'life2, 'life3, 'async_trait>(
        &'life0 self,
        ref_: &'life1 Ref,
        profile: &'life2 CapabilityProfile,
        ctx: &'life3 FetchContext,
    ) -> Pin<Box<dyn Future<Output = Result<FetchResult, FetchError>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait,
             'life2: 'async_trait,
             'life3: 'async_trait;

    // Provided method
    fn fetch_content<'life0, 'life1, 'life2, 'life3, 'async_trait>(
        &'life0 self,
        _ref_: &'life1 Ref,
        _profile: &'life2 CapabilityProfile,
        _ctx: &'life3 FetchContext,
    ) -> Pin<Box<dyn Future<Output = Result<Option<Bytes>, FetchError>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait,
             'life2: 'async_trait,
             'life3: 'async_trait { ... }
}

Expand description

The trait implemented by every Tier 1 / 2 / 3 fetcher.

Binding signature: docs/PUBLIC_API.md §2 (NORMATIVE — the wire shape of these three methods is semver-locked).

Required Methods§

Source

fn name(&self) -> &str

Stable name used in metadata ([doiget].source) and provenance rows. Conventional values: "crossref", "unpaywall", "arxiv", "openalex", "semantic-scholar", "doaj", "tdm-elsevier", etc. (see docs/SOURCES.md).

Source

fn can_serve(&self, profile: &CapabilityProfile, ref_: &Ref) -> bool

True if this source can plausibly serve the given ref under the runtime capability profile. Implementations MUST be fast and non-blocking; the orchestrator calls can_serve to decide whether to invoke fetch at all.

Source

fn fetch<'life0, 'life1, 'life2, 'life3, 'async_trait>( &'life0 self, ref_: &'life1 Ref, profile: &'life2 CapabilityProfile, ctx: &'life3 FetchContext, ) -> Pin<Box<dyn Future<Output = Result<FetchResult, FetchError>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait,

Perform the source-specific fetch.

Implementations:

acquire ctx.rate_limiter.acquire(self.name()).await,
fetch via ctx.http.fetch_bytes / ctx.http.fetch_pdf,
emit one LogEvent::Fetch row via ctx.log.append,
return a FetchResult.

The trait does NOT enforce these steps; it documents the protocol so concrete impls produce uniform audit trails (per docs/ARCHITECTURE.md §6 and docs/PROVENANCE_LOG.md §3).

Provided Methods§

Source

fn fetch_content<'life0, 'life1, 'life2, 'life3, 'async_trait>( &'life0 self, _ref_: &'life1 Ref, _profile: &'life2 CapabilityProfile, _ctx: &'life3 FetchContext, ) -> Pin<Box<dyn Future<Output = Result<Option<Bytes>, FetchError>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait,

Fetch the publisher’s own copy of the document itself, when this source holds one.

Distinct from Self::fetch, which resolves a record. A Tier-3 TDM source is consulted for two different reasons at two different points in the fetch, and conflating them is what #458 was:

fetch answers “who can tell me about this DOI?” and runs when Crossref could not;
fetch_content answers “who will give me the bytes?” and runs when the content leg was blocked — which is usually after Crossref answered perfectly well.

The default is Ok(None): “this source is metadata-only”. Stating it is the point. Before #458 the same fact was expressed by every Tier-3 impl setting FetchResult.pdf_bytes to None and saying so in a doc-comment, which the orchestrator could neither read nor act on — so it could not tell a source that had nothing to offer from one it had simply never asked.

Implementations that override it MUST use a PDF-validating fetch (HttpClient::fetch_pdf or HttpClient::fetch_pdf_with_headers). A publisher error page or a WAF holding response is a 200 with a body, and storing one under <safekey>.pdf would be worse than returning nothing.

§Errors

Any FetchError. Ok(None) means “not me”; Err means “me, and it went wrong”. The orchestrator keeps the original content-leg block either way, but records the two as different attempt outcomes.