Struct Source 

#[non_exhaustive]
pub struct Source {
    pub uri: String,
    pub loader: String,
    pub fetched_at: DateTime<Utc>,
    pub etag: Option<String>,
}

Where a Document originated. Survives every split and chunker pass — the leaf chunk knows the source URI of the parent document and which loader produced it.

etag enables idempotent re-ingestion: pipelines compare the loader-reported etag against the stored value and skip reprocessing when unchanged. Loaders that lack a natural etag (in-memory, ephemeral) leave it None and the pipeline falls back to time-based deduplication.

Fields (Non-exhaustive)

Non-exhaustive structs could have additional fields added in future. Therefore, non-exhaustive structs cannot be constructed in external crates using the traditional Struct { .. } syntax; cannot be matched against without a wildcard ..; and struct update syntax will not work.

uri: String

URI of the source — file:///path, s3://bucket/key, notion://workspace/page-id, etc. Pipelines treat this as opaque; loaders define the scheme.

loader: String

Stable identifier of the loader implementation. "web", "s3", "notion", etc. — used for dashboards and replay routing, not for behaviour gating. Carried as String (rather than &'static str) so persisted documents reconstruct via serde without forcing every loader name to be a string literal.

fetched_at: DateTime<Utc>

When the loader fetched this document. Pipelines stamp this at fetch time so re-ingestion ages-out properly.

etag: Option<String>

Optional content-version tag the source surfaces (HTTP ETag, S3 object etag, Notion last_edited_time hash). None when the source has no natural content-versioning signal.

Implementations

impl Source

pub fn now(uri: impl Into<String>, loader: impl Into<String>) -> Self

Build a source descriptor stamped at the current wall clock. Loaders whose natural fetch time differs from “now” (replay, batch import) construct via the struct literal.

pub fn with_etag(self, etag: impl Into<String>) -> Self

Builder-style etag attachment.

Trait Implementations

impl Clone for Source

fn clone(&self) -> Source

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Source

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for Source

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl PartialEq for Source

fn eq(&self, other: &Source) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for Source

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl Eq for Source

impl StructuralPartialEq for Source