Struct RawIngestRecord 

pub struct RawIngestRecord {
    pub id: String,
    pub source: IngestSource,
    pub metadata: IngestMetadata,
    pub payload: Option<IngestPayload>,
}

The inbound record for ingest.

RawIngestRecord is the primary input type for the ingest pipeline. It contains all information needed to process content: identification, source metadata, and optional payload.

Lifecycle

1. Create RawIngestRecord with raw data
2. Call ingest() to normalize
3. Receive CanonicalIngestRecord for downstream processing

Examples

Text Content

use ingest::{RawIngestRecord, IngestMetadata, IngestSource, IngestPayload};
use chrono::Utc;

let record = RawIngestRecord {
    id: "text-001".to_string(),
    source: IngestSource::RawText,
    metadata: IngestMetadata {
        tenant_id: Some("tenant".to_string()),
        doc_id: Some("doc".to_string()),
        received_at: Some(Utc::now()),
        original_source: None,
        attributes: None,
    },
    payload: Some(IngestPayload::Text(
        "  Content with   extra spaces  ".to_string()
    )),
};

Binary File

use ingest::{RawIngestRecord, IngestMetadata, IngestSource, IngestPayload};

let record = RawIngestRecord {
    id: "file-001".to_string(),
    source: IngestSource::File {
        filename: "image.png".to_string(),
        content_type: Some("image/png".to_string()),
    },
    metadata: IngestMetadata {
        tenant_id: Some("tenant".to_string()),
        doc_id: Some("doc-123".to_string()),
        received_at: None,
        original_source: Some("uploads/image.png".to_string()),
        attributes: None,
    },
    payload: Some(IngestPayload::Binary(vec![0x89, 0x50, 0x4E, 0x47])),
};

Fields

id: String

Unique identifier for this ingest operation.

This ID is used for:

• Tracing and log correlation
• Deterministic document ID derivation (when doc_id not provided)
• Deduplication and idempotency

Should be unique per ingest request. If not provided, a UUID should be generated by the caller.

Example

use ingest::RawIngestRecord;

let record = RawIngestRecord {
    id: "ingest-550e8400-e29b-41d4-a716-446655440000".to_string(),
    ..Default::default()
};

source: IngestSource

Source of the content.

Indicates where the content came from and affects validation rules. See IngestSource for details.

metadata: IngestMetadata

Metadata associated with the record.

Contains contextual information like tenant, timestamps, and custom attributes. See IngestMetadata for details.

payload: Option<IngestPayload>

Raw payload content.

The actual content being ingested. May be None for metadata-only events (e.g., IngestSource::Api).

See IngestPayload for the different payload types.

Trait Implementations

impl Clone for RawIngestRecord

fn clone(&self) -> RawIngestRecord

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for RawIngestRecord

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for RawIngestRecord

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for RawIngestRecord

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl PartialEq for RawIngestRecord

fn eq(&self, other: &RawIngestRecord) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for RawIngestRecord

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl Eq for RawIngestRecord

impl StructuralPartialEq for RawIngestRecord