Enum IngestPayload 

#[non_exhaustive]
pub enum IngestPayload {
    Text(String),
    TextBytes(Vec<u8>),
    Binary(Vec<u8>),
}

Raw payload content provided during ingest.

IngestPayload supports multi-modal content ingestion, allowing the same pipeline to handle text and binary data uniformly.

Payload Types

• Text(String): Clean UTF-8 text (will be whitespace-normalized)
• TextBytes(Vec<u8>): Raw bytes expected to be valid UTF-8 (will be validated + normalized)
• Binary(Vec<u8>): Arbitrary binary data (passed through unchanged)

Processing

Variant	Validation	Normalization	Size Limits
Text	None	Whitespace collapsed	Both limits
TextBytes	UTF-8	Whitespace collapsed	Both limits
Binary	Non-empty	None	Raw limit only

Examples

use ingest::IngestPayload;

// Text payload
let text = IngestPayload::Text("Hello world".to_string());

// Text from bytes (validates UTF-8)
let text_bytes = IngestPayload::TextBytes(b"Hello world".to_vec());

// Binary payload (preserved as-is)
let binary = IngestPayload::Binary(vec![0x89, 0x50, 0x4E, 0x47]); // PNG magic

Variants (Non-exhaustive)

Non-exhaustive enums could have additional variants added in future. Therefore, when matching against variants of non-exhaustive enums, an extra wildcard arm must be added to account for any future variants.

Text(String)

UTF-8 text payload for normalization and canonicalization.

This is the preferred variant for text content. The text will have whitespace collapsed during ingest.

Example

use ingest::IngestPayload;

let payload = IngestPayload::Text(
    "  Content with   extra whitespace  ".to_string()
);
// After ingest: "Content with extra whitespace"

TextBytes(Vec<u8>)

Raw UTF-8 bytes that will be decoded during ingest.

Use this variant when you have bytes that should be valid UTF-8 but need validation. Invalid UTF-8 will result in IngestError::InvalidUtf8.

Example

use ingest::IngestPayload;

let payload = IngestPayload::TextBytes(
    b"Hello from bytes".to_vec()
);

Error

use ingest::{IngestPayload, ingest, IngestError};
use ingest::{RawIngestRecord, IngestMetadata, IngestSource, IngestConfig};

let record = RawIngestRecord {
    id: "test".to_string(),
    source: IngestSource::RawText,
    metadata: IngestMetadata {
        tenant_id: Some("t".to_string()),
        doc_id: Some("d".to_string()),
        received_at: None,
        original_source: None,
        attributes: None,
    },
    payload: Some(IngestPayload::TextBytes(vec![0xFF, 0xFE])), // Invalid UTF-8
};

// This will fail with InvalidUtf8
// let result = ingest(record, &IngestConfig::default());

Binary(Vec<u8>)

Arbitrary binary payload for downstream processing.

Binary payloads are passed through unmodified (except for emptiness check). They are suitable for images, PDFs, audio files, and other non-text content.

Example

use ingest::IngestPayload;

// PNG file header
let payload = IngestPayload::Binary(vec![0x89, 0x50, 0x4E, 0x47]);

Validation

Empty binary payloads (zero bytes) are rejected with IngestError::EmptyBinaryPayload.

Trait Implementations

impl Clone for IngestPayload

fn clone(&self) -> IngestPayload

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for IngestPayload

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for IngestPayload

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl PartialEq for IngestPayload

fn eq(&self, other: &IngestPayload) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for IngestPayload

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl Eq for IngestPayload

impl StructuralPartialEq for IngestPayload