Struct FileChunk 

pub struct FileChunk { /* private fields */ }

Represents an immutable chunk of file data for processing

This is a Value Object in Domain-Driven Design terms - it represents data without identity that cannot be modified once created. Any “changes” create new instances, ensuring data integrity and preventing accidental mutations during processing.

Key Features

• Immutability: Once created, chunks cannot be modified
• Unique Identity: Each chunk has a UUID for tracking and identification
• Sequence Ordering: Maintains sequence numbers for proper file reassembly
• Integrity Verification: Optional checksums for data integrity validation
• Metadata Tracking: Creation timestamps and processing metadata

Design Principles

• Value Object: Compared by value, not identity
• Self-Validation: Validates its own data integrity
• Builder Pattern: Use methods like with_checksum() for modifications
• Thread Safety: Fully thread-safe due to immutability

Examples

Developer Notes

• Use builder methods like with_checksum() to create modified versions
• Processing stages should create new chunks rather than modifying existing ones
• This design prevents data corruption and ensures thread safety

Implementations

impl FileChunk

pub fn new( sequence_number: u64, offset: u64, data: Vec<u8>, is_final: bool, ) -> Result<Self, PipelineError>

Creates a new file chunk

Purpose

Creates an immutable file chunk value object for pipeline processing. Chunks are the fundamental unit of file processing in the adaptive pipeline.

Why

File chunking enables:

• Parallel processing of large files
• Memory-efficient streaming
• Independent processing units
• Granular error recovery

Arguments

• sequence_number - The order of this chunk in the file (0-based)
• offset - Byte offset in the original file where this chunk starts
• data - The actual chunk data bytes (must not be empty)
• is_final - Whether this is the last chunk in the file

Returns

• Ok(FileChunk) - Successfully created chunk with unique UUID
• Err(PipelineError::InvalidChunk) - Data is empty

Errors

Returns PipelineError::InvalidChunk when data is empty.

Side Effects

• Generates new UUID for chunk identification
• Sets creation timestamp to current UTC time
• Calculates chunk size from data length

Examples

Developer Notes

• Each chunk gets a unique UUID for tracking across pipeline stages
• Chunk size is automatically validated against system limits
• Checksum is initially None - use with_calculated_checksum() to add
• This is a Value Object - create new instances for “changes”

pub fn new_with_checksum( sequence_number: u64, offset: u64, data: Vec<u8>, checksum: String, is_final: bool, ) -> Result<Self, PipelineError>

Creates a new file chunk with checksum

Developer Notes

• This is a convenience constructor for chunks that already have checksums
• Prefer using new() followed by with_checksum() for clarity

pub fn id(&self) -> Uuid

Gets the chunk ID

pub fn sequence_number(&self) -> u64

Gets the sequence number

pub fn offset(&self) -> u64

Gets the offset in the original file

pub fn size(&self) -> &ChunkSize

Gets the chunk size

pub fn data(&self) -> &[u8]

Gets the chunk data (immutable reference)

pub fn checksum(&self) -> Option<&str>

Gets the checksum if available

pub fn is_final(&self) -> bool

Checks if this is the final chunk

pub fn created_at(&self) -> DateTime<Utc>

Gets the creation timestamp

pub fn data_len(&self) -> usize

Gets the actual data length

pub fn is_empty(&self) -> bool

Checks if the chunk is empty

pub fn with_data(&self, data: Vec<u8>) -> Result<Self, PipelineError>

Creates a new FileChunk with updated data

Developer Notes

• This creates a completely new chunk instance
• The old chunk remains unchanged (immutability)
• Checksum is cleared since data changed
• Use this pattern: let new_chunk = old_chunk.with_data(new_data).unwrap();

pub fn with_checksum(&self, checksum: String) -> Self

Creates a new FileChunk with a checksum

Developer Notes

• This preserves all other data and adds/updates the checksum
• Use this after processing: let verified_chunk = chunk.with_checksum(hash);

pub fn with_calculated_checksum(&self) -> Result<Self, PipelineError>

Creates a new FileChunk with calculated SHA-256 checksum

Developer Notes

• Calculates SHA-256 hash of current data
• Returns new chunk with checksum set
• Original chunk remains unchanged

pub fn without_data(&self) -> Self

Creates a new FileChunk without data (for security)

Developer Notes

• Creates new chunk with empty data vector
• Useful for secure cleanup while preserving metadata
• Checksum is cleared since data is gone

pub fn verify_integrity(&self) -> Result<bool, PipelineError>

Verifies the chunk integrity using the stored checksum

Purpose

Validates that chunk data has not been corrupted by comparing the stored SHA-256 checksum against a freshly calculated hash of the current data.

Why

Integrity verification provides:

• Detection of data corruption during processing or storage
• Confidence in pipeline operations
• Early error detection before expensive operations
• Compliance with data integrity requirements

Returns

• Ok(true) - Checksum matches, data is intact
• Ok(false) - Checksum mismatch, data corrupted
• Err(PipelineError::InvalidChunk) - No checksum available

Errors

Returns PipelineError::InvalidChunk when the chunk has no stored checksum.

Examples

Developer Notes

• This method is read-only and doesn’t modify the chunk
• Use before critical processing to ensure data integrity
• Consider verification before expensive operations like encryption

pub fn calculate_checksum(&self) -> Result<String, PipelineError>

Calculates SHA-256 checksum without modifying the chunk

Developer Notes

• This is a pure function - doesn’t modify the chunk
• Use when you need the checksum but don’t want to create a new chunk
• For creating a chunk with checksum, use with_calculated_checksum()

Trait Implementations

impl Clone for FileChunk

fn clone(&self) -> FileChunk

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for FileChunk

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for FileChunk

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl PartialEq for FileChunk

fn eq(&self, other: &FileChunk) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for FileChunk

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl Eq for FileChunk

impl StructuralPartialEq for FileChunk