Struct ProcessingMetrics 

pub struct ProcessingMetrics { /* private fields */ }

Processing metrics entity for comprehensive performance tracking and analysis.

ProcessingMetrics collects and maintains detailed performance data throughout pipeline execution. It provides real-time insights into processing speed, resource utilization, error rates, and overall operational health.

Metrics Categories

Processing Progress

• Bytes Processed: Total data processed and remaining
• Chunks Processed: Number of data chunks completed
• Completion Status: Progress percentage and estimated time remaining

Performance Metrics

• Throughput: Bytes per second and MB/s processing rates
• Duration: Total processing time and stage-specific timings
• Efficiency: Success rates and error statistics

File Information

• Input/Output Sizes: File sizes before and after processing
• Checksums: Integrity verification data
• Compression Ratios: Data reduction achieved

Stage Analytics

• Individual Stage Performance: Per-stage timing and throughput
• Resource Usage: Memory and CPU consumption by stage
• Error Tracking: Stage-specific error and warning counts

Usage Examples

Basic Metrics Tracking

Complete Processing Workflow

Error and Warning Tracking

Time Estimation

Merging Metrics from Multiple Sources

Serialization and Persistence

Metrics support serialization for logging and analysis:

Performance Considerations

• High-resolution timing uses Instant for accuracy
• Throughput calculations are performed on-demand
• Memory usage scales with the number of tracked stages
• Serialization excludes Instant fields for compatibility
• Real-time updates have minimal performance overhead

Implementations

impl ProcessingMetrics

pub fn new(bytes_total: u64, chunks_total: u64) -> Self

Creates new processing metrics

pub fn start(&mut self)

Starts the processing timer

pub fn end(&mut self)

Ends the processing timer

pub fn update_bytes_processed(&mut self, bytes: u64)

Updates bytes processed

pub fn add_bytes_processed(&mut self, bytes: u64)

Adds bytes processed

pub fn update_chunks_processed(&mut self, chunks: u64)

Updates chunks processed

pub fn add_chunks_processed(&mut self, chunks: u64)

Adds chunks processed

pub fn set_compression_ratio(&mut self, ratio: f64)

Sets the compression ratio

pub fn increment_errors(&mut self)

Increments error count

pub fn increment_warnings(&mut self)

Increments warning count

pub fn add_stage_metrics(&mut self, metrics: StageMetrics)

Adds stage metrics

pub fn bytes_processed(&self) -> u64

Gets bytes processed

pub fn bytes_total(&self) -> u64

Gets total bytes

pub fn chunks_processed(&self) -> u64

Gets chunks processed

pub fn chunks_total(&self) -> u64

Gets total chunks

pub fn processing_duration(&self) -> Option<Duration>

Gets processing duration

pub fn start_time(&self) -> Option<DateTime<Utc>>

Gets start time as DateTime<Utc>

pub fn end_time(&self) -> Option<DateTime<Utc>>

Gets end time as DateTime<Utc>

pub fn throughput_bytes_per_second(&self) -> f64

Gets throughput in bytes per second

pub fn throughput_mb_per_second(&self) -> f64

Gets throughput in MB/s

pub fn compression_ratio(&self) -> Option<f64>

Gets compression ratio

pub fn error_count(&self) -> u64

Gets error count

pub fn warning_count(&self) -> u64

Gets warning count

pub fn stage_metrics(&self) -> &HashMap<String, StageMetrics>

Gets stage metrics

pub fn input_file_size_bytes(&self) -> u64

Gets input file size in bytes

pub fn output_file_size_bytes(&self) -> u64

Gets output file size in bytes

pub fn input_file_size_mib(&self) -> f64

Gets input file size in MiB

pub fn output_file_size_mib(&self) -> f64

Gets output file size in MiB

pub fn input_file_checksum(&self) -> &Option<String>

Gets input file checksum

pub fn output_file_checksum(&self) -> &Option<String>

Gets output file checksum

pub fn progress_percentage(&self) -> f64

Calculates processing progress as percentage

pub fn chunk_progress_percentage(&self) -> f64

Calculates chunk progress as percentage

pub fn estimated_remaining_time(&self) -> Option<Duration>

Estimates remaining time

pub fn is_complete(&self) -> bool

Checks if processing is complete

pub fn success_rate(&self) -> f64

Calculates overall success rate

pub fn set_input_file_info(&mut self, size_bytes: u64, checksum: Option<String>)

Sets input file size and checksum

pub fn set_output_file_info( &mut self, size_bytes: u64, checksum: Option<String>, )

Sets output file size and checksum

pub fn merge(&mut self, other: &ProcessingMetrics)

Merges metrics from another instance

Trait Implementations

impl Clone for ProcessingMetrics

fn clone(&self) -> ProcessingMetrics

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ProcessingMetrics

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ProcessingMetrics

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for ProcessingMetrics

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for ProcessingMetrics

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.