Struct ChunkMetrics 

pub struct ChunkMetrics {
    pub chunks_seen: usize,
    pub parsed_per_ms: f64,
    pub parsed_moving: MovingAverage<20>,
    pub memory_bytes: usize,
    pub memory_moving: MovingAverage<20>,
    pub chunk_sizes: MovingAverage<20>,
    pub lines_per_chunk: usize,
    pub lines_moving: MovingAverage<20>,
    /* private fields */
}

We want to track how we’re doing so we know when to shift down into second gear and ride the clutch or when we’re winding up into high rpms and need to upshift.

Fields

chunks_seen: usize

Number of chunks processed

parsed_per_ms: f64

JSON parsing time: megabytes per millisecond

parsed_moving: MovingAverage<20>

how we’re tracking megabytes parsed per ms

memory_bytes: usize

memory usage in bytes

memory_moving: MovingAverage<20>

chunk_sizes: MovingAverage<20>

Tracking chunk sizes over time

lines_per_chunk: usize

lines_moving: MovingAverage<20>

Tracking the lines per chunk

Implementations

impl ChunkMetrics

pub fn new() -> Self

pub fn record_chunk_processing( &mut self, chunk_size: usize, duration: Duration, lines: usize, )

pub fn overall_throughput_mbps(&self) -> f64

pub fn processing_speed_mbps(&self) -> f64

pub fn speed_moving(&self) -> &MovingAverage<20>

pub fn memory_moving(&self) -> &MovingAverage<20>

pub fn chunk_sizes(&self) -> &MovingAverage<20>

pub fn should_adapt(&self, interval: usize) -> bool

We check if we should adapt every often. This means “check if we should adapt”, not “we should definitely adapt”.

Trait Implementations

impl Clone for ChunkMetrics

fn clone(&self) -> ChunkMetrics

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ChunkMetrics

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ChunkMetrics

fn default() -> Self

Returns the “default value” for a type.