Struct segments::MemoryLog

pub struct MemoryLog<T> { /* fields omitted */ }

Log is an inmemory commitlog (per topic) which splits data in segments. It drops the oldest segment when retention policies are crossed. Each segment is identified by base offset and a new segment is created when ever current segment crosses disk limit

Implementations

impl<T: Debug + Clone> MemoryLog<T>

pub fn new(max_segment_size: usize, max_segments: usize) -> MemoryLog<T>

Create a new log

pub fn append(&mut self, size: usize, record: T) -> (u64, u64)

Appends this record to the tail and returns the offset of this append. When the current segment is full, this also create a new segment and writes the record to it. This function also handles retention by removing head segment

pub fn next_offset(&self) -> (u64, u64)

pub fn read(&mut self, base_offset: u64, offset: usize) -> Option<T>

Read a record from correct segment

pub fn readv(
    &mut self,
    segment: u64,
    offset: u64
) -> (Option<u64>, u64, u64, Vec<T>)

Reads multiple packets from the disk and returns base offset and offset of the next log. 1st return = Done status is used by the caller to jump to next segment. When data of deleted segment is asked, returns data of the current head Note: Base offset is used to be able to pull directly from correct segment instead of Note: Uses an explicit index to identify correct segment. Kafka clients just use absolute Note: This method also returns full segment data when requested data is not of active segment. Set your max_segment size keeping tail latencies of all the concurrent connections mind (some runtimes support internal preemption using await points)