Crate commitlog

Expand description

The commit log is an append-only data structure that can be used in a variety of use-cases, such as tracking sequences of events, transactions or replicated state machines.

This implementation of the commit log data structure uses log segments that roll over at pre-defined maximum size boundaries. The messages appended to the log have a unique, monotonically increasing offset that can be used as a pointer to a log entry.

The index of the commit log logically stores the offset to a position in a log segment. The index and segments are separated, in that a segment file does not necessarily correspond to one particular segment file, it could contain file pointers to many segment files. In addition, index files are memory-mapped for efficient read and write access.

§Example

use commitlog::*;

fn main() {
    // open a directory called 'log' for segment and index storage
    let opts = LogOptions::new("log");
    let mut log = CommitLog::new(opts).unwrap();

    // append to the log
    log.append_msg("hello world").unwrap(); // offset 0
    log.append_msg("second message").unwrap(); // offset 1

    // read the messages
    let messages = log.read(0, ReadLimit::default()).unwrap();
    for msg in messages {
        println!("{} - {}", msg.offset(), String::from_utf8_lossy(msg.payload()));
    }

    // prints:
    //    0 - hello world
    //    1 - second message
}

Modules§

message: Message encoding used for the on-disk format for the log.
reader: Custom log reading.

Structs§

CommitLog: The commit log is an append-only sequence of messages.
LogOptions: Commit log options allow customization of the commit log behavior.
OffsetRange: Offset range of log append.
OffsetRangeIter: Iterator of offsets within an OffsetRange.
ReadLimit: Batch size limitation on read.

Enums§

AppendError: Error enum for commit log Append operation.
ReadError: Error enum for commit log read operation.

Type Aliases§