Crate commitlog

The commit log is an append-only data structure that can be used in a variety of use-cases, such as tracking sequences of events, transactions or replicated state machines.

This implementation of the commit log data structure uses log segments that roll over at pre-defined maximum size boundaries. The messages appended to the log have a unique, monotonically increasing offset that can be used as a pointer to a log entry.

The index of the commit log logically stores the offset to a position in a log segment. The index and segments are separated, in that a segment file does not necessarily correspond to one particular segment file, it could contain file pointers to many segment files. In addition, index files are memory-mapped for efficient read and write access.

Example

extern crate commitlog;

use commitlog::*;

fn main() {
    // open a directory called 'log' for segment and index storage
    let opts = LogOptions::new("log");
    let mut log = CommitLog::new(opts).unwrap();

    // append to the log
    log.append_msg("hello world").unwrap(); // offset 0
    log.append_msg("second message").unwrap(); // offset 1

    // read the messages
    let messages = log.read(0, ReadLimit::default()).unwrap();
    for msg in messages {
        println!("{} - {}", msg.offset(), String::from_utf8_lossy(msg.payload()));
    }

    // prints:
    //    0 - hello world
    //    1 - second message
}

Modules

message	Message encoding used for the on-disk format for the log.
reader	Custom log reading.

Structs

CommitLog	The commit log is an append-only sequence of messages.
LogOptions	Commit log options allow customization of the commit log behavior.
OffsetRange	Offset range of log append.
OffsetRangeIter	Iterator of offsets within an OffsetRange.
ReadLimit	Batch size limitation on read.

Enums

AppendError	Error enum for commit log Append operation.
ReadError	Error enum for commit log read operation.

Type Definitions

Offset

Offset of an appended log segment.