commitlog 0.0.6

Implementation of an append-only commit log backed by disk.

Crate commitlog [] [src]

The commit log is an append-only data structure that can be used in a variety of use-cases, such as tracking sequences of events, transactions or replicated state machines.

This implementation of the commit log data structure uses log segments that roll over at pre-defined maximum size boundaries. The messages appended to the log have a unique, monotonically increasing offset that can be used as a pointer to a log entry.

The index of the commit log logically stores the offset to a position in a log segment. The index and segments are separated, in that a segment file does not necessarily correspond to one particular segment file, it could contain file pointers to many segment files. In addition, index files are memory-mapped for efficient read and write access.

Example

extern crate commitlog;

use commitlog::*;

fn main() {
    // open a directory called 'log' for segment and index storage
    let opts = LogOptions::new("log");
    let mut log = CommitLog::new(opts).unwrap();

    // append to the log
    log.append_msg("hello world").unwrap(); // offset 0
    log.append_msg("second message").unwrap(); // offset 1

    // read the messages
    let messages = log.read(ReadPosition::Beginning, ReadLimit::Messages(2)).unwrap();
    for msg in messages {
        println!("{} - {}", msg.offset(), String::from_utf8_lossy(msg.payload()));
    }

    // prints:
    //    0 - hello world
    //    1 - second message
}

Structs

CommitLog

The commit log is an append-only sequence of messages.

LogOptions

Commit log options allow customization of the commit log behavior.

LogPosition

Last position read from the log.

Message

Messages contain finite-sized binary values with an offset from the beginning of the log.

MessageBuf

Buffer of message payloads prior to append to the log. The buffer allows a batch of messages to be appended to the log.

MessageSet

Readonly set of messages from the log.

Offset

Offset of an appended log segment.

OffsetRange

Offset range of log append.

OffsetRangeIter

Iterator of offsets within an OffsetRange.

Enums

AppendError

Error enum for commit log Append operation.

ReadError

Error enum for commit log read operation.

ReadLimit

Batch size limitation on read.

ReadPosition

Starting location of a read