Crate log 

OpenData Log - A key-oriented log system built on SlateDB.

OpenData Log provides a simple log abstraction where each key represents an independent log stream. Unlike traditional messaging systems with partitions, users write directly to keys and can create new keys as access patterns evolve.

Architecture

The log is built on SlateDB’s LSM tree. Writes append entries to the WAL and memtable, then flush to sorted string tables (SSTs). LSM compaction naturally organizes data for log locality, grouping entries by key prefix over time.

Key Concepts

• LogDb: The main entry point providing both read and write operations.
• LogDbReader: A read-only view of the log, useful for consumers that should not have write access.
• Sequence Numbers: Each entry is assigned a global sequence number at append time. Sequence numbers are monotonically increasing within a key’s log but not contiguous (other keys’ appends are interleaved).

Example

let config = Config { storage: StorageConfig::InMemory, ..Default::default() };
let log = LogDb::open(config).await?;

// Append records
let records = vec![
    Record { key: Bytes::from("orders"), value: Bytes::from("order-123") },
];
log.try_append(records).await?;
log.flush().await?;

// Scan a key's log
let mut iter = log.scan(Bytes::from("orders"), ..).await?;
while let Some(entry) = iter.next().await? {
    println!("seq={}, value={:?}", entry.sequence, entry.value);
}

Structs

AppendOutput

Output of an append operation.

Config

Configuration for opening a LogDb.

CountOptions

Options for count operations.

LogDb

The main log interface providing read and write operations.

LogDbBuilder

Builder for creating LogDb instances with custom options.

LogDbReader

A read-only view of the log.

LogEntry

An entry read from the log.

LogIterator

Iterator over log entries across multiple segments.

LogKey

A key returned from the listing iterator.

LogKeyIterator

Iterator over keys from listing entries.

ReaderConfig

Configuration for opening a LogDbReader.

Record

A record to be appended to the log.

ScanOptions

Options for scan operations.

Segment

A segment of the log.

SegmentConfig

Configuration for log segmentation.

Enums

AppendError

Error type for append operations that preserves the batch for retry.

Error

Error type for OpenData Log operations.

Traits

LogRead

Trait for read operations on the log.

Type Aliases

AppendResult

Result type alias for append operations.

Result

Result type alias for OpenData Log operations.

SegmentId

Unique identifier for a segment.

Sequence

Global sequence number for log entries.