1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
//! A buffered, multiplexing append log over pluggable object storage.
//!
//! object-log stores an ordered, offset-addressed log as immutable objects in any
//! [`BlobStore`] (memory, local filesystem, or — behind a feature — S3). It deals
//! only in **opaque payload bytes**: it knows nothing about record formats, Kafka,
//! or brokers. Three pieces:
//!
//! - [`BlobStore`] — the storage *port*: durable-on-return `put`, `get`,
//! `get_range`, `list`, `delete`. Adapters [`MemoryBlobStore`] and
//! [`LocalBlobStore`] ship in-crate.
//! - [`Sequencer`] — the *sequencing seam*: assigns offsets to durably-stored
//! batches and owns the offset→location index. [`InMemorySequencer`] ships
//! in-crate; a consumer plugs its own (e.g. a Kafka coordinator) and the engine
//! forwards each sequencer's [`Sequencer::Meta`] uninterpreted.
//! - [`LogEngine`] — buffers and group-commits many batches into one object, PUTs
//! it durably, then calls the sequencer, so PUT count is decoupled from produce
//! count. [`LogEngine::produce`] resolves at a requested [`Durability`].
//!
//! # Example
//!
//! ```
//! use object_log::{Durability, FlushConfig, InMemorySequencer, LogEngine, MemoryBlobStore, PartitionKey};
//! use bytes::Bytes;
//! use std::sync::Arc;
//!
//! # tokio::runtime::Runtime::new().unwrap().block_on(async {
//! let engine = LogEngine::new(
//! Arc::new(MemoryBlobStore::new()),
//! Arc::new(InMemorySequencer::new()),
//! FlushConfig::default(),
//! "log/",
//! );
//! let p = PartitionKey("events-0".into());
//!
//! let out = engine
//! .produce(p.clone(), Bytes::from_static(b"hello"), 1, (), Durability::Sequenced)
//! .await
//! .unwrap();
//! assert_eq!(out.base_offset, Some(0));
//!
//! let batches = engine.fetch(&p, 0, 1024).await.unwrap();
//! assert_eq!(batches[0].payload, "hello");
//! # });
//! ```
pub use ;
pub use ;
pub use ;
pub use ObjectLogError;
pub use ManifestSequencer;
pub use S3BlobStore;
pub use ;