Skip to main content

LogEngine

Struct LogEngine 

Source
pub struct LogEngine<S: Sequencer> { /* private fields */ }
Expand description

A buffered, multiplexing append-log engine over a BlobStore, with sequencing delegated to a Sequencer.

produce group-commits: many batches across many partitions are multiplexed into one object, PUT durably, then handed to the sequencer in a single call — so PUT count is decoupled from produce count. A single flush worker preserves per-PartitionKey arrival order and never splits a partition across concurrent commits.

Implementations§

Source§

impl<S> LogEngine<S>
where S: Sequencer + 'static, S::Meta: Send + 'static,

Source

pub fn new( blob: Arc<dyn BlobStore>, sequencer: Arc<S>, config: FlushConfig, key_prefix: impl Into<String>, ) -> Self

Create an engine over blob and sequencer with the given flush policy. Objects are keyed <key_prefix><counter>; pick a prefix unique to this engine instance if several share a store.

Source

pub async fn produce( &self, partition: PartitionKey, payload: Bytes, record_count: i32, meta: S::Meta, durability: Durability, ) -> Result<AppendOutcome, ObjectLogError>

Buffer a batch and resolve at the requested Durability.

Source

pub async fn flush(&self) -> Result<(), ObjectLogError>

Seal every batch enqueued at or before this call (barrier).

Use after Durability::Buffered produces to wait for durable PUT (and sequencing) of that prior work. Concurrent produces enqueued after this call are not required to finish. Empty buffer returns immediately.

Source

pub async fn fetch( &self, partition: &PartitionKey, offset: i64, max_bytes: usize, ) -> Result<Vec<FetchedBatch>, ObjectLogError>

Read batches covering offsets at/after offset, up to ~max_bytes.

Source

pub async fn truncate_before( &self, partition: &PartitionKey, offset: i64, ) -> Result<(), ObjectLogError>

Drop the partition’s log below offset and delete any object that thereby becomes fully unreferenced.

Source

pub fn buffer_stats(&self) -> BufferStats

Return a point-in-time snapshot of queued and in-flight payload bytes.

Source

pub fn pipeline_snapshot(&self) -> PipelineSnapshot

Inspect budget controller layers and counters (TD-004).

Trait Implementations§

Source§

impl<S: Sequencer> Drop for LogEngine<S>

Source§

fn drop(&mut self)

Executes the destructor for this type. Read more
Source§

fn pin_drop(self: Pin<&mut Self>)

🔬This is a nightly-only experimental API. (pin_ergonomics)
Execute the destructor for this type, but different to Drop::drop, it requires self to be pinned. Read more

Auto Trait Implementations§

§

impl<S> !RefUnwindSafe for LogEngine<S>

§

impl<S> !UnwindSafe for LogEngine<S>

§

impl<S> Freeze for LogEngine<S>

§

impl<S> Send for LogEngine<S>

§

impl<S> Sync for LogEngine<S>

§

impl<S> Unpin for LogEngine<S>

§

impl<S> UnsafeUnpin for LogEngine<S>

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.