Struct Oversized 

pub struct Oversized<E: Storage + Metrics, I: Record, V: Codec> { /* private fields */ }

Segmented journal for entries with oversized values.

Combines a fixed-size index journal with glob storage for variable-length values. Provides coordinated operations and crash recovery.

Implementations

impl<E: Storage + Metrics, I: Record + Send + Sync, V: CodecShared> Oversized<E, I, V>

pub async fn init(context: E, cfg: Config<V::Cfg>) -> Result<Self, Error>

Initialize with crash recovery validation.

Validates each index entry’s glob reference during replay. Invalid entries (pointing beyond glob size) are skipped, and the index journal is rewound to exclude trailing invalid entries.

pub async fn append( &mut self, section: u64, entry: I, value: &V, ) -> Result<(u64, u64, u32), Error>

Append entry + value.

Writes value to glob first, then writes index entry with the value location.

Returns (position, offset, size) where:

• position: Position in the index journal
• offset: Byte offset in glob
• size: Size of value in glob (including checksum)

pub async fn get(&self, section: u64, position: u64) -> Result<I, Error>

Get entry at position (index entry only, not value).

pub async fn last(&self, section: u64) -> Result<Option<I>, Error>

Get the last entry for a section, if any.

pub async fn get_value( &self, section: u64, offset: u64, size: u32, ) -> Result<V, Error>

Get value using offset/size from entry.

The offset should be the byte offset from append() or from the entry’s value_location().

pub async fn replay( &self, start_section: u64, start_position: u64, buffer: NonZeroUsize, ) -> Result<impl Stream<Item = Result<(u64, u64, I), Error>> + Send + '_, Error>

Replay index entries starting from given section.

Returns a stream of (section, position, entry) tuples.

pub async fn sync(&self, section: u64) -> Result<(), Error>

Sync both journals for given section.

pub async fn sync_all(&self) -> Result<(), Error>

Sync all sections.

pub async fn prune(&mut self, min: u64) -> Result<bool, Error>

Prune both journals. Returns true if any sections were pruned.

Prunes index first, then glob. This order ensures crash safety:

• If crash after index prune but before glob: orphan data in glob (acceptable)
• If crash before index prune: no change, retry works

pub async fn rewind( &mut self, section: u64, index_size: u64, ) -> Result<(), Error>

Rewind both journals to a specific section and index size.

This rewinds the section to the given index size and removes all sections after the given section. The value size is derived from the last entry.

pub async fn rewind_section( &mut self, section: u64, index_size: u64, ) -> Result<(), Error>

Rewind only the given section to a specific index size.

Unlike rewind, this does not affect other sections. The value size is derived from the last entry after rewinding the index.

pub async fn size(&self, section: u64) -> Result<u64, Error>

Get index size for checkpoint.

The value size can be derived from the last entry’s location when needed.

pub async fn value_size(&self, section: u64) -> Result<u64, Error>

Get the value size for a section, derived from the last entry’s location.

pub fn oldest_section(&self) -> Option<u64>

Returns the oldest section number, if any exist.

pub fn newest_section(&self) -> Option<u64>

Returns the newest section number, if any exist.

pub async fn destroy(self) -> Result<(), Error>

Destroy all underlying storage.