Struct ZstdVec 

pub struct ZstdVec<I, T>(/* private fields */);

Compressed storage using Zstd for maximum general-purpose compression.

Zstd (Zstandard) provides the best compression ratios among general-purpose algorithms, with good decompression speed. Ideal when storage is expensive.

Performance Characteristics

• Highest compression ratios (typically 3-5x, better than LZ4)
• Fast decompression (slower compression than LZ4)
• Works well with any data type

When to Use

• Storage space is expensive
• Can tolerate slower compression (decompression is fast)

Implementations

impl<I, T> ZstdVec<I, T>

where I: VecIndex, T: ZstdVecValue,

pub fn iter(&self) -> Result<ZstdVecIterator<'_, I, T>>

pub fn clean_iter(&self) -> Result<CleanZstdVecIterator<'_, I, T>>

pub fn dirty_iter(&self) -> Result<DirtyZstdVecIterator<'_, I, T>>

pub fn boxed_iter(&self) -> Result<BoxedVecIterator<'_, I, T>>

Trait Implementations

impl<I, T> AnyStoredVec for ZstdVec<I, T>

where I: VecIndex, T: ZstdVecValue,

fn db_path(&self) -> PathBuf

fn region(&self) -> &Region

fn header(&self) -> &Header

fn mut_header(&mut self) -> &mut Header

fn saved_stamped_changes(&self) -> u16

Number of stamped change files to keep for rollback support.

fn real_stored_len(&self) -> usize

The actual length stored on disk.

fn stored_len(&self) -> usize

The effective stored length (may differ from real_stored_len during truncation).

fn serialize_changes(&self) -> Result<Vec<u8>>

fn remove(self) -> Result<()>

Removes this vector’s region from the database.

fn flush(&mut self) -> Result<()>

fn safe_flush(&mut self, exit: &Exit) -> Result<()>

Flushes while holding the exit lock to ensure consistency during shutdown.

fn safe_write(&mut self, exit: &Exit) -> Result<()>

Writes to mmap without fsync, holding the exit lock. Data is visible to readers immediately but not durable until sync. Use this for performance when durability can be deferred.

fn update_stamp(&mut self, stamp: Stamp)

fn stamp(&self) -> Stamp

fn stamped_flush(&mut self, stamp: Stamp) -> Result<()>

impl<I, T> AnyVec for ZstdVec<I, T>

where I: VecIndex, T: ZstdVecValue,

fn version(&self) -> Version

fn name(&self) -> &str

fn len(&self) -> usize

fn index_type_to_string(&self) -> &'static str

Returns the string representation of the index type.

fn value_type_to_size_of(&self) -> usize

Returns the size in bytes of the value type.

fn region_names(&self) -> Vec<String>

Returns the list of region names used by this vector.

fn is_empty(&self) -> bool

fn region_name(&self) -> String

Returns the combined name of the vector.

fn etag(&self, stamp: Stamp, to: Option<i64>) -> String

Generates an ETag for this vector based on stamp and optional end index.

fn i64_to_usize(&self, i: i64) -> usize

Converts an i64 index to usize, supporting negative indexing (Python-style).

impl<I: Clone, T: Clone> Clone for ZstdVec<I, T>

fn clone(&self) -> ZstdVec<I, T>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<I: Debug, T: Debug> Debug for ZstdVec<I, T>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<I, T> Deref for ZstdVec<I, T>

type Target = CompressedVecInner<I, T, ZstdStrategy<T>>

The resulting type after dereferencing.

fn deref(&self) -> &Self::Target

Dereferences the value.

impl<I, T> DerefMut for ZstdVec<I, T>

fn deref_mut(&mut self) -> &mut Self::Target

Mutably dereferences the value.

impl<I, T> GenericStoredVec<I, T> for ZstdVec<I, T>

where I: VecIndex, T: ZstdVecValue,

fn read_value_from_bytes(&self, bytes: &[u8]) -> Result<T>

Deserializes a value from bytes. Implementors use their strategy.

fn write_value_to(&self, value: &T, buf: &mut Vec<u8>)

Writes a value to the provided buffer. Implementors use their strategy.

fn pushed(&self) -> &[T]

Returns the current pushed (uncommitted) values.

fn mut_pushed(&mut self) -> &mut Vec<T>

Returns a mutable reference to the current pushed (uncommitted) values.

fn reset(&mut self) -> Result<()>

Resets the vector state.

fn truncate_if_needed_at(&mut self, index: usize) -> Result<()>

Truncates the vector to the given usize index if the current length exceeds it.

fn reset_unsaved(&mut self)

Resets uncommitted changes.

fn is_dirty(&self) -> bool

Returns true if there are uncommitted changes (pushed values).

fn stamped_flush_with_changes(&mut self, stamp: Stamp) -> Result<()>

Flushes with the given stamp, saving changes to enable rollback.

fn rollback_before(&mut self, stamp: Stamp) -> Result<Stamp>

Rolls back changes to before the given stamp.

fn rollback(&mut self) -> Result<()>

Rolls back the most recent change set.

fn deserialize_then_undo_changes(&mut self, bytes: &[u8]) -> Result<()>

Deserializes change data and undoes those changes. Base implementation handles pushed and truncated data. RawVecInner overrides for holes/updated.

const SIZE_OF_T: usize = _

fn create_reader(&self) -> Reader

Creates a reader to the underlying region. Be careful with deadlocks - drop the reader before mutable ops.

fn read(&self, index: I, reader: &Reader) -> Result<T>

Reads value at index using provided reader.

fn read_once(&self, index: I) -> Result<T>

Reads value at index, creating a temporary reader. For multiple reads, prefer read() with a reused reader.

fn read_at(&self, index: usize, reader: &Reader) -> Result<T>

Reads value at usize index using provided reader.

fn write_values_to(&self, values: &[T], buf: &mut Vec<u8>)

Writes multiple values to the provided buffer.

fn read_at_once(&self, index: usize) -> Result<T>

Reads value at usize index, creating a temporary reader. For multiple reads, prefer read_at() with a reused reader.

fn read_unwrap(&self, index: I, reader: &Reader) -> T

Reads value at index using provided reader. Panics if read fails.

fn read_unwrap_once(&self, index: I) -> T

Reads value at index, creating a temporary reader. Panics if read fails. For multiple reads, prefer read_unwrap() with a reused reader.

fn read_at_unwrap(&self, index: usize, reader: &Reader) -> T

Reads value at usize index using provided reader. Panics if read fails.

fn read_at_unwrap_once(&self, index: usize) -> T

Reads value at usize index, creating a temporary reader. Panics if read fails. For multiple reads, prefer read_at_unwrap() with a reused reader.

fn get_pushed_or_read(&self, index: I, reader: &Reader) -> Result<Option<T>>

Gets value from pushed layer or storage using provided reader.

fn get_pushed_or_read_once(&self, index: I) -> Result<Option<T>>

Gets value from pushed layer or storage, creating a temporary reader. For multiple reads, prefer get_pushed_or_read() with a reused reader.

fn get_pushed_or_read_unwrap(&self, index: I, reader: &Reader) -> T

Gets value from pushed layer or storage using provided reader, unwrapping the result. Panics if the read fails or if the value doesn’t exist.

fn get_pushed_or_read_unwrap_once(&self, index: I) -> T

Gets value from pushed layer or storage, unwrapping the result. Panics if the read fails or if the value doesn’t exist. For multiple reads, prefer get_pushed_or_read_unwrap() with a reused reader.

fn get_pushed_or_read_at( &self, index: usize, reader: &Reader, ) -> Result<Option<T>>

Gets value from pushed layer or storage at usize index using provided reader. Does not check the updated layer.

fn get_pushed_or_read_at_once(&self, index: usize) -> Result<Option<T>>

Gets value from pushed layer or storage at usize index, creating a temporary reader. For multiple reads, prefer get_pushed_or_read_at() with a reused reader.

fn get_pushed_or_read_at_unwrap(&self, index: usize, reader: &Reader) -> T

Gets value from pushed layer or storage at usize index using provided reader, unwrapping the result. Panics if the read fails or if the value doesn’t exist.

fn get_pushed_or_read_at_unwrap_once(&self, index: usize) -> T

Gets value from pushed layer or storage at usize index, unwrapping the result. Panics if the read fails or if the value doesn’t exist. For multiple reads, prefer get_pushed_or_read_at_unwrap() with a reused reader.

fn get_pushed_at(&self, index: usize, stored_len: usize) -> Option<&T>

Gets value from pushed layer only (no disk reads).

fn len_(&self) -> usize

Returns the length including both stored and pushed (uncommitted) values.

fn pushed_len(&self) -> usize

Returns the number of pushed (uncommitted) values in the memory buffer.

fn is_pushed_empty(&self) -> bool

Returns true if there are no pushed (uncommitted) values.

fn has(&self, index: I) -> bool

Returns true if the index is within the length.

fn has_at(&self, index: usize) -> bool

Returns true if the usize index is within the length.

fn push(&mut self, value: T)

Pushes a new value to the end of the vector.

fn push_if_needed(&mut self, index: I, value: T) -> Result<()>

Pushes a value if the index equals the current length, otherwise does nothing if already exists. Returns an error if the index is too high.

fn truncate_push(&mut self, index: I, value: T) -> Result<()>

Pushes a value at the given index, truncating if necessary.

fn truncate_push_at(&mut self, index: usize, value: T) -> Result<()>

Pushes a value at the given usize index, truncating if necessary.

fn batch_limit_reached(&self) -> bool

Returns true if the pushed cache has reached the batch limit (~1GiB).

fn truncate_if_needed(&mut self, index: I) -> Result<()>

Truncates the vector to the given index if the current length exceeds it.

fn truncate_if_needed_with_stamp( &mut self, index: I, stamp: Stamp, ) -> Result<()>

Truncates the vector to the given index if needed, updating the stamp.

fn clear(&mut self) -> Result<()>

Clears all values from the vector.

fn validate_computed_version_or_reset(&mut self, version: Version) -> Result<()>

Validates the computed version against the stored version, resetting if they don’t match.

fn changes_path(&self) -> PathBuf

Returns the path to the changes directory for this vector.

fn stamped_flush_maybe_with_changes( &mut self, stamp: Stamp, with_changes: bool, ) -> Result<()>

Flushes with the given stamp, optionally saving changes for rollback.

fn vec_region_name(&self) -> String

Returns the region name for this vector.

impl<I, T> ImportableVec for ZstdVec<I, T>

where I: VecIndex, T: ZstdVecValue,

fn import(db: &Database, name: &str, version: Version) -> Result<Self>

Import from database, creating if needed.

fn import_with(options: ImportOptions<'_>) -> Result<Self>

Import with custom options.

fn forced_import(db: &Database, name: &str, version: Version) -> Result<Self>

Import from database, resetting on version/format mismatch.

fn forced_import_with(options: ImportOptions<'_>) -> Result<Self>

Forced import with custom options.

impl<'a, I, T> IntoIterator for &'a ZstdVec<I, T>

where I: VecIndex, T: ZstdVecValue,

type Item = T

The type of the elements being iterated over.

type IntoIter = CompressedVecIterator<'a, I, T, ZstdStrategy<T>>

Which kind of iterator are we turning this into?

fn into_iter(self) -> Self::IntoIter

Creates an iterator from a value.

impl<I, T> IterableVec<I, T> for ZstdVec<I, T>

where I: VecIndex, T: ZstdVecValue,

fn iter(&self) -> BoxedVecIterator<'_, I, T>

fn create_lookback( &self, skip: usize, window: usize, min_start: usize, ) -> Lookback<'_, I, T>

where I: VecIndex, T: VecValue,

Create a windowed lookback for efficient windowed access. Uses a ring buffer if many items will be processed, otherwise uses direct access.

impl<I, T> TypedVec for ZstdVec<I, T>

where I: VecIndex, T: ZstdVecValue,

type I = I

The index type used to address elements in this vector.

type T = T

The value type stored in this vector.