Struct IndexDef

pub struct IndexDef { /* private fields */ }

Definition of an index. It includes: name, function to extract index keys, and how much the index can lag on disk.

Implementations

impl IndexDef

pub fn new( name: impl ToString, index_func: impl Fn(&[u8]) -> Vec<IndexOutput> + Send + Sync + 'static, ) -> Self

Create an index definition.

index_func is the function to extract index keys from an entry.

The input is bytes of an entry (ex. the data passed to Log::append). The output is an array of index keys. An entry can have zero or more than one index keys for a same index.

The output can be an allocated slice of bytes, or a reference to offsets in the input. See IndexOutput for details.

The function should be pure and fast. i.e. It should not use inputs from other things, like the network, filesystem, or an external random generator.

For example, if the Log is to store git commits, and the index is to help finding child commits given parent commit hashes as index keys. This function gets the commit metadata as input. It then parses the input, and extract parent commit hashes as the output. A git commit can have 0 or 1 or 2 or even more parents. Therefore the output is a Vec.

name is the name of the index.

The name will be used as part of the index file name. Therefore do not use user-generated content here. And do not abuse this by using .. or /.

When adding new or changing index functions, make sure a different name is used so the existing index won’t be reused incorrectly.

pub fn lag_threshold(self, lag_threshold: u64) -> Self

Set how many bytes (as counted in the file backing Log) could be left not indexed on-disk.

This is related to Index implementation detail. Since it’s append-only and needs to write O(log N) data for updating a single entry. Allowing lagged indexes reduces writes and saves disk space.

The lagged part of the index will be built on-demand in-memory by Log::open.

Practically, this correlates to how fast func is.

Trait Implementations

impl Clone for IndexDef

fn clone(&self) -> IndexDef

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.