Struct DataOverlayFile 

pub struct DataOverlayFile {
    pub data_file: Option<DataFile>,
    pub committed_version: u64,
    pub coverage: Option<Coverage>,
}

An overlay file supplies new values for a subset of (row offset, field) cells within a fragment, without rewriting the fragment’s base data files. It is used for efficient updates when only a small fraction of rows and/or columns change.

On read, a cell is resolved by consulting the fragment’s overlays from newest to oldest: the first overlay that covers that (offset, field) wins; if none cover it, the value falls through to the base data file. Because deletions take precedence over overlays, an overlay value for an offset that is also marked deleted is dead and is ignored.

The overlay’s data file does NOT store a row-offset key column. Within a value column, the position of a covered offset’s value is the rank (0-based count of set bits below it) of that offset within the field’s coverage bitmap. Because fields may cover different offset sets, the value columns of a single overlay data file may have different lengths (which the Lance file format permits).

Fields

data_file: Option<DataFile>

The data file storing the overlay’s new cell values, one value column per field in data_file.fields. No row-offset key column is stored.

committed_version: u64

The dataset version at which this overlay became effective: the version of the commit that introduced it, NOT the version it was read from. It is stamped at commit time and re-stamped if the commit is retried, in the same way as the created-at / last-updated-at version sequences.

This drives two orderings:

• Versus index builds: an index whose dataset_version >= this value already incorporates this overlay. Otherwise the overlay’s covered cells are excluded from index results for the affected fields and re-evaluated against their current values (see the Data Overlay Files specification).
• Versus other overlays: when two overlays cover the same (offset, field), the one with the higher committed_version wins. Overlays that share a committed_version are ordered by their position in DataFragment.overlays, where a later entry is newer and wins.

coverage: Option<Coverage>

Which (offset, field) cells this overlay provides values for.

Trait Implementations

impl Clone for DataOverlayFile

fn clone(&self) -> DataOverlayFile

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for DataOverlayFile

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for DataOverlayFile

fn default() -> Self

Returns the “default value” for a type.

impl Eq for DataOverlayFile

impl Hash for DataOverlayFile

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl Message for DataOverlayFile

fn encoded_len(&self) -> usize

Returns the encoded length of the message without a length delimiter.

fn clear(&mut self)

Clears the message, resetting all fields to their default.

fn encode(&self, buf: &mut impl BufMut) -> Result<(), EncodeError>

where Self: Sized,

Encodes the message to a buffer.

fn encode_to_vec(&self) -> Vec<u8>

where Self: Sized,

Encodes the message to a newly allocated buffer.

fn encode_length_delimited( &self, buf: &mut impl BufMut, ) -> Result<(), EncodeError>

where Self: Sized,

Encodes the message with a length-delimiter to a buffer.

fn encode_length_delimited_to_vec(&self) -> Vec<u8>

where Self: Sized,

Encodes the message with a length-delimiter to a newly allocated buffer.

fn decode(buf: impl Buf) -> Result<Self, DecodeError>

where Self: Default,

Decodes an instance of the message from a buffer.

fn decode_length_delimited(buf: impl Buf) -> Result<Self, DecodeError>

where Self: Default,

Decodes a length-delimited instance of the message from the buffer.

fn merge(&mut self, buf: impl Buf) -> Result<(), DecodeError>

where Self: Sized,

Decodes an instance of the message from a buffer, and merges it into self.

fn merge_length_delimited(&mut self, buf: impl Buf) -> Result<(), DecodeError>

where Self: Sized,

Decodes a length-delimited instance of the message from buffer, and merges it into self.

impl PartialEq for DataOverlayFile

fn eq(&self, other: &DataOverlayFile) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for DataOverlayFile