Struct IndexMetadata 

pub struct IndexMetadata {
    pub uuid: Uuid,
    pub fields: Vec<i32>,
    pub covering_fields: Vec<i32>,
    pub name: String,
    pub dataset_version: u64,
    pub fragment_bitmap: Option<RoaringBitmap>,
    pub index_details: Option<Arc<Any>>,
    pub index_version: i32,
    pub created_at: Option<DateTime<Utc>>,
    pub base_id: Option<u32>,
    pub files: Option<Vec<IndexFile>>,
}

Index metadata

Fields

uuid: Uuid

Unique ID across all dataset versions.

fields: Vec<i32>

Fields to build the index.

fields[0] is always a column the index is keyed on. Trailing entries may instead be merely carried, not keyed on – see Self::covering_fields.

covering_fields: Vec<i32>

Fields whose values this index carries but is not keyed on.

Always a suffix of Self::fields, and never all of it, so fields[0] is always a column the index is keyed on. Empty for an index that carries no extra columns.

These ids also appear in Self::fields. That is deliberate: every consumer that reads fields as the index’s dependency set then covers them with no change.

name: String

Human readable index name

dataset_version: u64

The version of the dataset this index was last updated on

This is set when the index is created (based on the version used to train the index) This is updated when the index is updated or remapped

fragment_bitmap: Option<RoaringBitmap>

The fragment ids this index covers.

This may contain fragment ids that no longer exist in the dataset.

If this is None, then this is unknown.

index_details: Option<Arc<Any>>

Metadata specific to the index type

This is an Option because older versions of Lance may not have this defined. However, it should always be present in newer versions.

index_version: i32

The index version.

created_at: Option<DateTime<Utc>>

Timestamp when the index was created

This field is optional for backward compatibility. For existing indices created before this field was added, this will be None.

base_id: Option<u32>

The base path index of the index files. Used when the index is imported or referred from another dataset. Lance uses it as key of the base_paths field in Manifest to determine the actual base path of the index files.

files: Option<Vec<IndexFile>>

List of files and their sizes for this index segment. This enables skipping HEAD calls when opening indices and provides visibility into index storage size via describe_indices(). This is None if the file sizes are unknown. This happens for indices created before this field was added.

Implementations

impl IndexMetadata

pub fn effective_fragment_bitmap( &self, existing_fragments: &RoaringBitmap, ) -> Option<RoaringBitmap>

pub fn file_size_map(&self) -> HashMap<String, u64>

Returns a map of relative file paths to their sizes. Returns an empty map if file information is not available.

pub fn total_size_bytes(&self) -> Option<u64>

Returns the total size of all files in this index segment in bytes. Returns None if file information is not available.

pub fn deleted_fragment_bitmap( &self, existing_fragments: &RoaringBitmap, ) -> Option<RoaringBitmap>

Returns the set of fragments which are part of the fragment bitmap but no longer in the dataset.

pub fn results_are_row_addrs(&self) -> bool

True when the index reports matches as physical row addresses rather than row ids (ScalarIndex::results_are_row_addresses).

Such an index cannot follow its data through a rewrite: the addresses it stores name fragments and offsets, and neither kind supports remap.

pub fn keyed_fields(&self) -> &[i32]

The prefix of Self::fields this index is keyed on, with the carried columns of Self::covering_fields removed.

Only this prefix decides which column an index answers for; the full fields vector answers what invalidates it. Empty for a system index that declares no fields, and empty for a declaration longer than fields: decoding validates, but metadata built by a caller this build never validated does not, and failing closed beats an underflow.

pub fn keyed_field(&self) -> Option<i32>

The single column this index is keyed on, or None when it is keyed on several – a genuinely composite index – or on none at all.

Most selection paths are only defined for one keyed column, so they can compare this against the column they are resolving.

pub fn validate_covering_fields(&self) -> Result<()>

Check the covering declaration against Self::fields.

Carried columns must be a suffix of fields and must not consume all of it, so fields[0] is always a column the index is keyed on. An empty declaration is always valid, which is what keeps the system indices – mem_wal and frag_reuse, both of which commit no fields at all – passing this check.

The rules are checked from most to least specific, because one bad declaration usually trips several: an id that is not a field at all is reported ahead of the length and ordering rules, which would otherwise name a consequence instead of the cause.

Trait Implementations

impl Clone for IndexMetadata

fn clone(&self) -> IndexMetadata

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for IndexMetadata

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl DeepSizeOf for IndexMetadata

fn deep_size_of_children(&self, context: &mut Context) -> usize

fn deep_size_of(&self) -> usize

impl From<&IndexMetadata> for IndexMetadata

fn from(idx: &IndexMetadata) -> Self

Converts to this type from the input type.

impl PartialEq for IndexMetadata

fn eq(&self, other: &IndexMetadata) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for IndexMetadata

impl TryFrom<IndexMetadata> for IndexMetadata

type Error = Error

The type returned in the event of a conversion error.

fn try_from(proto: IndexMetadata) -> Result<Self>

Performs the conversion.