Struct Manifest 

pub struct Manifest {

    pub fields: Vec<Field>,
    pub schema_metadata: HashMap<String, Vec<u8>>,
    pub fragments: Vec<DataFragment>,
    pub version: u64,
    pub version_aux_data: u64,
    pub writer_version: Option<WriterVersion>,
    pub index_section: Option<u64>,
    pub timestamp: Option<Timestamp>,
    pub tag: String,
    pub reader_feature_flags: u64,
    pub writer_feature_flags: u64,
    pub max_fragment_id: Option<u32>,
    pub transaction_file: String,
    pub transaction_section: Option<u64>,
    pub next_row_id: u64,
    pub data_format: Option<DataStorageFormat>,
    pub config: HashMap<String, String>,
    pub table_metadata: HashMap<String, String>,
    pub base_paths: Vec<BasePath>,
    pub branch: Option<String>,
}

Manifest is a global section shared between all the files.

Fields

fields: Vec<Field>

All fields of the dataset, including the nested fields.

schema_metadata: HashMap<String, Vec<u8>>

Schema metadata.

fragments: Vec<DataFragment>

Fragments of the dataset.

version: u64

Snapshot version number.

version_aux_data: u64

The file position of the version auxiliary data.

• It is not inheritable between versions.
• It is not loaded by default during query.

writer_version: Option<WriterVersion>

The version of the writer that created this file.

This information may be used to detect whether the file may have known bugs associated with that writer.

index_section: Option<u64>

If present, the file position of the index metadata.

timestamp: Option<Timestamp>

Version creation Timestamp, UTC timezone

tag: String

Optional version tag

reader_feature_flags: u64

Feature flags for readers.

A bitmap of flags that indicate which features are required to be able to read the table. If a reader does not recognize a flag that is set, it should not attempt to read the dataset.

Known flags:

• 1: deletion files are present
• 2: row ids are stable and stored as part of the fragment metadata.
• 4: use v2 format (deprecated)
• 8: table config is present

writer_feature_flags: u64

Feature flags for writers.

A bitmap of flags that indicate which features must be used when writing to the dataset. If a writer does not recognize a flag that is set, it should not attempt to write to the dataset.

The flag identities are the same as for reader_feature_flags, but the values of reader_feature_flags and writer_feature_flags are not required to be identical.

max_fragment_id: Option<u32>

The highest fragment ID that has been used so far.

This ID is not guaranteed to be present in the current version, but it may have been used in previous versions.

For a single fragment, will be zero. For no fragments, will be absent.

transaction_file: String

Path to the transaction file, relative to {root}/_transactions. The file at that location contains a wire-format serialized Transaction message representing the transaction that created this version.

This string field “transaction_file” may be empty if no transaction file was written.

The path format is “{read_version}-{uuid}.txn” where {read_version} is the version of the table the transaction read from (serialized to decimal with no padding digits), and {uuid} is a hyphen-separated UUID.

transaction_section: Option<u64>

The file position of the transaction content. None if transaction is empty This transaction content begins with the transaction content length as u32 If the transaction proto message has a length of len, the message ends at len + 4

next_row_id: u64

The next unused row id. If zero, then the table does not have any rows.

This is only used if the “stable_row_ids” feature flag is set.

data_format: Option<DataStorageFormat>

The data storage format

This specifies what format is used to store the data files.

config: HashMap<String, String>

Table config.

Keys with the prefix “lance.” are reserved for the Lance library. Other libraries may wish to similarly prefix their configuration keys appropriately.

table_metadata: HashMap<String, String>

Metadata associated with the table.

This is a key-value map that can be used to store arbitrary metadata associated with the table.

This is different than configuration, which is used to tell libraries how to read, write, or manage the table.

This is different than schema metadata, which is used to describe the data itself and is attached to the output schema of scans.

base_paths: Vec<BasePath>

The base paths of data files.

This is used to determine the base path of a data file. In common cases data file paths are under current dataset base path. But for shallow cloning, importing file and other multi-tier storage cases, the actual data files could be outside of the current dataset. This field is used with the base_id in lance.file.File and lance.file.DeletionFile.

For example, if we have a dataset with base path s3://bucket/dataset, we have a DataFile with base_id 0, we get the actual data file path by: base_paths[id = 0] + /data/ + file.path the key(a.k.a index) starts from 0, increased by 1 for each new base path.

branch: Option<String>

The branch of the dataset. None means main branch.

Implementations

impl Manifest

pub fn index_section(&self) -> u64

Returns the value of index_section, or the default value if index_section is unset.

pub fn max_fragment_id(&self) -> u32

Returns the value of max_fragment_id, or the default value if max_fragment_id is unset.

pub fn branch(&self) -> &str

Returns the value of branch, or the default value if branch is unset.

pub fn transaction_section(&self) -> u64

Returns the value of transaction_section, or the default value if transaction_section is unset.

Trait Implementations

impl Clone for Manifest

fn clone(&self) -> Manifest

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Manifest

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for Manifest

fn default() -> Self

Returns the “default value” for a type.

impl From<&Manifest> for Manifest

fn from(m: &Manifest) -> Self

Converts to this type from the input type.

impl Message for Manifest

fn encoded_len(&self) -> usize

Returns the encoded length of the message without a length delimiter.

fn clear(&mut self)

Clears the message, resetting all fields to their default.

fn encode(&self, buf: &mut impl BufMut) -> Result<(), EncodeError>

where Self: Sized,

Encodes the message to a buffer.

fn encode_to_vec(&self) -> Vec<u8>

where Self: Sized,

Encodes the message to a newly allocated buffer.

fn encode_length_delimited( &self, buf: &mut impl BufMut, ) -> Result<(), EncodeError>

where Self: Sized,

Encodes the message with a length-delimiter to a buffer.

fn encode_length_delimited_to_vec(&self) -> Vec<u8>

where Self: Sized,

Encodes the message with a length-delimiter to a newly allocated buffer.

fn decode(buf: impl Buf) -> Result<Self, DecodeError>

where Self: Default,

Decodes an instance of the message from a buffer.

fn decode_length_delimited(buf: impl Buf) -> Result<Self, DecodeError>

where Self: Default,

Decodes a length-delimited instance of the message from the buffer.

fn merge(&mut self, buf: impl Buf) -> Result<(), DecodeError>

where Self: Sized,

Decodes an instance of the message from a buffer, and merges it into self.

fn merge_length_delimited(&mut self, buf: impl Buf) -> Result<(), DecodeError>

where Self: Sized,

Decodes a length-delimited instance of the message from buffer, and merges it into self.

impl Name for Manifest

const NAME: &'static str = "Manifest"

Simple name for this Message. This name is the same as it appears in the source .proto file, e.g. FooBar.

const PACKAGE: &'static str = "lance.table"

Package name this message type is contained in. They are domain-like and delimited by ., e.g. google.protobuf.

fn full_name() -> String

Fully-qualified unique name for this Message. It’s prefixed with the package name and names of any parent messages, e.g. google.rpc.BadRequest.FieldViolation. By default, this is the package name followed by the message name. Fully-qualified names must be unique within a domain of Type URLs.

fn type_url() -> String

Type URL for this Message, which by default is the full name with a leading slash, but may also include a leading domain name, e.g. type.googleapis.com/google.profile.Person. This can be used when serializing into the google.protobuf.Any type.

impl PartialEq for Manifest

fn eq(&self, other: &Manifest) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl TryFrom<Manifest> for Manifest

type Error = Error

The type returned in the event of a conversion error.

fn try_from(p: Manifest) -> Result<Self>

Performs the conversion.

impl StructuralPartialEq for Manifest