Struct ReadParams 

pub struct ReadParams {
    pub index_cache_size_bytes: usize,
    pub metadata_cache_size_bytes: usize,
    pub session: Option<Arc<Session>>,
    pub store_options: Option<ObjectStoreParams>,
    pub commit_handler: Option<Arc<dyn CommitHandler>>,
    pub file_reader_options: Option<FileReaderOptions>,
}

Customize read behavior of a dataset.

Fields

index_cache_size_bytes: usize

Size of the index cache in bytes. This cache stores index data in memory for faster lookups. The default is 6 GiB.

metadata_cache_size_bytes: usize

Size of the metadata cache in bytes. This cache stores metadata in memory for faster open table and scans. The default is 1 GiB.

session: Option<Arc<Session>>

If present, dataset will use this shared Session instead creating a new one.

This is useful for sharing the same session across multiple datasets.

store_options: Option<ObjectStoreParams>

commit_handler: Option<Arc<dyn CommitHandler>>

If present, dataset will use this to resolve the latest version

Lance needs to be able to make atomic updates to the manifest. This involves coordination between readers and writers and we can usually rely on the filesystem to do this coordination for us.

Some file systems (e.g. S3) do not support atomic operations. In this case, for safety, we recommend an external commit mechanism (such as dynamodb) and, on the read path, we need to reach out to that external mechanism to figure out the latest version of the dataset.

If this is not set then a default behavior is chosen that is appropriate for the filesystem.

If a custom object store is provided (via store_params.object_store) then this must also be provided.

file_reader_options: Option<FileReaderOptions>

File reader options to use when reading data files.

This allows control over features like caching repetition indices and validation.

Implementations

impl ReadParams

pub fn index_cache_size(&mut self, cache_size: usize) -> &mut Self

👎Deprecated since 0.30.0: Use index_cache_size_bytes instead, which accepts a size in bytes.

Set the cache size for indices. Set to zero, to disable the cache.

pub fn index_cache_size_bytes(&mut self, cache_size: usize) -> &mut Self

pub fn metadata_cache_size(&mut self, cache_size: usize) -> &mut Self

👎Deprecated since 0.30.0: Use metadata_cache_size_bytes instead, which accepts a size in bytes.

Set the cache size for the file metadata. Set to zero to disable this cache.

pub fn metadata_cache_size_bytes(&mut self, cache_size: usize) -> &mut Self

Set the cache size for the file metadata in bytes.

pub fn session(&mut self, session: Arc<Session>) -> &mut Self

Set a shared session for the datasets.

pub fn set_commit_lock<T: CommitLock + Send + Sync + 'static>( &mut self, lock: Arc<T>, )

Use the explicit locking to resolve the latest version

pub fn file_reader_options(&mut self, options: FileReaderOptions) -> &mut Self

Set the file reader options.

Trait Implementations

impl Clone for ReadParams

fn clone(&self) -> ReadParams

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ReadParams

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ReadParams

fn default() -> Self

Returns the “default value” for a type.