Skip to main content

FileMetadataCache

Trait FileMetadataCache 

Source
pub trait FileMetadataCache: CacheAccessor<Path, CachedFileMetadataEntry> {
    // Required methods
    fn cache_limit(&self) -> usize;
    fn update_cache_limit(&self, limit: usize);
    fn list_entries(&self) -> HashMap<Path, FileMetadataCacheEntry>;
}
Expand description

Cache for file-embedded metadata.

This cache stores per-file metadata in the form of CachedFileMetadataEntry, which includes the ObjectMeta for validation.

For example, the built in ListingTable uses this cache to avoid parsing Parquet footers multiple times for the same file.

DataFusion provides a default implementation, DefaultFilesMetadataCache, and users can also provide their own implementations to implement custom caching strategies.

The typical usage pattern is:

  1. Call get(path) to check for cached value
  2. If Some(cached), validate with cached.is_valid_for(&current_meta)
  3. If invalid or missing, compute new value and call put(path, new_value)

See crate::runtime_env::RuntimeEnv for more details.

Required Methods§

Source

fn cache_limit(&self) -> usize

Returns the cache’s memory limit in bytes.

Source

fn update_cache_limit(&self, limit: usize)

Updates the cache with a new memory limit in bytes.

Source

fn list_entries(&self) -> HashMap<Path, FileMetadataCacheEntry>

Retrieves the information about the entries currently cached.

Trait Implementations§

Source§

impl Debug for dyn FileMetadataCache

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more

Implementors§