Crate delta_kernel
source ·Expand description
§Engineinterface interfaces
The Engineinterface interfaces allow connectors to bring their own implementation of functionality such as reading parquet files, listing files in a file system, parsing a JSON string etc.
The EngineInterface
trait exposes methods to get sub-clients which expose the core
functionalities customizable by connectors.
§Expression handling
Expression handling is done via the ExpressionHandler
, which in turn allows the creation
of ExpressionEvaluator
s. These evaluators are created for a specific predicate Expression
and allow evaluation of that predicate for a specific batches of data.
§File system interactions
Delta Kernel needs to perform some basic operations against file systems like listing and reading files.
These interactions are encapsulated in the FileSystemClient
trait. Implementors must take take
care that all assumptions on the behavior if the functions - like sorted results - are respected.
§Reading log and data files
Delta Kernel requires the capability to read json and parquet files, which is exposed via the
JsonHandler
and ParquetHandler
respectively. When reading files, connectors are asked to
provide the context information it requires to execute the actual read. This is done by invoking
methods on the FileSystemClient
trait.
Re-exports§
pub use engine_data::DataVisitor;
pub use engine_data::EngineData;
pub use error::DeltaResult;
pub use error::Error;
pub use expressions::Expression;
pub use table::Table;
Modules§
- module for clients that are optionally built into the kernel
- Traits that engines need to implement in order to pass data between themselves and kernel.
- Utilities to make working with directory and file paths easier
- In-memory representation of snapshots of tables (snapshot is a table at given point in time, it has schema etc.)
Structs§
- The metadata that describes an object.
Traits§
- Interface encapsulating all clients needed by the Delta Kernel in order to read the Delta table.
- Interface for implementing an Expression evaluator.
- Provides expression evaluation capability to Delta Kernel.
- Provides file system related functionalities to Delta Kernel.
- Provides JSON handling functionality to Delta Kernel.
- Provides Parquet file related functionalities to Delta Kernel.
Type Aliases§
- Data read from a Delta table file and the corresponding scan file information.
- Delta table version is 8 byte unsigned int