Crate delta_kernel

Engineinterface interfaces

The Engineinterface interfaces allow connectors to bring their own implementation of functionality such as reading parquet files, listing files in a file system, parsing a JSON string etc.

The EngineInterface trait exposes methods to get sub-clients which expose the core functionalities customizable by connectors.

Expression handling

Expression handling is done via the ExpressionHandler, which in turn allows the creation of ExpressionEvaluators. These evaluators are created for a specific predicate Expression and allow evaluation of that predicate for a specific batches of data.

File system interactions

Delta Kernel needs to perform some basic operations against file systems like listing and reading files. These interactions are encapsulated in the FileSystemClient trait. Implementors must take take care that all assumptions on the behavior if the functions - like sorted results - are respected.

Reading log and data files

Delta Kernel requires the capability to read json and parquet files, which is exposed via the JsonHandler and ParquetHandler respectively. When reading files, connectors are asked to provide the context information it requires to execute the actual read. This is done by invoking methods on the FileSystemClient trait.

Re-exports

• pub use engine_data::DataVisitor;
• pub use engine_data::EngineData;
• pub use error::DeltaResult;
• pub use error::Error;
• pub use expressions::Expression;
• pub use table::Table;

Modules

• actions
• client
  module for clients that are optionally built into the kernel
• engine_data
  Traits that engines need to implement in order to pass data between themselves and kernel.
• error
• expressions
• path
  Utilities to make working with directory and file paths easier
• scan
• schema
• snapshot
  In-memory representation of snapshots of tables (snapshot is a table at given point in time, it has schema etc.)
• table

Structs

• FileMeta
  The metadata that describes an object.

Traits

• EngineInterface
  Interface encapsulating all clients needed by the Delta Kernel in order to read the Delta table.
• ExpressionEvaluator
  Interface for implementing an Expression evaluator.
• ExpressionHandler
  Provides expression evaluation capability to Delta Kernel.
• FileSystemClient
  Provides file system related functionalities to Delta Kernel.
• JsonHandler
  Provides JSON handling functionality to Delta Kernel.
• ParquetHandler
  Provides Parquet file related functionalities to Delta Kernel.

Type Aliases

• FileDataReadResult
  Data read from a Delta table file and the corresponding scan file information.
• FileDataReadResultIterator
• FileSlice
• Version
  Delta table version is 8 byte unsigned int