Expand description
§The Default Engine
The default implementation of Engine is DefaultEngine.
The underlying implementations use asynchronous IO. Async tasks are run on
a separate thread pool, provided by the TaskExecutor trait. Read more in
the executor module.
Modules§
- executor
- The default engine uses Async IO to read files, but the kernel APIs are all synchronous. Therefore, we need an executor to run the async IO on in the background.
- file_
stream - filesystem
- json
- Default Json handler implementation
- parquet
- Default Parquet handler implementation
- stats
- Statistics collection for Delta Lake file writes.
- storage
Structs§
- Default
Engine - Default
Engine Builder - Builder for creating
DefaultEngineinstances. - Default
Task Executor - Represents the default
TaskExecutor. The executor is created lazily to avoid unnecessary instantiations.
Functions§
- build_
add_ file_ metadata - Converts
DataFileMetadatainto Add actionEngineDatausing the partition values and table root from the providedWriteContext.