Expand description
DataFusion-backed OLAP engine.
Uses Apache DataFusion as the query engine with pluggable storage:
- InMemory:
Vec<RecordBatch>in aHashMap(default, lost on shutdown) - ArrowIpc: persistent
.arrowfiles per table - Parquet: persistent
.parquetfiles per table
§DML Strategy
DataFusion’s MemTable / ListingTable don’t support INSERT/UPDATE/DELETE DML.
Instead, we maintain our own table store (in-memory or file-backed) and
re-register tables with DataFusion after mutations. For queries,
DataFusion’s SQL engine executes against the registered tables.
Structs§
- Data
Fusion Engine - DataFusion-backed OLAP engine.
- Shared
Data Fusion Engine - A cheaply-cloneable,
Arc-wrappedDataFusionEnginethat implementsrhei_core::OlapEngine.