1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
//! # rig-ballista
//!
//! Apache Ballista + DataFusion + Iceberg companion crate for
//! [`rig-compose`](https://crates.io/crates/rig-compose).
//!
//! **Status:** scaffolding. The intent is to provide a `MetadataCatalog`-
//! shaped trait impl that reads Iceberg tables (via `datafusion-iceberg`)
//! and pushes prunable scans through a Ballista distributed query engine.
//! The seam exists today as `azreal::storage::MetadataCatalog`; this
//! crate will plug behind it without re-exporting Iceberg/Ballista types
//! into the rig-compose surface.
//!
//! ## Why a separate crate
//! - **MSRV isolation.** `iceberg-rust` 0.9 currently requires rustc 1.92;
//! pinning that into the rig-compose tree would force every consumer
//! onto the same toolchain.
//! - **Compile-cost isolation.** Ballista pulls in DataFusion, Arrow,
//! gRPC, Parquet — none of which the kernel needs.
//! - **Boundary discipline.** Ballista is a query-engine boundary, not a
//! file-format boundary. Keeping it behind a trait keeps the skill/tool
//! surface clean.
//!
//! ## Planned surface
//!
//! ```ignore
//! pub trait MetadataCatalog: Send + Sync {
//! async fn list_files(&self, partition: Option<&str>) -> Result<Vec<FileStats>, StorageError>;
//! async fn get(&self, id: FileId) -> Result<FileStats, StorageError>;
//! }
//!
//! pub struct BallistaIcebergCatalog { /* ... */ }
//! impl MetadataCatalog for BallistaIcebergCatalog { /* ... */ }
//! ```
//!
//! Concrete code will land once a throwaway companion verifies the
//! `iceberg-rust` + `datafusion-iceberg` + `ballista` combination
//! compiles on a recent stable toolchain.
/// Placeholder until the real catalog implementation lands.
;