skade-katalog 0.1.8

The katalog under skade: an embedded, single-file ACID Apache Iceberg catalog (redb) with time-travel snapshots and atomic multi-table release commits — the Norns recording the world's icebergs.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72

// Apache-2.0 licensed. See LICENSE-APACHE.

//! Pure-Rust, embedded Apache Iceberg catalog backed by [redb].
//!
//! `nornir-catalog` implements the `iceberg::Catalog` trait against a
//! single-file [redb] database. All catalog metadata (namespaces, namespace
//! properties, table-pointer rows) lives inside one `.redb` file. Iceberg
//! table metadata JSON / Avro manifests / Parquet data files live in whatever
//! storage backend the supplied `FileIO` points at (local fs, S3, GCS, …).
//!
//! ## Why redb?
//!
//! * **Pure Rust** — no `libsqlite3`, no JVM, no Postgres process.
//! * **ACID** — every catalog mutation is a single redb `WriteTransaction`,
//!   so concurrent writers in the same process are serialized correctly and
//!   crashes leave the database consistent.
//! * **No async runtime** in the storage layer — redb is sync; the
//!   `Catalog` trait wrapper is async only because the trait demands it.
//! * **Multi-table atomic commits** — because every redb write transaction
//!   spans every table in the database, this crate exposes
//!   [`RedbCatalog::atomic_release`] which commits N
//!   `iceberg::TableCommit` values atomically. Useful when one logical
//!   release ("publish bench_runs + dep_graph + components together") must
//!   either succeed entirely or roll back entirely.
//!
//! ## Quickstart
//!
//! ```no_run
//! use std::sync::Arc;
//! use iceberg::CatalogBuilder;
//! use iceberg::io::LocalFsStorageFactory;
//! use skade_katalog::RedbCatalogBuilder;
//!
//! # async fn run() -> anyhow::Result<()> {
//! let catalog = RedbCatalogBuilder::default()
//!     .db_path("/var/lib/myapp/catalog.redb")
//!     .warehouse_location("file:///var/lib/myapp/warehouse")
//!     .with_storage_factory(Arc::new(LocalFsStorageFactory))
//!     .load("myapp", std::collections::HashMap::new())
//!     .await?;
//! # Ok(()) }
//! ```
//!
//! ## What this crate is NOT
//!
//! * Not a distributed catalog. Single-process. Multiple processes opening
//!   the same file will be rejected by redb's file lock.
//! * Not a REST endpoint. If you need that, run something like Lakekeeper.
//! * Not schema-evolution-complete: iceberg-rust 0.9.1 does not yet expose
//!   public `TransactionAction`s for schema mutations. That's an upstream
//!   gap; once 0.10 lands it will work here unchanged.

mod atomic;
mod builder;
mod catalog;
mod error;
mod keys;
mod meta_cache;
mod pointer_cache;
mod static_index;
mod store;
mod table_cache;

pub use builder::{
    REDB_CATALOG_PROP_DB_PATH, REDB_CATALOG_PROP_DURABILITY,
    REDB_CATALOG_PROP_METADATA_CACHE_BYTES, REDB_CATALOG_PROP_TABLE_HANDLE_CACHE_CAPACITY,
    REDB_CATALOG_PROP_WAREHOUSE, RedbCatalogBuilder, WriteDurability,
};
pub use catalog::RedbCatalog;
pub use error::RedbCatalogError;
pub use meta_cache::DEFAULT_METADATA_CACHE_BYTES;
pub use table_cache::DEFAULT_TABLE_HANDLE_CACHE_CAPACITY;