Crate skade_katalog 

Pure-Rust, embedded Apache Iceberg catalog backed by redb.

skade-katalog implements the iceberg::Catalog trait against a single-file redb database. All catalog metadata (namespaces, namespace properties, table-pointer rows) lives inside one .redb file. Iceberg table metadata JSON / Avro manifests / Parquet data files live in whatever storage backend the supplied FileIO points at (local fs, S3, GCS, …).

Why redb?

• Pure Rust — no libsqlite3, no JVM, no Postgres process.
• ACID — every catalog mutation is a single redb WriteTransaction, so concurrent writers in the same process are serialized correctly and crashes leave the database consistent.
• No async runtime in the storage layer — redb is sync; the Catalog trait wrapper is async only because the trait demands it.
• Multi-table atomic commits — because every redb write transaction spans every table in the database, this crate exposes RedbCatalog::atomic_release which commits N iceberg::TableCommit values atomically. Useful when one logical release (“publish bench_runs + dep_graph + components together”) must either succeed entirely or roll back entirely.

Quickstart

use std::sync::Arc;
use iceberg::CatalogBuilder;
use iceberg::io::LocalFsStorageFactory;
use skade_katalog::RedbCatalogBuilder;

let catalog = RedbCatalogBuilder::default()
    .db_path("/var/lib/myapp/catalog.redb")
    .warehouse_location("file:///var/lib/myapp/warehouse")
    .with_storage_factory(Arc::new(LocalFsStorageFactory))
    .load("myapp", std::collections::HashMap::new())
    .await?;

What this crate is NOT

• Not a distributed catalog. Single-process. Multiple processes opening the same file will be rejected by redb’s file lock.
• Not a REST endpoint. If you need that, run something like Lakekeeper.
• Not schema-evolution-complete: iceberg-rust 0.9.1 does not yet expose public TransactionActions for schema mutations. That’s an upstream gap; once 0.10 lands it will work here unchanged.

Structs

BranchRetention

Retention policy for a branch (git branches never garbage-collect their own head, but Iceberg can expire older snapshots on them). All None uses the table-property defaults. Mirrors the fields of SnapshotRetention::Branch.

CommitEvent

A durable catalog commit became visible: the streaming spine’s unit of change. Published on a tokio::sync::broadcast after the redb write commits and the L1 mirror is updated, and also reconstructable from the durable [COMMIT_LOG] via crate::RedbCatalog::commits_since. Cheap to clone (Arc-backed strings); a dropped event on a no-receiver channel is fine — the broadcast is a low-latency hint, the durable log is the truth.

RedbCatalog

Embedded Iceberg catalog backed by redb.

RedbCatalogBuilder

Builder for RedbCatalog.

RefEntry

One entry of a table’s refs map: a branch or tag and the snapshot it names.

Enums

HealOutcome

What RedbCatalog::heal_table did.

RedbCatalogError

Errors that may arise from the redb catalog backend itself.

RefKind

Whether a ref is a movable branch or an immutable tag.

WriteDurability

Durability applied to every catalog-mutation commit. Maps onto redb’s durability levels.

Constants

COMMIT_EVENT_BUFFER

Broadcast ring capacity: how many commit events a lagging subscriber may fall behind before it observes RecvError::Lagged and must catch up via the durable crate::RedbCatalog::commits_since. The broadcast is intentionally lossy under slow consumers (bounded memory); the durable log recovers correctness.

DEFAULT_METADATA_CACHE_BYTES

Default budget for the parsed-metadata cache (128 MiB).

DEFAULT_TABLE_HANDLE_CACHE_BYTES

Default budget for the built-Table handle cache (256 MiB), in BYTES.

DEFAULT_TABLE_HANDLE_CACHE_CAPACITYDeprecated

Deprecated entry-count default. Kept so existing callers of crate::builder::RedbCatalogBuilder::table_handle_cache_capacity compile; a count is converted to a byte budget at the worst case per handle.

REDB_CATALOG_PROP_DB_PATH

Property key for the redb database file path. May be set either via RedbCatalogBuilder::db_path or in the props map passed to iceberg::CatalogBuilder::load.

REDB_CATALOG_PROP_DURABILITY

Property key for write durability. Values: immediate (default, fsync per commit), eventual, none. May be set via RedbCatalogBuilder::durability or in the props map.

REDB_CATALOG_PROP_METADATA_CACHE_BYTES

Property key for the L0 parsed-metadata cache budget, in bytes. May be set via RedbCatalogBuilder::metadata_cache_bytes or in the props map. A value of 0 disables the cache. Defaults to crate::DEFAULT_METADATA_CACHE_BYTES (128 MiB) when unset.

REDB_CATALOG_PROP_TABLE_HANDLE_CACHE_BYTES

Property key for the L1.5 built-Table handle cache budget, in bytes. May be set via RedbCatalogBuilder::table_handle_cache_bytes or in the props map. A value of 0 disables the cache. Defaults to crate::DEFAULT_TABLE_HANDLE_CACHE_BYTES (256 MiB) when unset.

REDB_CATALOG_PROP_TABLE_HANDLE_CACHE_CAPACITY

Deprecated property key: the L1.5 cache expressed as a handle COUNT. An entry count cannot bound a cache whose entries are unbounded — see [crate::table_cache]. Still honoured (converted to bytes at the worst case per handle) so existing configs keep working, and ignored when REDB_CATALOG_PROP_TABLE_HANDLE_CACHE_BYTES is also set.

REDB_CATALOG_PROP_WAREHOUSE

Property key for the warehouse root URI. May be set either via RedbCatalogBuilder::warehouse_location or in the props map passed to iceberg::CatalogBuilder::load.

TABLE_OBJECT_CACHE_BYTES

Per-handle budget for iceberg’s own manifest/manifest-list ObjectCache.

VERSION

This crate’s semantic version (CARGO_PKG_VERSION) — the canonical skade product version. Exposed so the skade-katalog serve binary and any co-located library caller can surface one consistent version string (paired with the build’s git commit) on a --version / health / version surface.

Functions

redb_commit_count

Process-global count of durable redb write transactions on the data / pointer-advance paths — the observable fsync count of the write path (group-commit, atomic_release{,_raw}, swap_table). One per flush, regardless of how many tables it advances; DDL commits do not bump it. Used to prove skade’s cross-table single-fsync ingest_multi collapses N per-table commits into one. See store::data_commit_count. Read the process-global durable data-commit (fsync) count. See [REDB_DATA_COMMITS]. Exposed at the crate root as crate::redb_commit_count.