Crate timelake 

Timelake: the open time-series lakehouse. Tables are Apache Iceberg tables on any object store, laid out for time series (partitioned by time, sorted by series and time), kept compact by the writer itself, and queried with DataFusion.

use timelake::{Lake, Session, TableOptions};

let lake = Lake::open("s3://bucket/warehouse", Default::default())?;
let opts = TableOptions {
    time_column: Some("time".into()),
    series_key: vec!["device_id".into()],
    ..Default::default()
};
let table = lake.create_table("iot.readings", &batch.schema(), opts).await?;
let (table, result) = table.append(&[batch], Default::default()).await?;

let session = Session::new(lake);
let rows = session.sql("SELECT * FROM iot.readings WHERE device_id = 7").await?.collect().await?;
let latest = session.latest("iot.readings").await?.collect().await?;

Re-exports

pub use arrow;

pub use datafusion;

Modules

bounds

Iceberg single-value serialization of column bounds (manifest lower_bounds/upper_bounds). https://iceberg.apache.org/spec/#binary-single-value-serialization

contract

The Timelake table contract (SPEC §7.1): which column is time, which columns identify a series, and the layout/maintenance policy derived from them. Stored as timelake.* Iceberg table properties (see property), so the contract travels with the table and other engines simply ignore it.

head

The head object (SPEC §19.2, P1): <table>/metadata/timelake-head, one object holding what a reader needs to plan a query at the current version: the table metadata, the live data files with their column statistics, and the last-value index when it fits. A cold query reads it instead of following the version hint, metadata, manifest list and manifests one by one.

property

Table property keys holding the Contract.

settings

The registry of Timelake’s settings (SPEC §21.9): every switch is declared once here, with its type, default, the levels it may be set at, and the version that added it. Table properties, SQL SET, the CLI’s timelake settings and the docs all come from SETTINGS.

storage

Object storage access. One object_store client serves everything: Iceberg metadata (through an adapter that implements iceberg-rust’s Storage trait), Parquet data written by Timelake, and DataFusion scans. Any object_store backend works: S3 and S3-compatible stores (R2, RustFS, MinIO), GCS, Azure, and local disk.

summary

Snapshot summary keys: Timelake’s own, and the standard Iceberg counters Timelake maintains.

Structs

AppendOptions

Options for Table::append.

AppendResult

What Table::append did.

CommitInfo

How a commit went.

Contract

The parsed contract of an existing table.

Lake

A warehouse: a root location holding namespaces and tables. Cheap to clone; clones share the storage client and caches.

LiveEntry

A manifest entry of a data file, with inherited snapshot id and sequence numbers resolved.

MaintainOptions

Options for Table::maintain. The default does everything, rewriting each partition into sorted files of the target size.

MaintainReport

What Table::maintain did.

Session

A SQL session over a lake. Tables are addressed as namespace.table (or just table in the default namespace) and always read at their latest committed version.

SessionSettings

Timelake settings for one session; unset ones follow each table’s properties.

Table

A table at one committed version. Cheap to clone; call Table::refresh to see newer commits.

TableName

A validated table name: namespace.table, or just table in the DEFAULT_NAMESPACE. Both parts use [A-Za-z0-9_-], so they are safe as path segments.

TableOptions

Options for creating a table. Only time_column is required for a time-series table; leave it unset for a plain (dimension) table such as device metadata.

UpgradeOptions

Options for Table::upgrade.

UpgradeReport

What Table::upgrade did (or, with dry_run, would do).

Enums

Chunk

Time partition granularity: how much time one partition covers.

Error

Errors from Timelake operations.

RollupRefresh

Which rollups a refresh brings up to date.

RollupStatement

CREATE, DROP or REFRESH MATERIALIZED ROLLUP.

Statement

One SQL statement, as Session::parse splits a script.

UpgradeStep

One change an upgrade makes, in the order it makes them.

Constants

DEFAULT_NAMESPACE

Namespace of table names given without one.

Functions

align_batch

Casts and reorders a batch to the table schema by column name. Missing nullable columns are filled with nulls; extra columns are an error (add them to the schema first).

file_partition

Partition value of a data file written by Timelake (None if unpartitioned).

flush

Waits for the background writes this process started. Call it before the process exits, or the last commit’s version hint and head may not be updated: readers still find the newest version, but need extra requests to do so. The CLI and the Python package call it for you.

format_duration

Formats whole seconds of a duration in the largest exact unit (90d, 36h, 45s), the inverse of parse_duration.

latest_per_series

Newest row per series among batches, or None without a time column or rows. On equal times the later insertion wins, then the later row (later input position), matching dedup’s “latest write wins”.

parse_duration

Parses durations like 90d, 12h, 30m, 45s, 2w, 1y, or spelled out (1 day, 3 hours).

Type Aliases

Result

Result with Timelake’s Error.