Struct ShardRouter 

pub struct ShardRouter { /* private fields */ }

Routes writes to per-shard memtables based on point timestamps.

Each shard covers a duration of shard_duration (default: 1 hour). Writes within ±ooo_shard_tolerance shards of the current active shard are accepted; writes outside this window are rejected.

Implementations

impl ShardRouter

pub fn new(config: ShardRouterConfig) -> Self

Create a new shard router with the given configuration.

pub fn insert(&self, point: &Point) -> Result<()>

Insert a point, routing it to the appropriate shard.

Errors

Returns MemtableError::ShardOutOfRange if the point’s timestamp maps to a shard outside the tolerance window.

pub fn insert_with_wal_seq(&self, point: &Point, wal_seq: u64) -> Result<()>

Insert a point with an associated WAL sequence number.

Errors

Returns MemtableError::ShardOutOfRange if the shard is outside tolerance.

pub fn admit(&self, timestamp: Timestamp) -> Result<ShardId>

Admit a timestamp: decide whether a write for it may be accepted.

This is the out-of-order tolerance check, split from the insert so a caller can run it before making the write durable. A point that fails admission never reaches the WAL, so what the caller was told was rejected cannot come back through replay.

Admitting a timestamp in a newer shard advances the active shard, exactly as an insert would.

Errors

Returns MemtableError::ShardOutOfRange if the timestamp maps to a shard outside the tolerance window.

pub fn insert_admitted(&self, point: &Point, wal_seq: u64) -> Result<()>

Insert a point that has already passed admit and been made durable.

No tolerance check: the decision was taken before the WAL append, and re-taking it here would let a concurrent writer that advanced the active shard in between turn an acknowledged, durable write into a rejected one. The WAL is the truth; what is in it is inserted.

Errors

Returns an error if the shard’s memtable insertion fails.

pub fn insert_replay(&self, point: &Point, wal_seq: u64) -> Result<()>

Insert a point during WAL replay, bypassing shard tolerance checks.

During crash recovery the WAL may contain records in a different order than the original ensure_shard_allowed evaluations (because WAL append order can differ from shard-check order under concurrent writes). Replaying with tolerance checks would reject points that were validly accepted at write time. This method skips the tolerance gate and updates the active shard monotonically so that subsequent live writes see the correct baseline.

Errors

Returns an error if the shard’s memtable insertion fails.

pub fn flush_shard(&self, shard_id: ShardId) -> Result<Vec<FlushResult>>

Flush a specific shard’s memtable to segments.

Equivalent to flush_shard_with with a registration step that does nothing.

Errors

See flush_shard_with.

pub fn flush_shard_with<F>( &self, shard_id: ShardId, register: F, ) -> Result<Vec<FlushResult>>

where F: FnMut(&[FlushResult]) -> Result<()>,

Freeze the shard’s active memtable, write it to segments, hand the results to register, and release the memtable only once registration succeeded — see FlushController::flush_frozen_with.

The router lock is released before the write starts: it used to be held for the whole segment write, and parking_lot’s writer-preferring RwLock then queued every insert behind the first write that needed a new shard.

Errors

Returns an error if the shard doesn’t exist or the flush fails.

pub fn active_shard(&self) -> Option<ShardId>

The newest shard a write has been admitted to, if any.

The out-of-order window is anchored here rather than at the wall clock: a device that was offline for days and resumes with old timestamps is still writing “now” as far as the engine is concerned.

pub fn seed_active_shard(&self, shard: ShardId)

Anchor the out-of-order window at shard if no write has anchored it yet.

open() seeds it from the newest timestamp in the catalog, so the window a database enforces after a clean restart is the one it enforced before — and so the rollup materialiser knows which buckets are final without waiting for the first live write.

pub fn retire_idle_shards(&self)

Drop the router entries of shards that hold no data and can no longer receive live writes — everything below the out-of-order window. They are recreated lazily if a backfill reaches them.

Without this a long-running database accumulated one empty controller per hour of uptime, forever.

pub fn shard_ids(&self) -> Vec<ShardId>

Every shard the router currently holds a memtable for.

pub fn live_series(&self) -> HashSet<String>

The canonical form of every series held in memory, across every shard’s active and frozen memtables.

Read by the cardinality repair that runs after retention or GC drops segments: what the database holds is the segments on disk plus this, and anything else left in the budget is a series whose data is gone.

pub fn live_tag_pairs(&self) -> BTreeSet<(String, String)>

Every (tag key, tag value) pair held in memory, across every shard.

The memtable half of label discovery — see Memtable::visit_tags.

pub fn total_memory(&self) -> usize

Returns the total memory usage across all shards.

pub fn total_interner_memory(&self) -> usize

Heap bytes held by every memtable’s string interner.

Not included in total_memory, which counts rows and index entries. The interner grows with cardinality rather than with row count, so it is worth its own number.

pub fn min_unflushed_wal_seq(&self) -> Option<u64>

The smallest WAL sequence number held by any memtable — active or frozen, in any shard — that is not yet in a registered segment. None when every memtable is empty.

pub fn scan( &self, series_key: &SeriesKey, min_ts: Timestamp, max_ts: Timestamp, ) -> Vec<Point>

Scan a specific series across all shards within a time range.

pub fn scan_measurement( &self, measurement: &str, min_ts: Timestamp, max_ts: Timestamp, ) -> Vec<Point>

Scan all points belonging to a specific measurement across all shards.

Returns points sorted by timestamp for the given measurement, filtered to [min_ts, max_ts].

pub fn shard_for_timestamp(&self, timestamp: Timestamp) -> ShardId

Map a timestamp to its shard ID.

Trait Implementations

impl Debug for ShardRouter

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.