Skip to main content

Crate spacedb_durability

Crate spacedb_durability 

Source
Expand description

§spacedb-durability — SpaceDB Layer 2 (cold path)

Durability that survives any home dying: a dataset is sealed into a content-addressed, Reed-Solomon erasure-coded snapshot whose shards spread across diverse homes, so losing homes is recoverable rather than fatal.

M4-S1 (here) is the foundation — the erasure math and the verifiable manifest, over opaque (already-encrypted) snapshot bytes: encode_snapshot → (Manifest, shards), and reconstruct_snapshot from any k-of-n survivors with full integrity checking. Higher slices add the placement and shard-store seams (M4-S2) and the repair loop (M4-S3); MATA implements those over maestro-disco.

Open-core (MIT): no MATA dependency. Erasure operates on ciphertext, so a hosting home stores shards it cannot read.

Structs§

Fleet
A set of homes. Supports killing/reviving nodes to simulate failures.
Manifest
The durable description of an erasure-coded snapshot — everything needed to verify and reconstruct it from a sufficient subset of shards.
MemShardStore
In-memory content-addressed shard store. For tests and single-machine use; loses everything on drop.
Node
One home in the fleet: an id, a failure domain, a shard store, and whether it is currently reachable.
Placement
A placement record: which target holds each shard, indexed by shard index.
ReclaimReport
The outcome of a reclamation pass.
ReclaimedCopy
One surplus copy that was dropped.
RepairReport
The outcome of a repair pass.
ReplicaHealth
A snapshot’s replica-count health.
Shard
One erasure-coded shard: its position in the code and its bytes (a ciphertext fragment). Stored on a host, which cannot read it.
ShardRef
A manifest’s reference to one shard: its index and content hash. This is how a provided shard is verified before use.
TargetId
Identifier of a target node (a home).
TargetInfo
A candidate target for placement: its id and failure domain.

Enums§

DurabilityError
HealthStatus
The durability status of a snapshot.

Traits§

ShardStore
One host’s content-addressed shard storage.

Functions§

allocate
Assign num_shards shards across targets, one shard per target, spread across distinct failure domains. Requires at least num_shards targets.
distribute
Write each shard to the home it was placed on. Errors if a placed target is unknown or offline (placement should only assign online targets).
encode_snapshot
Erasure-code snapshot into data_shards data + parity_shards parity shards, returning the Manifest and the n shards. Any data_shards of them reconstruct the original.
health
Assess a snapshot’s replica-count health against the current fleet.
reachable_shard_count
How many of a snapshot’s shards are currently reachable — placed on an online home that actually holds the bytes.
reclaim
Drop surplus copies of a snapshot’s shards — copies held by online homes that the placement no longer references — but only for shards whose live placed copy is currently reachable. A no-op (empty report) when there is no safe surplus.
reconstruct_snapshot
Reconstruct the original snapshot from available shards, using manifest. Every provided shard is verified against the manifest (a tampered shard is rejected before use), and the rebuilt snapshot is hash-verified. Errors with DurabilityError::InsufficientShards if fewer than k distinct valid shards are present.
recover
Reconstruct the snapshot from the shards currently held by online homes. Tolerates up to parity missing homes; errors with DurabilityError::InsufficientShards if fewer than k shards survive.
repair
Restore a snapshot’s replica count by regenerating its missing shards onto fresh homes. A no-op (empty report) if nothing is missing. Errors with DurabilityError::InsufficientShards if fewer than k shards survive (the snapshot can’t be reconstructed to regenerate the rest), or DurabilityError::NotEnoughTargets if there aren’t enough fresh homes.
surplus_shard_count
How many surplus copies of a snapshot’s shards exist — bytes held by an online home that the placement does not reference (e.g. a repaired-away shard whose original home rejoined). These are reclaimable cost, orthogonal to the under-replication HealthStatus: a fully Healthy snapshot can still carry surplus copies worth dropping (crate::reclaim).

Type Aliases§

DurabilityResult