Struct ShardSpec 

pub struct ShardSpec {
    pub id: String,
    pub descriptor: Value,
    pub size_estimate: Option<u64>,
}

One independently-processable slice of a shardable source.

The descriptor is opaque to the coordinator — only the producing/consuming connector interprets it (e.g. {"pk_range":[0,1000]} for a key-range split, {"prefix":"dt=2026-06-11/"} for an object split). It must round-trip through JSON unchanged so the coordinator can persist it and hand it back to apply_shard on the worker that claims the shard.

Fields

id: String

Stable identifier, unique within a run’s shard set. Also used as the per-shard state-key suffix ({run}::{shard.id}), so it must be a valid state-key segment (no : — see validate_state_key).

descriptor: Value

Connector-specific, opaque shard descriptor. Persisted verbatim and passed back to apply_shard.

size_estimate: Option<u64>

Optional relative size estimate (rows / bytes / objects — the unit is the connector’s choice, only comparisons within one run’s set matter) used for skew-aware assignment. None when the source cannot cheaply estimate; the coordinator then falls back to count-based balancing.

Implementations

impl ShardSpec

pub fn whole() -> Self

The single whole-dataset shard a non-shardable source enumerates to.

Its id is "0" and its descriptor is JSON null; apply_shard’s default ignores it, so the source streams its entire dataset unchanged.

pub fn new(id: impl Into<String>, descriptor: Value) -> Self

A shard with the given id and descriptor and no size estimate.

pub fn with_size(self, size: u64) -> Self

Attach a relative size estimate (builder style).

pub fn is_whole(&self) -> bool

Whether this is the whole-dataset shard (id "0", null descriptor) — the no-op shard a non-shardable source produces.

Trait Implementations

impl Clone for ShardSpec

fn clone(&self) -> ShardSpec

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ShardSpec

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for ShardSpec

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Eq for ShardSpec

impl PartialEq for ShardSpec

fn eq(&self, other: &ShardSpec) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl Serialize for ShardSpec

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for ShardSpec