Struct ObjectStoreShardSource 

pub struct ObjectStoreShardSource { /* private fields */ }

A CorpusRowSource streaming v1 shards out of an ObjectStore with a bounded prefetch window. See the module docs for the determinism and residency contracts.

Implementations

impl ObjectStoreShardSource

pub fn open(store: Arc<dyn ObjectStore>) -> Result<Self, ShardError>

Open a source over every shard the store lists. Headers are probed with 32-byte range reads so total_rows / width are known up front without fetching any payload.

Trait Implementations

impl CorpusRowSource for ObjectStoreShardSource

fn total_rows(&self) -> u64

Total rows across every shard in this source.

fn width(&self) -> usize

Activation width p (columns per row).

fn batch_rows(&self) -> usize

Rows handed back per next_batch (may be smaller for the final batch).

fn reset(&mut self)

Rewind to the first row so the next next_batch replays from the start.

fn next_batch(&mut self) -> Result<Option<RowBatch>, ShardError>

Yield the next deterministic batch, or Ok(None) at end of corpus.