Struct ShardedConCorpus 

pub struct ShardedConCorpus { /* private fields */ }

Multi-shard campaign corpus: root/shard_XXXX/ each holds an independent ConCorpus.

Implementations

impl ShardedConCorpus

pub fn open(root: impl AsRef<Path>, n_shards: u32) -> Result<Self>

Create or open a sharded root. If manifest missing, writes one with n_shards.

pub fn n_shards(&self) -> u32

pub fn root(&self) -> &Path

pub fn shard_for_traj(traj_id: TrajId, n_shards: u32) -> u32

pub fn shard_mut(&mut self, shard_id: u32) -> Result<&ConCorpus>

Open one shard env (creates dir). Safe for many processes to open different shards.

pub fn open_shard_for_traj( root: impl AsRef<Path>, traj_id: TrajId, ) -> Result<(u32, ConCorpus)>

Open only the shard for traj_id (HPC rank typically owns one shard).

pub fn open_shard(root: impl AsRef<Path>, shard_id: u32) -> Result<ConCorpus>

Open a single shard by id (rank r uses open_shard(root, r % n)).

pub fn append_trajectory_path( &mut self, traj_id: TrajId, file: impl AsRef<Path>, ) -> Result<u32>

pub fn append_trajectory_str( &mut self, traj_id: TrajId, contents: &str, source: impl Into<String>, ) -> Result<u32>

pub fn append_trajectory_frames( &mut self, traj_id: TrajId, frames: &[ConFrame], source: impl Into<String>, ) -> Result<u32>

pub fn select(&mut self, sel: &Select) -> Result<Vec<FrameKey>>

Fan-out select across all shards (read-only; opens missing shards).

pub fn get_frame_text(&mut self, key: FrameKey) -> Result<String>

pub fn reindex_all(&mut self) -> Result<u32>

impl ShardedConCorpus

pub fn join_to_single_env(&mut self, dst: impl AsRef<Path>) -> Result<u32>

Join: copy all frames from every shard into a new single-env corpus at dst (traj_id preserved; collision if same traj_id appears on two shards is an error). Secondary indexes built via normal append/prepare on each blob. Reversible with Self::split_single_to_sharded using the same n_shards and traj routing.

pub fn split_single_to_sharded( single: &ConCorpus, dst_root: impl AsRef<Path>, n_shards: u32, ) -> Result<u32>

Split: read a single-env corpus and write a new sharded root at dst_root with n_shards (rewrites manifest). Traj_id preserved; routing is traj_id % n_shards.