Skip to main content

CheckpointSourceFactory

Trait CheckpointSourceFactory 

Source
pub trait CheckpointSourceFactory: WorkerSourceFactory {
    const CHECKPOINT_KIND: &'static str;
}
Expand description

Stable factory identity for explicitly checkpointable stream sources.

Implement this alongside CheckpointableSource on the factory’s source type. CHECKPOINT_KIND identifies the implementation and cursor format; .checkpointable("contents-version") separately identifies the actual data. Change the kind when a cursor format or its meaning changes, and change the contents identity when records change. See crate::ExactStreamDataLoader for a complete successful implementation.

An ordinary factory cannot acquire exact replay just by setting an identity:

ⓘ
use rusttorch_data::*;
use std::convert::Infallible;
struct Factory;
impl WorkerSourceFactory for Factory {
    type Sample = usize;
    type Error = Infallible;
    type Source = std::iter::Empty<Result<WorkerRecord<usize>, Infallible>>;
    fn create(&self, _: WorkerContext) -> Result<Self::Source, Infallible> { Ok(std::iter::empty()) }
}
StreamDataLoaderBuilder::new(Factory).collate(VecCollate)
    .checkpointable("ordinary").build().unwrap();

Opaque transforms also need an explicit WorkerCheckpoint implementation or an explicit StatelessWorker/TransactionalWorker adapter:

ⓘ
use rusttorch_data::*;
use std::convert::Infallible;
struct Source;
impl Iterator for Source {
    type Item = Result<WorkerRecord<usize>, Infallible>;
    fn next(&mut self) -> Option<Self::Item> { None }
}
impl CheckpointableSource for Source {
    type Sample = usize;
    type Error = Infallible;
    type State = ();
    fn error_sequence(&self, error: &Infallible) -> SequenceId { match *error {} }
    fn snapshot(&self) {}
    fn validate_snapshot(&self, _: &()) -> Result<(), Infallible> { Ok(()) }
    fn restore_validated(&mut self, _: &()) {}
}
struct Factory;
impl WorkerSourceFactory for Factory {
    type Sample = usize;
    type Error = Infallible;
    type Source = Source;
    fn create(&self, _: WorkerContext) -> Result<Source, Infallible> { Ok(Source) }
}
impl CheckpointSourceFactory for Factory { const CHECKPOINT_KIND: &'static str = "empty.v1"; }
let transform = FnTransform::new(|sample: usize, _: &TaskContext| Ok::<_, Infallible>(sample));
StreamDataLoaderBuilder::new(Factory).transform(transform).collate(VecCollate)
    .checkpointable("opaque").build().unwrap();

Required Associated Constants§

Source

const CHECKPOINT_KIND: &'static str

Stable source kind and state-format version, for example records.v1.

Dyn Compatibility§

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§