pub trait CheckpointSourceFactory: WorkerSourceFactory {
const CHECKPOINT_KIND: &'static str;
}Expand description
Stable factory identity for explicitly checkpointable stream sources.
Implement this alongside CheckpointableSource on the factory’s source
type. CHECKPOINT_KIND identifies the implementation and cursor format;
.checkpointable("contents-version") separately identifies the actual data.
Change the kind when a cursor format or its meaning changes, and change the
contents identity when records change. See crate::ExactStreamDataLoader
for a complete successful implementation.
An ordinary factory cannot acquire exact replay just by setting an identity:
use rusttorch_data::*;
use std::convert::Infallible;
struct Factory;
impl WorkerSourceFactory for Factory {
type Sample = usize;
type Error = Infallible;
type Source = std::iter::Empty<Result<WorkerRecord<usize>, Infallible>>;
fn create(&self, _: WorkerContext) -> Result<Self::Source, Infallible> { Ok(std::iter::empty()) }
}
StreamDataLoaderBuilder::new(Factory).collate(VecCollate)
.checkpointable("ordinary").build().unwrap();Opaque transforms also need an explicit WorkerCheckpoint implementation or
an explicit StatelessWorker/TransactionalWorker adapter:
use rusttorch_data::*;
use std::convert::Infallible;
struct Source;
impl Iterator for Source {
type Item = Result<WorkerRecord<usize>, Infallible>;
fn next(&mut self) -> Option<Self::Item> { None }
}
impl CheckpointableSource for Source {
type Sample = usize;
type Error = Infallible;
type State = ();
fn error_sequence(&self, error: &Infallible) -> SequenceId { match *error {} }
fn snapshot(&self) {}
fn validate_snapshot(&self, _: &()) -> Result<(), Infallible> { Ok(()) }
fn restore_validated(&mut self, _: &()) {}
}
struct Factory;
impl WorkerSourceFactory for Factory {
type Sample = usize;
type Error = Infallible;
type Source = Source;
fn create(&self, _: WorkerContext) -> Result<Source, Infallible> { Ok(Source) }
}
impl CheckpointSourceFactory for Factory { const CHECKPOINT_KIND: &'static str = "empty.v1"; }
let transform = FnTransform::new(|sample: usize, _: &TaskContext| Ok::<_, Infallible>(sample));
StreamDataLoaderBuilder::new(Factory).transform(transform).collate(VecCollate)
.checkpointable("opaque").build().unwrap();Required Associated Constants§
Sourceconst CHECKPOINT_KIND: &'static str
const CHECKPOINT_KIND: &'static str
Stable source kind and state-format version, for example records.v1.
Dyn Compatibility§
This trait is not dyn compatible.
In older versions of Rust, dyn compatibility was called "object safety".