Skip to main content

CheckpointFresh

Struct CheckpointFresh 

Source
pub struct CheckpointFresh { /* private fields */ }
Expand description

Builder state after a dataset identity enables a fresh exact checkpoint.

Exact checkpointing requires explicit replay evidence; an arbitrary dataset is not enabled merely by assigning an identity:

ⓘ
use std::convert::Infallible;
use rusttorch_data::{DataLoader, Dataset, VecCollate};
struct Rows;
impl Dataset for Rows {
    type Sample = i64;
    type Error = Infallible;
    fn len(&self) -> usize { 1 }
    fn get(&self, _: usize) -> Result<i64, Infallible> { Ok(1) }
}
let _ = DataLoader::builder(Rows)
    .collate(VecCollate)
    .dataset_identity("rows".to_owned())
    .build();

Ordinary crate::TensorDataset rows share storage and are therefore not exact-checkpoint capable until crate::TensorDataset::into_replay_safe succeeds:

ⓘ
use rusttorch_core::Tensor;
use rusttorch_data::{DataLoader, TensorDataset, VecCollate};
let dataset = TensorDataset::new(vec![Tensor::from_slice(&[1_i64])]).unwrap();
let _ = DataLoader::builder(dataset)
    .collate(VecCollate)
    .dataset_identity("tensor".to_owned())
    .build();

An opaque closure transform has no restorable state contract unless the caller explicitly selects a stateless or transactional adapter:

ⓘ
use std::convert::Infallible;
use rusttorch_data::{
    DataLoader, Dataset, FnTransform, ReplaySafeDataset, ReplaySafeMap,
    TaskContext, VecCollate,
};
struct Rows;
impl Dataset for Rows {
    type Sample = i64;
    type Error = Infallible;
    fn len(&self) -> usize { 1 }
    fn get(&self, _: usize) -> Result<i64, Infallible> { Ok(1) }
}
impl ReplaySafeDataset for Rows {}
let transform = FnTransform::new(|value, _: &TaskContext| Ok::<_, Infallible>(value));
let _ = DataLoader::builder(ReplaySafeMap::new(Rows))
    .transform(transform)
    .collate(VecCollate)
    .dataset_identity("rows".to_owned())
    .build();

Closure-backed collation is likewise opaque without an explicit Checkpointable implementation:

ⓘ
use std::convert::Infallible;
use rusttorch_data::{
    DataLoader, Dataset, FnCollate, ReplaySafeDataset, ReplaySafeMap,
};
struct Rows;
impl Dataset for Rows {
    type Sample = i64;
    type Error = Infallible;
    fn len(&self) -> usize { 1 }
    fn get(&self, _: usize) -> Result<i64, Infallible> { Ok(1) }
}
impl ReplaySafeDataset for Rows {}
let _ = DataLoader::builder(ReplaySafeMap::new(Rows))
    .collate(FnCollate::new(|values| Ok::<_, Infallible>(values)))
    .dataset_identity("rows".to_owned())
    .build();

Closure-backed sampler and explicit-batch sources do not expose stable configuration/cursor state:

ⓘ
use std::convert::Infallible;
use rusttorch_data::{
    DataLoader, Dataset, FnSampler, ReplaySafeDataset, ReplaySafeMap,
    VecCollate,
};
struct Rows;
impl Dataset for Rows {
    type Sample = i64;
    type Error = Infallible;
    fn len(&self) -> usize { 1 }
    fn get(&self, _: usize) -> Result<i64, Infallible> { Ok(1) }
}
impl ReplaySafeDataset for Rows {}
let _ = DataLoader::builder(ReplaySafeMap::new(Rows))
    .sampler(FnSampler::new(Some(1), |_| [0].into_iter()))
    .collate(VecCollate)
    .dataset_identity("rows".to_owned())
    .build();
ⓘ
use std::convert::Infallible;
use rusttorch_data::{
    DataLoader, Dataset, FnBatchSource, ReplaySafeDataset, ReplaySafeMap,
    VecCollate,
};
struct Rows;
impl Dataset for Rows {
    type Sample = i64;
    type Error = Infallible;
    fn len(&self) -> usize { 1 }
    fn get(&self, _: usize) -> Result<i64, Infallible> { Ok(1) }
}
impl ReplaySafeDataset for Rows {}
let _ = DataLoader::builder(ReplaySafeMap::new(Rows))
    .batch_sampler(FnBatchSource::new(Some(1), |_| vec![vec![0]].into_iter()))
    .collate(VecCollate)
    .dataset_identity("rows".to_owned())
    .build();

Worker replay supports explicit immutable datasets and checkpointable transforms. Zero workers use the same tagged envelope:

use std::convert::Infallible;
use rusttorch_data::{
    DataLoader, Dataset, ReplaySafeDataset, ReplaySafeMap, VecCollate,
};
struct Rows;
impl Dataset for Rows {
    type Sample = i64;
    type Error = Infallible;
    fn len(&self) -> usize { 1 }
    fn get(&self, _: usize) -> Result<i64, Infallible> { Ok(1) }
}
impl ReplaySafeDataset for Rows {}
let mut loader = DataLoader::builder(ReplaySafeMap::new(Rows))
    .workers(0)
    .collate(VecCollate)
    .dataset_identity("rows".to_owned())
    .build().unwrap();
let state = loader.iter().checkpoint().unwrap();
let mut resumed = DataLoader::builder(ReplaySafeMap::new(Rows))
    .workers(0).collate(VecCollate).dataset_identity("rows".to_owned())
    .resume_from(state).build().unwrap();
assert_eq!(resumed.iter().next().unwrap().unwrap(), vec![1]);

Trait Implementations§

Source§

impl Clone for CheckpointFresh

Source§

fn clone(&self) -> CheckpointFresh

Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Debug for CheckpointFresh

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> Read<Exclusive, BecauseExclusive> for T
where T: ?Sized,

Source§

impl<T> Same for T

Source§

type Output = T

Should always be Self
Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, !>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

Source§

fn vzip(self) -> V