Skip to main content

batches

Function batches 

Source
pub fn batches<I, T, E>(
    source: I,
    batch_size: usize,
    drop_last: bool,
) -> Result<impl Iterator<Item = Result<Vec<T>, E>>>
where I: Iterator<Item = Result<T, E>>,
Expand description

Batches an ordinary fallible sample iterator into vectors.

Samples are moved into one pre-sized vector per batch. A short final batch is omitted when drop_last is true. Source errors are yielded once and then terminate the returned iterator.

§Errors

Returns RustTorchError::InvalidConfiguration when batch_size is zero.

§Examples

An ordinary fallible iterator is enough for streaming data:

use rusttorch_data::batches;

let records = ["10", "20", "30"].into_iter().map(str::parse::<i64>);
let loader = batches(records, 2, false).expect("batch size is nonzero");
let batches = loader
    .collect::<Result<Vec<_>, _>>()
    .expect("every record is an integer");

assert_eq!(batches, vec![vec![10, 20], vec![30]]);