pub fn batches<I, T, E>(
source: I,
batch_size: usize,
drop_last: bool,
) -> Result<impl Iterator<Item = Result<Vec<T>, E>>>Expand description
Batches an ordinary fallible sample iterator into vectors.
Samples are moved into one pre-sized vector per batch. A short final batch
is omitted when drop_last is true. Source errors are yielded once and
then terminate the returned iterator.
§Errors
Returns RustTorchError::InvalidConfiguration when batch_size is zero.
§Examples
An ordinary fallible iterator is enough for streaming data:
use rusttorch_data::batches;
let records = ["10", "20", "30"].into_iter().map(str::parse::<i64>);
let loader = batches(records, 2, false).expect("batch size is nonzero");
let batches = loader
.collect::<Result<Vec<_>, _>>()
.expect("every record is an integer");
assert_eq!(batches, vec![vec![10, 20], vec![30]]);