Skip to main content

Crate incin_data

Crate incin_data 

Source
Expand description

§Incin Data

Provides dataset and dataloader abstractions for the Incin framework. Handles batching, shuffling, and data transformation pipelines.

§Examples

Using a basic DataLoader with an in-memory Dataset:

ⓘ
use incin_data::prelude::*;
use incin_data::{Dataset, DataLoader};
use incin::prelude::*;

struct MyDataset {
    images: Tensor<s![100, 3, 224, 224], CpuBackendImpl>,
    labels: Tensor<s![100], CpuBackendImpl>,
}

impl Dataset for MyDataset {
    type Item = (Tensor<s![3, 224, 224], CpuBackendImpl>, Tensor<s![], CpuBackendImpl>);
     
    fn len(&self) -> usize {
        100
    }
     
    fn get(&self, idx: usize) -> Option<Self::Item> {
        if idx >= 100 { return None; }
        let img = self.images.slice_dyn(vec![idx..idx+1, 0..3, 0..224, 0..224]).unwrap();
        let label = self.labels.slice_dyn(vec![idx..idx+1]).unwrap();
        // Return properly shaped items...
        todo!()
    }
}

fn main() {
    let dataset = MyDataset {
        images: Tensor::zeros(()).unwrap(),
        labels: Tensor::zeros(()).unwrap(),
    };
     
    // Create a dataloader with batch size 32, shuffling enabled
    let loader = DataLoader::new(dataset).batch_size(32).shuffle(true);
     
    for batch in loader.iter() {
        let (images, labels) = batch;
        // train model...
    }
}

Re-exports§

pub use dataset::Dataset;
pub use downloader::Downloader;
pub use loader::Collate;
pub use loader::DataLoader;
pub use transforms::CenterCrop;
pub use transforms::Compose;
pub use transforms::Normalize;
pub use transforms::RandomHorizontalFlip;
pub use transforms::Scale;
pub use transforms::Transform;

Modules§

dataset
Dataset.
downloader
Downloader.
hub
Hub.
loader
Loader.
prelude
Prelude.
transforms
Transforms and data processing pipelines. Data transformation and augmentation pipeline.
vision
Vision.