Crate burn_dataset 

Burn Dataset

Datasets, transformations and data sources for Burn.

Applications enable the dataset feature of burn (also enabled by train) and use this crate as burn::data::dataset.

• Dataset is random access by index. InMemDataset holds items in memory, and the sqlite feature adds SqliteDataset for datasets larger than memory.
• transform: mapping, sampling, shuffling, windowing, partial views, selection and composition of datasets, applied lazily.
• source: dataset downloads, including Hugging Face datasets.
• vision, nlp, audio: ready-made datasets such as MNIST, CIFAR, image folders, AG News and Speech Commands, behind the features of the same names.

Feature flags

• sqlite: SQLite-backed datasets, using the Turso engine. sqlite-bundled is a deprecated alias.
• vision, nlp, audio: domain datasets and loaders. builtin-sources enables the downloadable vision and NLP datasets.
• dataframe: datasets backed by a Polars DataFrame.
• network: file downloads.
• fake: generated datasets for tests.
• tracing: instrument operations with the tracing crate.

Modules

source

Sources for datasets.

transform

Dataset Transformations

Structs

DatasetError

Default error type used by Dataset implementations that don’t define their own.

DatasetIterator

Dataset iterator.

HuggingfaceDatasetLoadersqlite

Load a dataset from huggingface datasets.

InMemDataset

Dataset where all items are stored in ram.

SqliteDatasetsqlite

This struct represents a dataset where all items are stored in an SQLite database. Each instance of this struct corresponds to a specific table within the SQLite database, and allows for interaction with the data stored in the table in a structured and typed manner.

SqliteDatasetStoragesqlite

The SqliteDatasetStorage struct represents a SQLite database for storing datasets. It consists of an optional name, a database file path, and a base directory for storage.

SqliteDatasetWritersqlite

This SqliteDatasetWriter struct is a SQLite database writer dedicated to storing datasets. It retains the current writer’s state and its database connection.

Enums

ImporterErrorsqlite

Error type for HuggingfaceDatasetLoader.

RowErrorsqlite

Error raised while deserializing a row into the target type.

SqliteDatasetErrorsqlite

Sqlite dataset error.

Traits

Dataset

The dataset trait defines a basic collection of items with a predefined size.

Type Aliases

Resultsqlite

Result type for the sqlite dataset.