Crate rskit_dataset 

Dataset collection framework — streaming sources, transforms, targets, and schema validation.

Re-exports

pub use collector::Collector;

pub use collector::CollectorConfig;

pub use collector::CollectorResult;

pub use collector::NullProgress;

pub use collector::ProgressCallback;

pub use manifest::CacheStatus;

pub use manifest::Manifest;

pub use manifest::SourceEntry;

pub use manifest::SourceStats;

pub use record::BoxRecordStream;

pub use record::CsvReader;

pub use record::CsvWriter;

pub use record::DatasetFormat;

pub use record::DatasetReader;

pub use record::DatasetRecord;

pub use record::DatasetWriter;

pub use record::JsonArrayReader;

pub use record::JsonArrayWriter;

pub use record::JsonLinesReader;

pub use record::JsonLinesWriter;

pub use record::filter_records;

pub use record::select_columns;

pub use schema::DatasetSchema;

pub use schema::validate_record;

pub use source::BoxDataStream;

pub use source::Source;

pub use stream::DatasetStreamExt;

pub use target::PublishResult;

pub use target::Target;

pub use transform::ResizeTransform;

pub use transform::Transform;

Modules

collector

Collector — orchestrates source → transform → target pipelines.

manifest

.manifest.json cache logic — skip re-downloading completed sources.

record

Streaming row/record dataset abstractions.

schema

Schema validation delegated to rskit-schema.

source

Source trait — pull data from any origin.

stream

Stream adapters for composing dataset items with rskit-stream.

target

Target trait — publish collected data to a destination.

transform

Transform trait and built-in transforms.

Structs

DataItem

A single data sample flowing through the dataset pipeline.

DataPayload

Payload for a single dataset item.

DatasetLimits

Runtime limits for dataset streaming and bounded materialization.

Enums

Label

Binary classification label.

MediaType

Supported media types.

Constants

DEFAULT_MAX_IN_MEMORY_BYTES

Default threshold above which payloads should be represented as files.