Module transform 

Stateless and stateful transform traits plus built-in transforms. Transform - composable operations on samples.

Transforms are the second community extension point. Built-in transforms cover the common cases (map, filter, shuffle, batch). Community transforms can add augmentation, normalization, tokenization, etc.

Stateless vs Stateful

• [Transform]: Stateless transforms process samples independently. They run in parallel across worker threads. Examples: map, filter, flat_map.

• [StatefulTransform]: Stateful transforms maintain internal buffers and require ordered processing. They run serially in the collector thread. Examples: shuffle, batch.

Structs

BatchAccumulator

Batch transform - accumulate N samples into a group.

FilterTransform

Filter transform - keep only samples that match a predicate.

FlatMapTransform

FlatMap transform - apply a function that yields zero or more samples.

MapTransform

Map transform - apply a function to each sample.

ShuffleBuffer

Shuffle transform - randomize sample order using a reservoir buffer.

Enums

TransformResult

Result of applying a transform to a sample.

Traits

StatefulTransform

A stateful transform that can buffer, reorder, or group samples.

Transform

A stateless transform that operates on individual samples.