Struct TextDataset 

pub struct TextDataset { /* private fields */ }

A dataset of text samples with labels.

Implementations

impl TextDataset

pub fn new( texts: Vec<String>, labels: Vec<usize>, vocab: Vocab, max_length: usize, ) -> Self

Creates a new TextDataset.

pub fn from_samples<T: Tokenizer>( samples: &[(String, usize)], tokenizer: &T, min_freq: usize, max_length: usize, ) -> Self

Creates a TextDataset from raw text samples with a tokenizer.

pub fn vocab(&self) -> &Vocab

Returns the vocabulary.

pub fn num_classes(&self) -> usize

Returns the number of classes.

pub fn max_length(&self) -> usize

Returns the maximum sequence length.

Trait Implementations

impl Dataset for TextDataset

type Item = (Tensor<f32>, Tensor<f32>)

The type of items in the dataset.

fn len(&self) -> usize

Returns the number of items in the dataset.

fn get(&self, index: usize) -> Option<Self::Item>

Gets an item by index.

fn is_empty(&self) -> bool

Returns true if the dataset is empty.