Struct linfa_preprocessing::tf_idf_vectorization::FittedTfIdfVectorizer

pub struct FittedTfIdfVectorizer { /* fields omitted */ }

Counts the occurrences of each vocabulary entry, learned during fitting, in a sequence of texts and scales them by the inverse document document frequency defined by the method. Each vocabulary entry is mapped to an integer value that is used to index the count in the result.

Implementations

impl FittedTfIdfVectorizer

pub fn nentries(&self) -> usize

Number of vocabulary entries learned during fitting

pub fn vocabulary(&self) -> &Vec<String>

Constains all vocabulary entries, in the same order used by the transform method.

pub fn method(&self) -> &TfIdfMethod

Returns the inverse document frequency method used in the tansform method

pub fn transform<T: ToString, D: Data<Elem = T>>(
    &self,
    x: &ArrayBase<D, Ix1>
) -> CsMat<f64>

Given a sequence of n documents, produces an array of size (n, vocabulary_entries) where column j of row i is the number of occurrences of vocabulary entry j in the text of index i, scaled by the inverse document frequency. Vocabulary entry j is the string at the j-th position in the vocabulary.

pub fn transform_files<P: AsRef<Path>>(
    &self,
    input: &[P],
    encoding: EncodingRef,
    trap: DecoderTrap
) -> CsMat<f64>