Struct TfidfTransformer 

pub struct TfidfTransformer<F> {
    pub norm: TfidfNorm,
    pub use_idf: bool,
    pub smooth_idf: bool,
    pub sublinear_tf: bool,
    /* private fields */
}

An unfitted TF-IDF transformer.

Fits IDF weights from a term-count matrix and transforms new count matrices into TF-IDF weighted matrices.

Examples

use ferrolearn_preprocess::tfidf::{TfidfTransformer, TfidfNorm};
use ndarray::array;

let counts = array![
    [3.0_f64, 0.0, 1.0],
    [2.0, 0.0, 0.0],
    [3.0, 0.0, 0.0],
    [4.0, 0.0, 0.0],
    [3.0, 2.0, 0.0],
    [3.0, 0.0, 2.0],
];
let tfidf = TfidfTransformer::<f64>::new();
let fitted = tfidf.fit(&counts).unwrap();
let result = fitted.transform(&counts).unwrap();
assert_eq!(result.shape(), counts.shape());

Fields

norm: TfidfNorm

Row normalization mode.

use_idf: bool

Whether to use IDF weighting.

smooth_idf: bool

Whether to smooth IDF: ln((1+n)/(1+df)) + 1.

sublinear_tf: bool

Whether to apply sublinear TF scaling: 1 + ln(tf).

Implementations

impl<F: Float + Send + Sync + 'static> TfidfTransformer<F>

pub fn new() -> Self

Create a new TfidfTransformer with default settings.

pub fn norm(self, norm: TfidfNorm) -> Self

Set the row normalization mode.

pub fn use_idf(self, use_idf: bool) -> Self

Set whether to use IDF weighting.

pub fn smooth_idf(self, smooth: bool) -> Self

Set whether to smooth IDF.

pub fn sublinear_tf(self, sublinear: bool) -> Self

Set whether to apply sublinear TF scaling.

pub fn fit( &self, counts: &Array2<F>, ) -> Result<FittedTfidfTransformer<F>, FerroError>

Fit the transformer by computing IDF from a term-count matrix.

Errors

Returns FerroError::InsufficientSamples if the matrix has zero rows.

Trait Implementations

impl<F: Clone> Clone for TfidfTransformer<F>

fn clone(&self) -> TfidfTransformer<F>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<F: Debug> Debug for TfidfTransformer<F>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<F: Float + Send + Sync + 'static> Default for TfidfTransformer<F>

fn default() -> Self

Returns the “default value” for a type.