Skip to main content

Module preprocess

Module preprocess 

Source
Expand description

Text preprocessing utilities

This module provides functionality for text normalization, cleaning, and other preprocessing operations.

Structs§

BasicNormalizer
Basic text normalizer that handles case folding and unicode normalization
BasicTextCleaner
Text cleaner for removing special characters, extra whitespace, and stopwords
TextPreprocessor
Pipeline for text preprocessing that combines normalization and cleaning

Traits§

TextCleaner
Trait for text cleaning operations
TextNormalizer
Trait for text normalization operations