Struct TextSummarizer 

pub struct TextSummarizer {
    pub config: SummarizerConfig,
    pub document_frequencies: HashMap<String, u32>,
    pub total_documents: u32,
    /* private fields */
}

Extractive text summarizer combining TF-IDF and TextRank.

use ipfrs_semantic::text_summarizer::{TextSummarizer, SummarizerConfig, SummarizationMethod};

let config = SummarizerConfig {
    method: SummarizationMethod::TfIdf { top_n: 2 },
    ..SummarizerConfig::default()
};
let mut summarizer = TextSummarizer::new(config);
let result = summarizer.summarize(
    "The sky is blue. The ocean is also blue. Grass is green. Mountains are tall."
).unwrap();
assert_eq!(result.summary_sentences.len(), 2);

Fields

config: SummarizerConfig

Active configuration.

document_frequencies: HashMap<String, u32>

Corpus-level document frequencies: term → count of documents containing that term.

total_documents: u32

Total number of documents added to the corpus.

Implementations

impl TextSummarizer

pub fn new(config: SummarizerConfig) -> Self

Create a new summarizer with the given configuration.

pub fn summarize( &mut self, text: &str, ) -> Result<TextSummaryResult, SummarizerError>

Summarize text using the configured method.

Errors

Returns SummarizerError::EmptyText when text is blank, or SummarizerError::InsufficientSentences when fewer sentences are found than the algorithm requires.

pub fn add_to_corpus(&mut self, text: &str)

Add text to the external IDF corpus.

Each sentence in text is treated as a document for IDF purposes. Calling this before summarize improves IDF estimates, especially for domain-specific vocabulary.

pub fn stats(&self) -> TextSummarizerStats

Return usage statistics.

pub fn split_sentences(&self, text: &str) -> Vec<String>

Split text into sentences on ., !, or ? followed by whitespace or end of string.

pub fn tokenize_sentence(&self, sentence: &str) -> Vec<String>

Tokenize a sentence: lowercase, keep alphanumeric chars, drop stop words.

pub fn tfidf_vector( &self, tokens: &[String], all_sentences_tokens: &[Vec<String>], ) -> HashMap<String, f64>

Compute the TF-IDF vector for one sentence relative to the given sentence corpus.

IDF is smoothed: ln((1 + N) / (1 + df)) + 1 where N is the number of sentences used as corpus. If the summarizer has an external corpus its frequencies are blended in (additive).

pub fn cosine_similarity( a: &HashMap<String, f64>, b: &HashMap<String, f64>, ) -> f64

Cosine similarity between two TF-IDF vectors.

Returns 0.0 when either vector is empty or has zero magnitude.

pub fn textrank_scores( similarity_matrix: &[Vec<f64>], damping: f64, max_iter: u32, ) -> Vec<f64>

Run PageRank-style iteration on the sentence similarity matrix.

Initialises scores uniformly and iterates until the maximum per-node delta is less than 1e-6 or max_iter is reached.

Trait Implementations

impl Clone for TextSummarizer

fn clone(&self) -> TextSummarizer

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TextSummarizer

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.