Struct CompressionSummarizer 

pub struct CompressionSummarizer { /* private fields */ }

Sentence compression by dropping low-importance tokens.

The importance of each token is computed via a TF-IDF-inspired heuristic using term frequency within the sentence and inverse document frequency estimated from a small built-in stop-word list.

Example

use scirs2_text::abstractive_summary::CompressionSummarizer;

let cs = CompressionSummarizer::new();
let compressed = cs.compress_sentence("The very quick brown fox jumped lazily", 0.6);
assert!(!compressed.is_empty());

Implementations

impl CompressionSummarizer

pub fn new() -> Self

Create a CompressionSummarizer with the built-in English stop-word list.

pub fn with_stop_words(stop_words: HashSet<String>) -> Self

Create a CompressionSummarizer with a custom stop-word list.

pub fn importance_score(&self, token: &str, sentence_tokens: &[String]) -> f64

Compute the importance score of a single token given its sentence context.

Score components:

• Term frequency within the sentence.
• Stop-word penalty (×0.1 if the token is a stop word).
• Length bonus: longer tokens receive a slight boost.
• Capitalisation bonus: capitalised mid-sentence tokens receive a boost (heuristic for proper nouns).

pub fn compress_sentence(&self, sentence: &str, ratio: f64) -> String

Compress sentence by retaining only the fraction ratio of tokens with the highest importance scores.

ratio is clamped to (0.0, 1.0]. A ratio of 1.0 keeps all tokens. Tokens are retained in their original order.

Returns an empty string if the sentence has no words.

Trait Implementations

impl Default for CompressionSummarizer

fn default() -> Self

Returns the “default value” for a type.