Struct TopicModelExtractor 

pub struct TopicModelExtractor { /* private fields */ }

Production-quality collapsed Gibbs sampling LDA topic extractor.

Example

use ipfrs_semantic::topic_model_extractor::{TopicModelExtractor, ExtractorConfig};

let config = ExtractorConfig { num_topics: 3, num_iterations: 100, ..Default::default() };
let mut extractor = TopicModelExtractor::new(config);

let docs = vec![
    ("d1", "rust programming language systems"),
    ("d2", "python data science machine learning"),
    ("d3", "rust memory safety ownership"),
    ("d4", "python neural network deep learning"),
    ("d5", "rust async await future tokio"),
    ("d6", "machine learning gradient descent optimisation"),
];
extractor.fit(&docs).unwrap();
let topics = extractor.topics().unwrap();
assert_eq!(topics.len(), 3);

Implementations

impl TopicModelExtractor

pub fn new(config: ExtractorConfig) -> Self

Create a new extractor with the given configuration.

pub fn fit(&mut self, docs: &[(&str, &str)]) -> Result<(), ExtractorError>

Fit the model on a slice of (doc_id, text) pairs.

pub fn topics(&self) -> Result<Vec<ExtractorTopic>, ExtractorError>

Return all topics sorted by prevalence descending.

pub fn document_topics( &self, doc_id: &str, ) -> Result<ExtractorDocumentTopics, ExtractorError>

Return the topic distribution for a document that was in the training corpus.

pub fn infer_topics( &self, text: &str, ) -> Result<ExtractorDocumentTopics, ExtractorError>

Infer topics for a new (unseen) document using 5 Gibbs passes.

pub fn top_words( &self, topic_id: usize, n: usize, ) -> Result<Vec<ExtractorTopicWord>, ExtractorError>

Return the top-n words for a given topic.

pub fn similar_topics( &self, topic_id: usize, top_k: usize, ) -> Result<Vec<(usize, f64)>, ExtractorError>

Return topics most similar to topic_id ranked by cosine similarity.

pub fn assign_label( &mut self, topic_id: usize, label: String, ) -> Result<(), ExtractorError>

Assign a human-readable label to a topic.

pub fn stats(&self) -> Result<ModelStats, ExtractorError>

Return aggregate model statistics including perplexity.

Trait Implementations

impl Debug for TopicModelExtractor

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.