Struct BuiltInEmbedder 

pub struct BuiltInEmbedder { /* private fields */ }

A TF-IDF vectorizer that works entirely offline with no external dependencies. Call BuiltInEmbedder::fit with a corpus to build the vocabulary, then use Embedder::embed to compute vectors.

Implementations

impl BuiltInEmbedder

pub fn new() -> Self

Create an empty embedder. You must call Self::fit before embedding.

pub fn fit(&mut self, documents: &[&str])

Build the vocabulary and IDF weights from a corpus of documents.

The vocabulary is capped at 1024 terms. Terms are selected by document frequency (the most widely occurring terms across documents come first).

Trait Implementations

impl Debug for BuiltInEmbedder

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for BuiltInEmbedder

fn default() -> Self

Returns the “default value” for a type.

impl Embedder for BuiltInEmbedder

fn embed(&self, text: &str) -> PunchResult<Vec<f32>>

Compute an embedding vector for a single piece of text.

fn embed_batch(&self, texts: &[&str]) -> PunchResult<Vec<Vec<f32>>>

Compute embedding vectors for a batch of texts.

fn dimensions(&self) -> usize

The dimensionality of vectors produced by this embedder.