pub struct HashEmbedder { /* private fields */ }Expand description
SimHash-based embedder that produces deterministic embeddings locally.
§Algorithm
For each word in the input text:
- Hash the word with
FnvHasher(FNV-1a, deterministic cross-platform). - For each dimension
iin 0..dimension, check bit(i % 64)of the hash XORed withi. If the bit is set, add +1.0 to that dimension; otherwise add -1.0. - After processing all words, normalize the vector to unit length (L2 norm).
The default dimension is 384.
Implementations§
Source§impl HashEmbedder
impl HashEmbedder
Sourcepub fn new(dimension: usize) -> Self
pub fn new(dimension: usize) -> Self
Create a new HashEmbedder with the specified embedding dimension.
Sourcepub fn default_dimension() -> Self
pub fn default_dimension() -> Self
Create a HashEmbedder with the default dimension of 384.
Sourcepub fn compute_embedding(&self, text: &str) -> Vec<f32>
pub fn compute_embedding(&self, text: &str) -> Vec<f32>
Compute the SimHash embedding synchronously.
This is the underlying synchronous implementation. The async
Embedder::embed trait method delegates to this.
Trait Implementations§
Source§impl Embedder for HashEmbedder
impl Embedder for HashEmbedder
Source§fn embed<'life0, 'life1, 'async_trait>(
&'life0 self,
text: &'life1 str,
) -> Pin<Box<dyn Future<Output = Result<Vec<f32>, EmbeddingError>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
fn embed<'life0, 'life1, 'async_trait>(
&'life0 self,
text: &'life1 str,
) -> Pin<Box<dyn Future<Output = Result<Vec<f32>, EmbeddingError>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
Generate a vector embedding for the given text.
Source§fn embed_batch<'life0, 'life1, 'async_trait>(
&'life0 self,
texts: &'life1 [String],
) -> Pin<Box<dyn Future<Output = Result<Vec<Vec<f32>>, EmbeddingError>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
fn embed_batch<'life0, 'life1, 'async_trait>(
&'life0 self,
texts: &'life1 [String],
) -> Pin<Box<dyn Future<Output = Result<Vec<Vec<f32>>, EmbeddingError>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
Batch embed multiple texts. Read more
Auto Trait Implementations§
impl Freeze for HashEmbedder
impl RefUnwindSafe for HashEmbedder
impl Send for HashEmbedder
impl Sync for HashEmbedder
impl Unpin for HashEmbedder
impl UnsafeUnpin for HashEmbedder
impl UnwindSafe for HashEmbedder
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
Source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more