pub struct Embedder { /* private fields */ }Expand description
Embedder backed by a fixed pool of independent sessions, for the same
reason as Reranker: fastembed::TextEmbedding::embed takes &mut self, and ingestion is embed-bound, so a shared single-session mutex
serializes every writer in the process. Each slot keeps the original
lazy-load/idle-unload lifecycle independently.
Implementations§
Source§impl Embedder
impl Embedder
pub fn new() -> Result<Self>
Sourcepub fn with_sessions(n: usize) -> Result<Self>
pub fn with_sessions(n: usize) -> Result<Self>
Builds a pool of n independent sessions, eager-loaded up front so
construction fails fast exactly like the single-session constructor
always has.
pub fn with_sessions_and_activations( n: usize, activations: Activations, ) -> Result<Self>
Sourcepub fn with_checkout_timeout(self, timeout: Duration) -> Self
pub fn with_checkout_timeout(self, timeout: Duration) -> Self
Overrides the default checkout wait bound. Mainly useful for tests that need to observe the timeout path without a long-running test.
pub fn pool_size(&self) -> usize
pub fn metrics(&self) -> EmbedMetricsSnapshot
pub fn embed(&self, text: &str) -> Result<Vec<f32>>
pub fn embed_query(&self, text: &str) -> Result<Vec<f32>>
Sourcepub fn embed_query_within(&self, text: &str, wait: Duration) -> Result<Vec<f32>>
pub fn embed_query_within(&self, text: &str, wait: Duration) -> Result<Vec<f32>>
embed_query for a caller with a deadline: it takes only a loaded
session, waiting at most wait, and fails with EmbedTimeoutError
rather than load the model; a session dropped while idle is reloaded
in the background for later calls.
pub fn embed_batch(&self, texts: &[&str]) -> Result<Vec<Vec<f32>>>
pub fn drop_if_idle(&self, idle_secs: u64)
Sourcepub fn resident_bytes(&self) -> u64
pub fn resident_bytes(&self) -> u64
Allocator growth across every slot’s load and first use; a dropped slot adds 0.
Sourcepub fn slot_resident_bytes(&self) -> Vec<u64>
pub fn slot_resident_bytes(&self) -> Vec<u64>
Allocator growth across each slot’s load and first use, 0 for a dropped slot.
pub fn dimension(&self) -> usize
Trait Implementations§
Auto Trait Implementations§
impl Freeze for Embedder
impl RefUnwindSafe for Embedder
impl Send for Embedder
impl Sync for Embedder
impl Unpin for Embedder
impl UnsafeUnpin for Embedder
impl UnwindSafe for Embedder
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> Downcast for Twhere
T: Any,
impl<T> Downcast for Twhere
T: Any,
Source§fn into_any(self: Box<T>) -> Box<dyn Any>
fn into_any(self: Box<T>) -> Box<dyn Any>
Box<dyn Trait> (where Trait: Downcast) to Box<dyn Any>, which can then be
downcast into Box<dyn ConcreteType> where ConcreteType implements Trait.Source§fn into_any_rc(self: Rc<T>) -> Rc<dyn Any>
fn into_any_rc(self: Rc<T>) -> Rc<dyn Any>
Rc<Trait> (where Trait: Downcast) to Rc<Any>, which can then be further
downcast into Rc<ConcreteType> where ConcreteType implements Trait.Source§fn as_any(&self) -> &(dyn Any + 'static)
fn as_any(&self) -> &(dyn Any + 'static)
&Trait (where Trait: Downcast) to &Any. This is needed since Rust cannot
generate &Any’s vtable from &Trait’s.Source§fn as_any_mut(&mut self) -> &mut (dyn Any + 'static)
fn as_any_mut(&mut self) -> &mut (dyn Any + 'static)
&mut Trait (where Trait: Downcast) to &Any. This is needed since Rust cannot
generate &mut Any’s vtable from &mut Trait’s.Source§impl<T> DowncastSend for T
impl<T> DowncastSend for T
Source§impl<T> DowncastSync for T
impl<T> DowncastSync for T
impl<T> Fruit for T
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§fn in_current_span(self) -> Instrumented<Self> ⓘ
fn in_current_span(self) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more