pub struct EmbeddingsSession { /* private fields */ }Expand description
Implementations§
Source§impl EmbeddingsSession
impl EmbeddingsSession
Sourcepub async fn new(model: &Model) -> Result<EmbeddingsSession, FoundryLocalError>
pub async fn new(model: &Model) -> Result<EmbeddingsSession, FoundryLocalError>
Open an embeddings session on a loaded model.
Returns FoundryLocalError::Validation if the model’s task is not
"embeddings".
Sourcepub async fn embed(
&self,
input: impl Into<String>,
) -> Result<Vec<f32>, FoundryLocalError>
pub async fn embed( &self, input: impl Into<String>, ) -> Result<Vec<f32>, FoundryLocalError>
Embed a single text input, returning its dense vector.
Sourcepub async fn embed_batch(
&self,
inputs: Vec<String>,
) -> Result<Vec<Vec<f32>>, FoundryLocalError>
pub async fn embed_batch( &self, inputs: Vec<String>, ) -> Result<Vec<Vec<f32>>, FoundryLocalError>
Embed a batch of text inputs, returning one dense vector per input (in order).
Sourcepub fn into_session(self) -> Session
pub fn into_session(self) -> Session
Consume this handle, yielding the underlying base Session.
Methods from Deref<Target = Session>§
Sourcepub async fn process_request(
&self,
request: Request,
) -> Result<Response, FoundryLocalError>
pub async fn process_request( &self, request: Request, ) -> Result<Response, FoundryLocalError>
Process a request and return the complete response.
Runs on a blocking worker thread; the returned future resolves when generation finishes.
§Cancellation
The returned future is cancel-on-drop: if it is dropped before it
resolves — for example via tokio::time::timeout, tokio::select!, or
aborting the task — the in-flight native request is cancelled and
generation stops as soon as possible rather than running to completion on
the detached worker. This mirrors the drop-cancels-generation behaviour of
process_streaming_request.
Sourcepub fn process_streaming_request(&self, request: Request) -> ItemStream
pub fn process_streaming_request(&self, request: Request) -> ItemStream
Process a request, streaming each output Item as it is
produced.
The returned ItemStream yields items until generation completes;
dropping it cancels generation.
Sourcepub async fn set_options(
&self,
options: RequestOptions,
) -> Result<(), FoundryLocalError>
pub async fn set_options( &self, options: RequestOptions, ) -> Result<(), FoundryLocalError>
Apply session-scoped options that persist across subsequent requests.
Sourcepub fn create_input_queue(&self) -> Result<ItemQueue, FoundryLocalError>
pub fn create_input_queue(&self) -> Result<ItemQueue, FoundryLocalError>
Create an ItemQueue for streaming incremental input into a request.
Attach the returned queue to a Request via
Request::with_input_queue, then push items as they become available.
Trait Implementations§
Source§impl Clone for EmbeddingsSession
impl Clone for EmbeddingsSession
Source§fn clone(&self) -> EmbeddingsSession
fn clone(&self) -> EmbeddingsSession
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read more