pub struct RemoteBackend { /* private fields */ }Expand description
Remote inference client. Reuses a single HTTP client for connection pooling.
Implementations§
Source§impl RemoteBackend
impl RemoteBackend
pub fn new() -> Self
Sourcepub async fn generate(
&self,
schema: &ModelSchema,
prompt: &str,
context: Option<&str>,
temperature: f64,
max_tokens: usize,
) -> Result<String, InferenceError>
pub async fn generate( &self, schema: &ModelSchema, prompt: &str, context: Option<&str>, temperature: f64, max_tokens: usize, ) -> Result<String, InferenceError>
Generate text via a remote API.
Sourcepub async fn embed(
&self,
schema: &ModelSchema,
texts: &[String],
) -> Result<Vec<Vec<f32>>, InferenceError>
pub async fn embed( &self, schema: &ModelSchema, texts: &[String], ) -> Result<Vec<Vec<f32>>, InferenceError>
Generate embeddings via a remote API (OpenAI-compatible only for now).
Trait Implementations§
Auto Trait Implementations§
impl Freeze for RemoteBackend
impl !RefUnwindSafe for RemoteBackend
impl Send for RemoteBackend
impl Sync for RemoteBackend
impl Unpin for RemoteBackend
impl UnsafeUnpin for RemoteBackend
impl !UnwindSafe for RemoteBackend
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
Source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more