pub struct OutsourceBackend { /* private fields */ }Expand description
An llmvm backend that forwards requests to known hosted providers.
Implementations§
Source§impl OutsourceBackend
impl OutsourceBackend
pub fn new(config: OutsourceConfig) -> Self
Trait Implementations§
Source§impl Backend for OutsourceBackend
impl Backend for OutsourceBackend
Source§fn generate<'life0, 'async_trait>(
&'life0 self,
request: BackendGenerationRequest,
) -> Pin<Box<dyn Future<Output = Result<BackendGenerationResponse, ProtocolError>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
fn generate<'life0, 'async_trait>(
&'life0 self,
request: BackendGenerationRequest,
) -> Pin<Box<dyn Future<Output = Result<BackendGenerationResponse, ProtocolError>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
Generate text and return the whole response.
Source§fn generate_stream<'life0, 'async_trait>(
&'life0 self,
request: BackendGenerationRequest,
) -> Pin<Box<dyn Future<Output = Result<NotificationStream<BackendGenerationResponse>, ProtocolError>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
fn generate_stream<'life0, 'async_trait>(
&'life0 self,
request: BackendGenerationRequest,
) -> Pin<Box<dyn Future<Output = Result<NotificationStream<BackendGenerationResponse>, ProtocolError>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
Request text generation and return an asynchronous stream of generated tokens.
Auto Trait Implementations§
impl Freeze for OutsourceBackend
impl RefUnwindSafe for OutsourceBackend
impl Send for OutsourceBackend
impl Sync for OutsourceBackend
impl Unpin for OutsourceBackend
impl UnwindSafe for OutsourceBackend
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more