pub struct InferenceGatewayClient { /* private fields */ }
Expand description
Client for interacting with the Inference Gateway API
Implementations§
Source§impl InferenceGatewayClient
impl InferenceGatewayClient
Sourcepub fn new_default() -> Self
pub fn new_default() -> Self
Creates a new client instance with default configuration pointing to http://localhost:8080/v1
Sourcepub fn with_tools(self, tools: Option<Vec<Tool>>) -> Self
pub fn with_tools(self, tools: Option<Vec<Tool>>) -> Self
Sourcepub fn with_token(self, token: impl Into<String>) -> Self
pub fn with_token(self, token: impl Into<String>) -> Self
Trait Implementations§
Source§impl Debug for InferenceGatewayClient
Implement Debug for InferenceGatewayClient
impl Debug for InferenceGatewayClient
Implement Debug for InferenceGatewayClient
Source§impl InferenceGatewayAPI for InferenceGatewayClient
impl InferenceGatewayAPI for InferenceGatewayClient
Source§fn generate_content_stream(
&self,
provider: Provider,
model: &str,
messages: Vec<Message>,
) -> impl Stream<Item = Result<SSEvents, GatewayError>> + Send
fn generate_content_stream( &self, provider: Provider, model: &str, messages: Vec<Message>, ) -> impl Stream<Item = Result<SSEvents, GatewayError>> + Send
Stream content generation directly using the backend SSE stream.
Source§async fn list_models(&self) -> Result<ListModelsResponse, GatewayError>
async fn list_models(&self) -> Result<ListModelsResponse, GatewayError>
Lists available models from all providers Read more
Source§async fn list_models_by_provider(
&self,
provider: Provider,
) -> Result<ListModelsResponse, GatewayError>
async fn list_models_by_provider( &self, provider: Provider, ) -> Result<ListModelsResponse, GatewayError>
Lists available models by a specific provider Read more
Source§async fn generate_content(
&self,
provider: Provider,
model: &str,
messages: Vec<Message>,
) -> Result<CreateChatCompletionResponse, GatewayError>
async fn generate_content( &self, provider: Provider, model: &str, messages: Vec<Message>, ) -> Result<CreateChatCompletionResponse, GatewayError>
Generates content using a specified model Read more
Source§async fn list_tools(&self) -> Result<ListToolsResponse, GatewayError>
async fn list_tools(&self) -> Result<ListToolsResponse, GatewayError>
Lists available MCP tools Read more
Source§async fn health_check(&self) -> Result<bool, GatewayError>
async fn health_check(&self) -> Result<bool, GatewayError>
Checks if the API is available
Auto Trait Implementations§
impl Freeze for InferenceGatewayClient
impl !RefUnwindSafe for InferenceGatewayClient
impl Send for InferenceGatewayClient
impl Sync for InferenceGatewayClient
impl Unpin for InferenceGatewayClient
impl !UnwindSafe for InferenceGatewayClient
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more