Crate inference_gateway_sdk 

Inference Gateway SDK for Rust

This crate provides a Rust client for the Inference Gateway API, allowing interaction with various LLM providers through a unified interface.

Data types in [crate::generated::schemas] are generated from the upstream openapi.yaml and re-exported at the crate root. Run task generate-types to regenerate them after a spec bump.

Modules

defaults

Generation of default values for serde.

error

Error types.

Structs

ChatCompletionChoice

ChatCompletionChoice

ChatCompletionChoiceLogprobs

Log probability information for the choice.

ChatCompletionMessageToolCall

ChatCompletionMessageToolCall

ChatCompletionMessageToolCallChunk

ChatCompletionMessageToolCallChunk

ChatCompletionMessageToolCallFunction

The function that the model called.

ChatCompletionStreamChoice

ChatCompletionStreamChoice

ChatCompletionStreamChoiceLogprobs

Log probability information for the choice.

ChatCompletionStreamOptions

Options for streaming response. Only set this when you set stream: true.

ChatCompletionStreamResponseDelta

A chat completion delta generated by streamed model responses.

ChatCompletionTokenLogprob

ChatCompletionTokenLogprob

ChatCompletionTokenLogprobTopLogprobsItem

ChatCompletionTokenLogprobTopLogprobsItem

ChatCompletionTool

ChatCompletionTool

CompletionUsage

Usage statistics for the completion request.

Config

Config

CreateChatCompletionRequest

CreateChatCompletionRequest

CreateChatCompletionResponse

Represents a chat completion response returned by model, based on the provided input.

CreateChatCompletionStreamResponse

Represents a streamed chunk of a chat completion response returned by the model, based on the provided input.

Endpoints

Endpoints

Error

Error

FunctionObject

FunctionObject

FunctionParameters

The parameters the functions accepts, described as a JSON Schema object. See the guide for examples, and the JSON Schema reference for documentation about the format. Omitting parameters defines a function with an empty parameter list.

ImageContentPart

Image content part

ImageUrl

Image URL configuration

InferenceGatewayClient

Client for interacting with the Inference Gateway API

ListModelsResponse

Response structure for listing models

ListToolsResponse

Response structure for listing MCP tools

McpTool

An MCP tool definition

Message

Message structure for provider requests

Model

Common model information

ProviderSpecificResponse

Provider-specific response format. Examples:

SSEvents

Stream of Server-Sent Events (SSE) yielded by InferenceGatewayAPI::generate_content_stream.

SsEvent

SsEvent

TextContentPart

Text content part

ToolCallExtraContent

Provider-specific opaque data attached to a tool call. The contents are not interpreted by the gateway, but must be echoed back verbatim on the next request that references this tool call. Currently used by Google Gemini extended-thinking models to carry the per-call thought_signature. Other providers may ignore the field.

ToolCallExtraContentGoogle

Google Gemini-specific extra content.

Enums

ChatCompletionToolType

The type of the tool. Currently, only function is supported.

ContentPart

A content part within a multimodal message

FinishReason

The reason the model stopped generating tokens. This will be stop if the model hit a natural stop point or a provided stop sequence, length if the maximum number of tokens specified in the request was reached, content_filter if content was omitted due to a flag from our content filters, tool_calls if the model called a tool.

GatewayError

Custom error types for the Inference Gateway SDK

ImageContentPartType

Content type identifier

ImageUrlDetail

Image detail level for vision processing

MessageContent

Message content - either text or multimodal content parts

MessageRole

Role of the message sender

Provider

Provider

ProviderAuthType

Authentication type for providers

SsEventEvent

SsEventEvent

TextContentPartType

Content type identifier

Traits

InferenceGatewayAPI

Core API interface for the Inference Gateway