Struct Service 

pub struct Service { /* private fields */ }

Service layer - Business logic for LLM operations

This layer sits between handlers (HTTP) and client (LLM communication). It handles:

• Model selection and validation
• Delegating to the appropriate client methods
• Business-level error handling

Implementations

impl Service

pub fn new(config: &LlmBackendSettings) -> Result<Self>

Create a new service with the specified backend configuration

pub async fn chat( &self, model: Option<&str>, messages: Vec<Message>, tools: Option<Vec<Tool>>, ) -> Result<Response>

Chat with a specific model (non-streaming)

If model is None, uses the default model from configuration.

pub async fn chat_stream_ollama( &self, model: Option<&str>, messages: Vec<Message>, format: StreamFormat, ) -> Result<UnboundedReceiverStream<String>>

Chat with streaming (Ollama format)

If model is None, uses the default model from configuration.

pub async fn chat_stream_ollama_with_tools( &self, model: Option<&str>, messages: Vec<Message>, tools: Option<Vec<Tool>>, format: StreamFormat, ) -> Result<UnboundedReceiverStream<String>>

Chat with streaming (Ollama format) with tools support

If model is None, uses the default model from configuration.

pub async fn chat_stream_openai( &self, model: Option<&str>, messages: Vec<Message>, tools: Option<Vec<Tool>>, format: StreamFormat, ) -> Result<UnboundedReceiverStream<String>>

Chat with streaming (OpenAI format)

If model is None, uses the default model from configuration.

pub async fn list_models(&self) -> Result<Vec<Model>>

List available models

pub async fn validate_model(&self, model: &str) -> Result<bool>

Validate if a model is available