Skip to main content

llm_chat_with_usage

Function llm_chat_with_usage 

Source
pub async fn llm_chat_with_usage(
    llm: &LlmConfig,
    system: &str,
    user: &str,
) -> Result<LlmResponse, String>
Expand description

Sends a chat completion request to the LLM and returns both the content and token usage from the API response.

Retries transient failures (network errors, HTTP 429/5xx, invalid responses) with a short backoff, and returns the last underlying error instead of collapsing everything into a generic “server down” message.

§Errors

Returns the last underlying error as a human-readable string when every attempt fails (transport error, non-success HTTP status, response that is not valid JSON, or a response missing choices[0].message.content).