Skip to main content

invoke

infinity_provider_models_dev::openai

Function invoke 

Source 
pub async fn invoke(
    backend: &Backend<'_>,
    upstream_model: &str,
    default_max_tokens: Option<u64>,
    request: CompletionRequest,
) -> Result<ModelStream, CompletionError>
Expand description

Stream one completion from an OpenAI-compatible endpoint.

§Errors

Returns CompletionError::provider when the endpoint rejects the request (with the provider’s retry classification) and CompletionError::ResponseError when a live stream breaks mid-flight. Individual stream items may also yield errors.