Skip to main content

GenerationRequest

openai_protocol::common

Trait GenerationRequest 

Source 
pub trait GenerationRequest: Send + Sync {
    // Required methods
    fn is_stream(&self) -> bool;
    fn get_model(&self) -> Option<&str>;
    fn extract_text_for_routing(&self) -> String;

    // Provided methods
    fn routing_tokens(&self) -> Option<&[i32]> { ... }
    fn rid(&self) -> Option<&str> { ... }
}
Expand description

Trait for unified access to generation request properties Implemented by ChatCompletionRequest, CompletionRequest, GenerateRequest, EmbeddingRequest, RerankRequest, and ResponsesRequest

Required Methods§

Source

fn is_stream(&self) -> bool

Check if the request is for streaming

Source

fn get_model(&self) -> Option<&str>

Get the model name if specified

Source

fn extract_text_for_routing(&self) -> String

Extract text content for routing decisions

Provided Methods§

Source

fn routing_tokens(&self) -> Option<&[i32]>

Token IDs for routing when the request is already tokenized. Some(_) routes on the token radix tree instead of the decimal-string rendering of the same IDs.

Source

fn rid(&self) -> Option<&str>

Client-provided request id, when the protocol carries one. Routing may derive a session-affinity key from it; a batch reports its first id.

Dyn Compatibility§

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§