Trait ModelInfo

pub trait ModelInfo:
    Send
    + Sync
    + Debug {
    // Required methods
    fn context_window(&self) -> usize;
    fn max_output_tokens(&self) -> usize;
}

Model that can be converted to a string ID for API requests

Required Methods

fn context_window(&self) -> usize

Context window size in tokens

fn max_output_tokens(&self) -> usize

Maximum number of output tokens NOTE: we may want to do something smart here to have this be context-dependent. for example if you set the right headers for anthropic, 3.7 can output 128k instead of 64k.

Implementors

impl ModelInfo for Claude

impl ModelInfo for GPT

impl ModelInfo for Gemini

impl ModelInfo for Mistral