Struct OpenAiRunnerRegistry 

pub struct OpenAiRunnerRegistry { /* private fields */ }

Route-local registry of model runners exposed through OpenAI model ids.

Implementations

impl OpenAiRunnerRegistry

pub fn new() -> Self

Returns an empty runner registry.

pub fn with_runner( self, model: impl Into<String>, runner: Arc<dyn ModelRunner>, ) -> Self

Returns the registry with runner registered under OpenAI model id model.

pub fn register( &mut self, model: impl Into<String>, runner: Arc<dyn ModelRunner>, )

Registers runner under OpenAI model id model.

pub fn cards(&self) -> Vec<ModelCard>

Returns the model card for each registered runner.

When a runner’s own card model differs from the registered id, the card is relabeled with the registered id and the original is preserved under a runner-model extra field.

pub fn infer( &self, cx: &mut Cx, model: &str, request: ModelRequest, ) -> Result<ModelResponse>

Dispatches request to the runner registered for model.

Returns a model_not_found error when no runner matches model.

pub fn is_empty(&self) -> bool

Returns true when no runners are registered.

Trait Implementations

impl Clone for OpenAiRunnerRegistry

fn clone(&self) -> OpenAiRunnerRegistry

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for OpenAiRunnerRegistry

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for OpenAiRunnerRegistry

fn default() -> OpenAiRunnerRegistry

Returns the “default value” for a type.