List of all items

Structs

• SecretBox
• batch::ExecuteBatch
• batch::Message
• batch::SamplingParams
• cost::CostEstimate
• deployment::Budget
• deployment::Deployment
• deployment::RateLimits
• deployment::Replica
• deployment::RetryPolicy
• deployment::Serving
• deployment::Timeouts
• runner::RunHandle
• runtime::CircuitBreakerConfig
• tokens::TokenChunk
• tokens::TokenUsage
• tokens::Tokens

Enums

• batch::ContentPart
• batch::MessageContent
• batch::Role
• deployment::BudgetAction
• deployment::CapacityPolicy
• deployment::DeploymentValidationError
• error::InferenceError
• runner::SessionRebuildCause
• runner::WeightSource
• runtime::JitterKind
• runtime::ProviderKind
• runtime::RuntimeConfig
• runtime::RuntimeKind
• runtime::TransportKind
• tokens::FinishReason

Traits

• ExposeSecret
• cost::EstimateCost
• runner::ModelRunner

Functions

• cost::from_rates
• registry::infer_runtime
• runner::lift_internal

Type Aliases

• SecretString
• error::InferenceResult
• runner::CudaContextHandle