Expand description
Core trait definitions for APR Federation
These traits define the contract for federation components. Implementations can be swapped for different backends (NATS, Redis, etcd, etc.)
Structs§
- Federation
Builder - Builder for creating federation gateways
- Gateway
Stats - Gateway statistics
- Inference
Request - Inference request metadata
- Inference
Response - Inference response
- ModelId
- Unique identifier for a model instance in the federation
- Model
Metadata - Model metadata stored in catalog
- Node
Health - Node health information
- NodeId
- Unique identifier for a node within a region
- QoSRequirements
- Quality of Service requirements
- Region
Id - Unique identifier for a region/cluster
- Route
Candidate - Route candidate with scoring details
- Route
Scores - Breakdown of routing scores
- Route
Target - Selected route target
Enums§
- Capability
- Model capabilities that can be queried
- Circuit
State - Circuit breaker state
- Federation
Error - Error types for federation operations
- Health
State - Health state enum
- Load
Balance Strategy - Load balancing strategy
- Privacy
Level - Privacy/compliance level for data routing
Traits§
- Circuit
Breaker Trait - Circuit breaker for fault tolerance
- Gateway
Middleware - Middleware that can wrap a gateway
- Gateway
Trait - Gateway - the main entry point for federation requests
- Health
Checker Trait - Health checker - monitors node health across federation
- Model
Catalog Trait - Model catalog - tracks available models across the federation
- Router
Trait - Router - selects the best node for a request
- Routing
Policy Trait - Routing policy that influences node selection
- Token
Stream - Streaming token interface
Type Aliases§
- BoxFuture
- Boxed future for async trait methods
- Federation
Result - Result type alias for federation operations