Expand description
§Tiny Dancer: Production-Grade AI Agent Routing System
High-performance neural routing system for optimizing LLM inference costs.
This crate provides:
- FastGRNN model inference (sub-millisecond latency)
- Feature engineering for candidate scoring
- Model optimization (quantization, pruning)
- Uncertainty quantification with conformal prediction
- Circuit breaker patterns for graceful degradation
- SQLite/AgentDB integration
- Training infrastructure with knowledge distillation
Re-exports§
pub use error::Result;pub use error::TinyDancerError;pub use model::FastGRNN;pub use model::FastGRNNConfig;pub use router::Router;pub use training::generate_teacher_predictions;pub use training::Trainer;pub use training::TrainingConfig;pub use training::TrainingDataset;pub use training::TrainingMetrics;pub use types::Candidate;pub use types::RouterConfig;pub use types::RoutingDecision;pub use types::RoutingMetrics;pub use types::RoutingRequest;pub use types::RoutingResponse;
Modules§
- circuit_
breaker - Circuit breaker pattern for graceful degradation
- error
- Error types for Tiny Dancer
- feature_
engineering - Feature engineering for candidate scoring
- model
- FastGRNN model implementation
- optimization
- Model optimization techniques (quantization, pruning, knowledge distillation)
- router
- Main routing engine combining all components
- storage
- SQLite/AgentDB integration for persistent storage
- training
- FastGRNN training pipeline with knowledge distillation
- types
- Core types for Tiny Dancer routing system
- uncertainty
- Uncertainty quantification with conformal prediction
Constants§
- VERSION
- Version of the Tiny Dancer library