Expand description
Octoroute - Intelligent multi-model router for self-hosted LLMs
This library provides intelligent routing between multiple local LLM endpoints based on task complexity, importance, and resource availability.
Modulesยง
- cli
- Command-line interface for Octoroute
- config
- Configuration management for Octoroute
- error
- Error types for Octoroute
- handlers
- HTTP request handlers for Octoroute API
- metrics
- Prometheus metrics collection for Octoroute
- middleware
- Middleware modules for request processing
- models
- Model client and selection logic
- router
- Routing logic for Octoroute
- shared
- Shared utilities used across multiple handlers
- telemetry
- Telemetry and observability setup