Expand description
Language model inference abstractions and implementations.
This module provides the core abstraction for language model inference, including the base trait that all providers must implement.
Structs§
- Inference
Config - Inference configuration parameters
- Inference
Output Error - Error type for inference operations that don’t produce any outputs
- Scored
Output - A scored output from a language model
Traits§
- Base
Language Model - Abstract base trait for language model inference