Expand description
InferenceProvider port — abstraction over LLM inference backends.
Structs§
- Inference
Request - Request structure for LLM inference.
- Inference
Response - Response structure from LLM inference.
- Usage
- Token usage information.
Traits§
- Inference
Provider - Port: perform LLM inference.