Expand description
§Dynamo LLM
The dynamo.llm crate is a Rust library that provides a set of traits and types for building
distributed LLM inference solutions.
Modules§
- backend
- Backend
- common
- disagg_
router - discovery
- engines
- gguf
- http
- hub
- kv_
router - local_
model - mocker
- model_
card - model_
type - preprocessor
- The Preprocessor consists of the following modules
- protocols
- Dynamo LLM Protocols
- recorder
- request_
template - tokenizers
- tokens
- Types and utilities for handling sequences of tokens, including block creation and hashing.
- types
Functions§
- file_
json_ field - Reads a JSON file, extracts a specific field, and deserializes it into type T.