Expand description
§Dynamo LLM
The dynamo.llm crate is a Rust library that provides a set of traits and types for building
distributed LLM inference solutions.
Modules§
- backend
- Backend
- common
- disagg_
router - engines
- gguf
- http
- key_
value_ store - Interface to a traditional key-value store such as etcd. “key_value_store” spelt out because in AI land “KV” means something else.
- kv_
router - model_
card - model_
type - preprocessor
- The Preprocessor consists of the following modules
- protocols
- Dynamo LLM Protocols
- recorder
- tokenizers
- tokens
- types