Module llm_backend

dynamo_llm::protocols::common

Module llm_backend 

Source

Re-exports§

pub use super::FinishReason;
pub use super::preprocessor::PreprocessedRequest;

Structs§

BackendOutput
EmbeddingsEngineOutput
Raw output from embedding engines containing embedding vectors
LLMEngineOutput
The LLM engine and backnd with manage it’s own state, specifically translating how a given request/slot is managed on that particular backend.
TopLogprob

Type Aliases§

LogProbs
TokenType
TopLogprobs