Expand description
The Preprocessor consists of the following modules
translation: This module converts the allowed Ingress message types to the corresponding internal representation.apply: This module applies ModelConfig defaults to any empty optional fields specifiedprompt: This module applies any prompt template logic to the internal Request object.tokenize: This module tokenizes the formatted prompt string and returns the token ids.
The Preprocessor will accept any IngressRequest and transform it to a BackendRequest.
Re-exports§
pub use crate::protocols::common::llm_backend::BackendOutput;pub use crate::protocols::common::llm_backend::PreprocessedRequest;pub use crate::protocols::common::preprocessor::PreprocessedEmbeddingRequest;