Skip to main content

render_chat_prompt

ferrum_server::chat_template

Function render_chat_prompt 

Source 
pub fn render_chat_prompt(messages: &[ChatMessage], model_id: &str) -> String
Expand description

Render OpenAI-style chat messages into the prompt string the model was trained on.

Detects model family from the request’s model field:

  • qwen (Qwen2 / Qwen2.5 / Qwen3): ChatML with <|im_start|> / <|im_end|>
  • llama 3: <|start_header_id|>...<|end_header_id|> + <|eot_id|>
  • fallback: TinyLlama-style <|system|> / <|user|> / <|assistant|> with </s> separators

All templates end with the assistant header so the first generated token becomes the reply content (no extra role prefix).