Expand description
Prompt Formatting
Standalone, runtime-free chat-template / prompt formatting for
OpenAI-compatible inference frontends. Renders HuggingFace chat_template
jinja2 (via minijinja + minijinja-contrib pycompat), handles tool
usage formatting and generation-prompt handling.
Consumers implement OAIChatLikeRequest for their request type (or use
the ready-made impl for dynamo-protocols’ OpenAI chat request) and render
with a PromptFormatter built from a HuggingFace tokenizer_config.json
(ChatTemplate).
This crate is a bridge between OpenAI request types (dynamo_protocols)
and the HF chat-template engine (minijinja); it does not depend on
tokenizer internals. dynamo-tokenizers is re-exported for convenience.
Re-exports§
pub use dynamo_tokenizers;
Modules§
- deepseek
- Native (non-jinja) prompt formatters for DeepSeek model families.
- inkling
- Native Inkling chat renderer.
Structs§
- Chat
Template - Template for chat models including bos/eos/unk as well as the chat template.
- Chat
Template Value - Support older tool use patterns where the tool use template was separate from the default/chat template.
Modern patterns use a single template with a
tool_usekey, e.g. - Context
Mixins - NoOp
Formatter
Enums§
- Prompt
Context Mixin - Selects which context-mixin behaviors a template renders with.
- Prompt
Formatter - Prompt
Input - Text
Input - Token
Input
Traits§
- OAIChat
Like Request - Trait that defines a request that can map to an OpenAI-like request.
- OAIPrompt
Formatter
Functions§
- deepseek_
formatter_ for - If the model is a DeepSeek family whose HF repo doesn’t ship a Jinja
chat_template, return the native Rust formatter for it. ReturnsNonefor everything else (the caller then loads the HFtokenizer_config.jsontemplate viaPromptFormatter::from_parts). - may_
be_ fix_ tool_ schema - Fix a tool schema that is missing
type/properties.pubso consumers can normalize their owntoolsvalue when implementingcrate::OAIChatLikeRequest::tools. - native_
formatter_ for - Select a native formatter for model families that do not ship a usable HF
chat_template. - thinking_
bool_ from_ args - Shared helper: extract a boolean thinking toggle from
chat_template_args.