Prompt Templates for LLMs

chat-prompts is part of LlamaEdge API Server project. It provides a collection of prompt templates that are used to generate prompts for the LLMs (See models in huggingface.co/second-state).

Prompt Templates

The available prompt templates are listed below:

baichuan-2

Prompt string

以下内容为人类用户与与一位智能助手的对话。

用户:你好！
助手:

Example: second-state/Baichuan2-13B-Chat-GGUF

codellama-instruct

Prompt string

<s>[INST] <<SYS>>
Write code to solve the following coding problem that obeys the constraints and passes the example test cases. Please wrap your code answer using ```: <</SYS>>

{prompt} [/INST]

Example: second-state/CodeLlama-13B-Instruct-GGUF

codellama-super-instruct

Prompt string

<s>Source: system\n\n {system_prompt} <step> Source: user\n\n {user_message_1} <step> Source: assistant\n\n {ai_message_1} <step> Source: user\n\n {user_message_2} <step> Source: assistant\nDestination: user\n\n

Example: second-state/CodeLlama-70b-Instruct-hf-GGUF

chatml

Prompt string

<|im_start|>system
{system_message}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

Example: second-state/Yi-34B-Chat-GGUF

chatml-tool

Prompt string

<|im_start|>system\n{system_message} Here are the available tools: <tools> [{tool_1}, {tool_2}] </tools> Use the following pydantic model json schema for each tool call you will make: {"properties": {"arguments": {"title": "Arguments", "type": "object"}, "name": {"title": "Name", "type": "string"}}, "required": ["arguments", "name"], "title": "FunctionCall", "type": "object"} For each function call return a json object with function name and arguments within <tool_call></tool_call> XML tags as follows:\n<tool_call>\n{"arguments": <args-dict>, "name": <function-name>}\n</tool_call><|im_end|>
<|im_start|>user
{user_message}<|im_end|>
<|im_start|>assistant

Example

<|im_start|>system\nYou are a function calling AI model. You are provided with function signatures within <tools></tools> XML tags. You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions. Here are the available tools: <tools> [{"type":"function","function":{"name":"get_current_weather","description":"Get the current weather in a given location","parameters":{"type":"object","properties":{"location":{"type":"string","description":"The city and state, e.g. San Francisco, CA"},"format":{"type":"string","description":"The temperature unit to use. Infer this from the users location.","enum":["celsius","fahrenheit"]}},"required":["location","format"]}}},{"type":"function","function":{"name":"predict_weather","description":"Predict the weather in 24 hours","parameters":{"type":"object","properties":{"location":{"type":"string","description":"The city and state, e.g. San Francisco, CA"},"format":{"type":"string","description":"The temperature unit to use. Infer this from the users location.","enum":["celsius","fahrenheit"]}},"required":["location","format"]}}}] </tools> Use the following pydantic model json schema for each tool call you will make: {"properties": {"arguments": {"title": "Arguments", "type": "object"}, "name": {"title": "Name", "type": "string"}}, "required": ["arguments", "name"], "title": "FunctionCall", "type": "object"} For each function call return a json object with function name and arguments within <tool_call></tool_call> XML tags as follows:\n<tool_call>\n{"arguments": <args-dict>, "name": <function-name>}\n</tool_call><|im_end|>
<|im_start|>user
Hey! What is the weather like in Beijing?<|im_end|>
<|im_start|>assistant

Example: second-state/Hermes-2-Pro-Llama-3-8B-GGUF

deepseek-chat

Prompt string

User: {user_message_1}

Assistant: {assistant_message_1}<｜end▁of▁sentence｜>User: {user_message_2}

Assistant:

Example: second-state/Deepseek-LLM-7B-Chat-GGUF

deepseek-chat-2

Prompt string

<|begin_of_sentence|>{system_message}

User: {user_message_1}

Assistant: {assistant_message_1}<|end_of_sentence|>User: {user_message_2}

Assistant:

Example: second-state/DeepSeek-Coder-V2-Lite-Instruct-GGUF

deepseek-chat-25

Prompt string

<|begin_of_sentence|>{system_message}<|User|>{user_message_1}<|Assistant|>{assistant_message_1}<|end_of_sentence|><|User|>{user_message_2}<|Assistant|>

deepseek-coder

Prompt string

{system}
### Instruction:
{question_1}
### Response:
{answer_1}
<|EOT|>
### Instruction:
{question_2}
### Response:

Example: second-state/Deepseek-Coder-6.7B-Instruct-GGUF

embedding
- Prompt string This prompt template is only used for embedding models. It works as a placeholder, therefore, it has no concrete prompt string.
- Example: second-state/E5-Mistral-7B-Instruct-Embedding-GGUF

functionary-31

Prompt string

<|start_header_id|>system<|end_header_id|>

Environment: ipython

Cutting Knowledge Date: December 2023


You have access to the following functions:

Use the function 'get_current_weather' to 'Get the current weather'
{"name":"get_current_weather","description":"Get the current weather","parameters":{"type":"object","properties":{"location":{"type":"string","description":"The city and state, e.g. San Francisco, CA"}},"required":["location"]}}


Think very carefully before calling functions.
If a you choose to call a function ONLY reply in the following format:
<{start_tag}={function_name}>{parameters}{end_tag}
where

start_tag => `<function`
parameters => a JSON dict with the function argument name as key and function argument value as value.
end_tag => `</function>`

Here is an example,
<function=example_function_name>{"example_name": "example_value"}</function>

Reminder:
- If looking for real time information use relevant functions before falling back to brave_search
- Function calls MUST follow the specified format, start with <function= and end with </function>
- Required parameters MUST be specified
- Only call one function at a time
- Put the entire function call reply on one line

<|eot_id|><|start_header_id|>user<|end_header_id|>

What is the weather like in Beijing today?<|eot_id|><|start_header_id|>assistant<|end_header_id|>

Example: second-state/functionary-small-v3.1-GGUF

functionary-32

Prompt string

<|start_header_id|>system<|end_header_id|>

You are capable of executing available function(s) if required.
Only execute function(s) when absolutely necessary.
Ask for the required input to:recipient==all
Use JSON for function arguments.
Respond in this format:
>>>${recipient}
${content}
Available functions:
// Supported function definitions that should be called when necessary.
namespace functions {

    // Get the current weather
    type get_current_weather = (_: {

        // The city and state, e.g. San Francisco, CA
        location: string,

    }) => any;


} // namespace functions<|eot_id|><|start_header_id|>user<|end_header_id|>

What is the weather like in Beijing today?<|eot_id|><|start_header_id|>assistant<|end_header_id|>

Example: second-state/functionary-small-v3.2-GGUF

gemma-instruct

Prompt string

<bos><start_of_turn>user
{user_message}<end_of_turn>
<start_of_turn>model
{model_message}<end_of_turn>model

Example: second-state/gemma-2-27b-it-GGUF

glm-4-chat

Prompt string

[gMASK]<|system|>
{system_message}<|user|>
{user_message_1}<|assistant|>
{assistant_message_1}

Example: second-state/glm-4-9b-chat-GGUF

human-assistant
- Prompt string
```
Human: {input_1}\n\nAssistant:{output_1}Human: {input_2}\n\nAssistant:
```
- Example: second-state/OrionStar-Yi-34B-Chat-Llama-GGUF
intel-neural
- Prompt string
```
### System:
{system}
### User:
{usr}
### Assistant:
```
- Example: second-state/Neural-Chat-7B-v3-3-GGUF

llama-2-chat

Prompt string

<s>[INST] <<SYS>>
{system_message}
<</SYS>>

{user_message_1} [/INST] {assistant_message} </s><s>[INST] {user_message_2} [/INST]

llama-3-chat

Prompt string

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

{{ system_prompt }}<|eot_id|><|start_header_id|>user<|end_header_id|>

{{ user_message_1 }}<|eot_id|><|start_header_id|>assistant<|end_header_id|>

{{ model_answer_1 }}<|eot_id|><|start_header_id|>user<|end_header_id|>

{{ user_message_2 }}<|eot_id|><|start_header_id|>assistant<|end_header_id|>

mediatek-breeze

Prompt string

<s>{system_message}  [INST] {user_message_1} [/INST] {assistant_message_1} [INST] {user_message_2} [/INST]

Example: second-state/Breeze-7B-Instruct-v1_0-GGUF

mistral-instruct

Prompt string

<s>[INST] {user_message_1} [/INST]{assistant_message_1}</s>[INST] {user_message_2} [/INST]{assistant_message_2}</s>

Example: second-state/Mistral-7B-Instruct-v0.3-GGUF

mistrallite

Prompt string

<|prompter|>{user_message}</s><|assistant|>{assistant_message}</s>

Example: second-state/MistralLite-7B-GGUF

mistral-tool

Prompt string

[INST] {user_message_1} [/INST][TOOL_CALLS] [{tool_call_1}]</s>[TOOL_RESULTS]{tool_result_1}[/TOOL_RESULTS]{assistant_message_1}</s>[AVAILABLE_TOOLS] [{tool_1},{tool_2}][/AVAILABLE_TOOLS][INST] {user_message_2} [/INST]

Example

[INST] Hey! What is the weather like in Beijing and Tokyo? [/INST][TOOL_CALLS] [{"name":"get_current_weather","arguments":{"location": "Beijing, CN", "format": "celsius"}}]</s>[TOOL_RESULTS]Fine, with a chance of showers.[/TOOL_RESULTS]Today in Auckland, the weather is expected to be partly cloudy with a high chance of showers. Be prepared for possible rain and carry an umbrella if you're venturing outside. Have a great day!</s>[AVAILABLE_TOOLS] [{"type":"function","function":{"name":"get_current_weather","description":"Get the current weather in a given location","parameters":{"type":"object","properties":{"location":{"type":"string","description":"The city and state, e.g. San Francisco, CA"},"unit":{"type":"string","enum":["celsius","fahrenheit"]}},"required":["location"]}}},{"type":"function","function":{"name":"predict_weather","description":"Predict the weather in 24 hours","parameters":{"type":"object","properties":{"location":{"type":"string","description":"The city and state, e.g. San Francisco, CA"},"unit":{"type":"string","enum":["celsius","fahrenheit"]}},"required":["location"]}}}][/AVAILABLE_TOOLS][INST] What is the weather like in Beijing now?[/INST]

Example: second-state/Mistral-7B-Instruct-v0.3-GGUF

nemotron-chat

<extra_id_0>System
{system_message}
<extra_id_1>User
{user_message_1}<extra_id_1>Assistant
{assistant_message_1}
<extra_id_1>User
{user_message_2}<extra_id_1>Assistant
{assistant_message_2}
<extra_id_1>User
{user_message_3}
<extra_id_1>Assistant\n

Example: second-state/Nemotron-Mini-4B-Instruct-GGUF

nemotron-tool

<extra_id_0>System
{system_message}
<tool> {tool_1} </tool>
<tool> {tool_2} </tool>


<extra_id_1>User
{user_message_1}<extra_id_1>Assistant
<toolcall> {tool_call_message} </toolcall>
<extra_id_1>Tool
{tool_result_message}
<extra_id_1>Assistant\n

Example: second-state/Nemotron-Mini-4B-Instruct-GGUF

octopus
- Prompt string
```
{system_prompt}\n\nQuery: {input_text} \n\nResponse:
```
- Example: second-state/Octopus-v2-GGUF
openchat
- Prompt string
```
GPT4 User: {prompt}<|end_of_turn|>GPT4 Assistant:
```
- Example: second-state/OpenChat-3.5-0106-GGUF
phi-2-instruct
- Prompt string
```
Instruct: <prompt>\nOutput:
```
- Example: second-state/phi-2-GGUF

phi-3-chat

Prompt string

<|system|>
{system_message}<|end|>
<|user|>
{user_message_1}<|end|>
<|assistant|>
{assistant_message_1}<|end|>
<|user|>
{user_message_2}<|end|>
<|assistant|>

Example: second-state/Phi-3-medium-4k-instruct-GGUF

solar-instruct
- Prompt string
```
<s> ### User:
{user_message}

\### Assistant:
{assistant_message}</s>
```
- Example: second-state/SOLAR-10.7B-Instruct-v1.0-GGUF
stablelm-zephyr
- Prompt string
```
<|user|>
{prompt}<|endoftext|>
<|assistant|>
```
- Example: second-state/stablelm-2-zephyr-1.6b-GGUF
vicuna-1.0-chat
- Prompt string
```
{system} USER: {prompt} ASSISTANT:
```
- Example: second-state/Wizard-Vicuna-13B-Uncensored-GGUF
vicuna-1.1-chat
- Prompt string
```
USER: {prompt}
ASSISTANT:
```
- Example: second-state/ChatAllInOne-Yi-34B-200K-V1-GGUF

vicuna-llava

Prompt string

<system_prompt>\nUSER:<image_embeddings>\n<textual_prompt>\nASSISTANT:

Example: second-state/Llava-v1.6-Vicuna-7B-GGUF

wizard-coder
- Prompt string
```
{system}

### Instruction:
{instruction}

### Response:
```
- Example: second-state/WizardCoder-Python-7B-v1.0-GGUF

zephyr

Prompt string

<|system|>
{system_prompt}</s>
<|user|>
{prompt}</s>
<|assistant|>

Example: second-state/Zephyr-7B-Beta-GGUF

chat-prompts 0.17.2

Prompt Templates for LLMs

Prompt Templates