Crate mini_prompt

Lightweight abstractions for using LLMs via a providers API.

Simple calls:

let mut backend = callers::Openrouter::<models::Gemma27B3>::default();
let resp =
    backend.simple_call("How much wood could a wood-chuck chop").await;

If you are looking for more control over the input, you can use call instead of simple_call.

With tools:

let backend = callers::Anthropic::<models::ClaudeHaiku35>::default();
let mut session = ToolsSession::new(
            backend,
            vec![
                (
                    ToolInfo::new("flubb", "Performs the flubb action.", None).into(),
                    Box::new(move |_args| {
                        r#"{"status": "success", "message": "flubb completed successfully"}"#
                            .to_string()
                    }),
                ),
            ],
        );

let resp =
    session.simple_call("Go ahead and flubb for me").await;

Structured output:

let mut backend = callers::Openrouter::<models::Gemma27B3>::default();
let resp =
    backend.simple_call("Whats 2+2? output the final answer as JSON within triple backticks (A markdown code block with json as the language).").await;

let json = markdown_codeblock(&resp.unwrap(), &MarkdownOptions::json()).unwrap();
let p: serde_json::Value = serde_json_lenient::from_str(&json).expect("json decode");

Re-exports

pub use callers::ModelCaller;

pub use tools::ToolsSession;

Modules

callers

Types that drive a model providers API.

data_model

Wire-format types to use when driving LLM APIs.

models

Types representing the different LLMs which can be used.

parse

Helpers and utilities for extracting structured data from LLM output.

tools

Tool-calling scaffolding.

Structs

CallBase

The basic parameters for a (possibly multi-turn) model call.

CallResp

The response from the model for generating a single turn.

ToolInfo

Describes the parameters and use of a tool made available to an LLM.

Turn

Describes a round of model input or output.

Enums

CallErr

Describes an error which occurred during a model call.

FinishReason

Describes the reason a model stopped providing tokens.

Message

A unit of data written or read from the model.

Role

The context of data in or out of the model.