Expand description
Providers for the nachalnik agent runtime: somebody else’s HTTP API, read carefully.
The runtime ships no provider and never will: it has no network, and a kernel that opened a socket would have an opinion about who you talk to. Without this crate every adopter would write the same streamed HTTP client before asking a model anything.
Everything in here is an Endpoint: an address, a key, a model identifier, a listing, and
something to say when the server is slow. What splits the crate in two is whether the model
behind it answers in turns.
A Dialect does. It implements nachalnik::Provider, so a kernel can be handed one and a
session is what comes out. There are three, each behind the feature of the same name, and
openai is on by default:
openai-POST /chat/completions,choices[].delta, tool calls assembled from fragments. What OpenRouter, ollama, vLLM, LM Studio, Together and most of the rest speak. WithOpenAiCompatible::responses, OpenAI’s Responses API instead: ordered items, and a reasoning model’s thinking sealed and sent back rather than redone every turn.gemini- Google’s owngenerateContent:candidates[].content.parts, whole calls, and orderedthoughtparts, which is the dialectnachalnik::Content::Blockswas built for.anthropic- Anthropic’s Messages API: typed blocks, whole calls whose arguments stream in fragments, and signed thinking that has to go back as it came. OpenRouter speaks it too.
An Endpoint that is not a Dialect answers something other than a turn, and there is no
kernel in its path at all:
system1- System One models, behind thesystem1feature: any of the ones OpenRouter serves, or a self-hosted engine answering the same route. Typed questions put to a state and answered with probabilities: no text, no tool calls, nothing to stream. It is for the decisions a program makes around a conversation rather than the conversation itself.
let provider = Arc::new(OpenAiCompatible::new(
"openai/gpt-5",
"https://openrouter.ai/api/v1",
std::env::var("OPENROUTER_API_KEY")?,
));
provider.probe().await;
let kernel = Kernel::new(Config::default());
kernel.set_provider(provider);note: nothing in this crate reads the environment. Where requests go, which key pays and what
limit to measure against are the caller’s to decide and say: a library that quietly picked up
OPENAI_API_KEY would spend somebody’s money on a variable exported for another reason.
OpenAiCompatible::new takes the address and the key, and
OpenAiCompatible::with_context_limit a limit to use instead of the advertised one.
note: every dialect reports fragments through a nachalnik::DeltaSink and prints nothing: the
screen belongs to whoever owns it.
Re-exports§
pub use crate::anthropic::Anthropic;pub use crate::gemini::Gemini;pub use crate::openai::OpenAiCompatible;pub use crate::system1::SystemOne;
Modules§
- anthropic
- Anthropic’s Messages API dialect: an assistant turn is an ordered list of typed blocks.
- conformance
- A suite any provider can be held to, whichever of the dialects it speaks.
- gemini
- Google’s dialect: an assistant turn is an ordered list of parts.
- openai
- A
Providerthat speaks the OpenAI chat-completions dialect over HTTP, streamed. - system1
- System One models: typed questions put to a state, answered with numbers.
Clientspeaks to any of them.
Structs§
- Attribution
- The app a request is being made on behalf of, for an endpoint that keeps a ranking of them.
- Published
- What an endpoint publishes about one parameter beyond its name; see
Dialect::published.
Traits§
- Dialect
- An
Endpointwhose model answers in turns: the wire format a conversation is carried in. - Endpoint
- The half of a model’s service its user drives, not the kernel’s.
Functions§
- install_
crypto - Installs the
rustlscryptographic provider, and does nothing if already installed. - is_
openrouter - Whether an address is OpenRouter’s. Takes a whole URL or a bare authority.
- out_
of_ quota - Whether an error means an account is out of free requests for the day, rather than having hit a momentary upstream limit.
- same_
model - Whether a listed identifier names the model being asked about, allowing for the decorations a
listing adds: Google’s
models/prefix and ollama’s implicit:latesttag. - too_
long - Reads a server’s complaint that a request was longer than the model takes, against what the caller already knows the model takes.