Skip to main content

Crate graphrt_llm

Crate graphrt_llm 

Source
Expand description

§graphrt-llm

Model providers and model-calling nodes for graphrt.

  • ChatModel: provider-neutral streaming chat interface.
  • openai (feature openai): any OpenAI-compatible endpoint.
  • anthropic (feature anthropic): the Anthropic Messages API.
  • ModelRegistry: named providers, registered as a host service.
  • Nodes: llm, question-classifier, parameter-extractor and agent (a tool-calling loop over graphrt_core::tool::ToolRegistry).
let mut registry = graphrt_core::Registry::with_builtins();
graphrt_llm::register(&mut registry);
let models = graphrt_llm::ModelRegistry::from_env()?;
let services = graphrt_core::Services::new().with(models);

Re-exports§

pub use error::LlmError;
pub use registry::ModelRegistry;
pub use types::ChatDelta;
pub use types::ChatModel;
pub use types::ChatRequest;
pub use types::ChatResponse;
pub use types::ChatStream;
pub use types::ContentPart;
pub use types::FinishReason;
pub use types::InvalidToolArguments;
pub use types::Message;
pub use types::ResponseFormat;
pub use types::Role;
pub use types::ToolCall;
pub use types::ToolChoice;
pub use types::collect;

Modules§

anthropic
Anthropic Messages API provider (streaming, tool use, extended thinking).
credential_config
Declarative provider credentials: build providers whose keys come from a CredentialSource instead of a static API key.
credentials
Dynamic credentials for model providers.
error
Model errors and their mapping to node errors.
mock
Scripted model for tests and offline examples (feature mock, on by default). Scripted model for tests and offline examples.
nodes
Model-calling nodes.
openai
OpenAI-compatible chat completions provider.
providers
Config-driven model providers.
registry
Named model providers, registered as a host service.
sse
Minimal Server-Sent Events parser.
types
Provider-neutral chat types.
typesafe
Client for TypeSafe’s System One decision API (model family “Jev”).

Functions§

register
Register the llm, question-classifier, parameter-extractor and agent node kinds.