Crate rullama_inference 

rullama-inference

LLM-driven workhorses for the rullama.

This crate is the home for everything in the framework that drives an LLM call (chat / completion) or constructs prompts for one. It depends on rullama-agent for coordination primitives (locks, message bus, agent lifecycle, runtime context) — coordination is what holds agents together; inference is what they do.

Extracted from rullama-agent in Phase 11f to separate the “what holds agents together” (rullama-agent, the coordination crate) from “what makes them think” (this crate).

Modules

• chat_agent — streaming chat completion loop with per-user session management
• task_agent — autonomous task execution loop
• planner_agent — LLM-powered dynamic task planning
• judge_agent — LLM-powered cycle evaluation
• validator_agent / validation_agent — LLM-driven validation
• validation_loop — quality-check loop wrapping validation agents
• cycle_orchestrator — Plan → Work → Judge cycle driver
• plan_executor — execution of LLM-generated plans
• summarization — history compaction via LLM
• system_prompts — registry of agent prompt templates

Re-exports

pub use builder::AgentBuilder;

pub use agent_hooks::*;

pub use chat_agent::*;

pub use context::*;

pub use cycle_orchestrator::*;

pub use judge_agent::*;

pub use plan_executor::*;

pub use planner_agent::*;

pub use pool::*;

pub use runtime::*;

pub use summarization::*;

pub use system_prompts::*;

pub use task_agent::*;

pub use task_orchestrator::*;

pub use validation_agent::*;

pub use validation_loop::*;

pub use validator_agent::*;

Modules

agent_hooks

Agent Loop Hooks - Granular control over the agent execution loop

builder

Quickstart builder for ChatAgent.

chat_agent

A simple chat agent that processes messages through an LLM provider with tool support.

context

Agent Context - environment for autonomous task execution

cycle_orchestrator

Cycle Orchestrator - Plan→Work→Judge loop

judge_agent

Judge Agent - LLM-powered cycle evaluator

plan_executor

Plan Executor Agent - Executes plans by orchestrating task execution

planner_agent

Planner Agent - LLM-powered dynamic task planner

pool

Agent Pool - Manages a pool of background task agents

runtime

Agent Runtime - Generic execution loop for autonomous agents

summarization

LLM-powered conversation summarization.

system_prompts

System Prompt Registry for rullama agents.

task_agent

TaskAgent - Autonomous agent that executes a task in a loop using AI + tools

task_orchestrator

Task Orchestrator - Bridges TaskManager and AgentPool

validation_agent

Independent Validation Agent for operation verification

validation_loop

Validation Loop - Enforces quality checks before agent completion

validator_agent

ValidatorAgent - Standalone read-only agent that runs external validators