Struct Agent 

pub struct Agent { /* private fields */ }

The agent: manages conversation state, context pipeline, and the prompt → LLM → tool → LLM loop.

use orion_core::{Agent, AgentConfig, ContextConfig, InferenceParams};

let mut agent = Agent::new(AgentConfig {
    system_prompt: "You are a coding assistant.".into(),
    inference_params: InferenceParams {
        max_tokens: 4096,
        temperature: 0.4,
        context_size: 8192,
        n_threads: 6,
    },
    context_config: ContextConfig {
        max_context_tokens: 8192,
        max_response_tokens: 4096,
        ..Default::default()
    },
    ..Default::default()
});

// Change settings on the fly.
agent.set_system_prompt("You are a pirate.");
agent.set_inference_params(InferenceParams { temperature: 1.2, ..Default::default() });
agent.clear();

Implementations

impl Agent

pub fn new(config: AgentConfig) -> Self

Create an agent with the given config and the default ChatML template.

pub fn with_template( config: AgentConfig, template: Arc<dyn ChatTemplate>, ) -> Self

Create an agent with an explicit chat template.

pub fn messages(&self) -> &[Message]

The current conversation messages, in order.

pub fn config(&self) -> &AgentConfig

The agent’s current configuration.

pub fn template(&self) -> &dyn ChatTemplate

The chat template currently in use.

pub fn set_system_prompt(&mut self, prompt: impl Into<String>)

Replace the system prompt used for subsequent prompts.

pub fn set_inference_params(&mut self, params: InferenceParams)

Replace the inference parameters used for subsequent generations.

pub fn set_context_config(&mut self, config: ContextConfig)

Replace the context-management configuration.

pub fn set_prune_strategy(&mut self, strategy: PruneStrategy)

Select the strategy used when the conversation overflows the budget.

pub fn set_pinned(&mut self, message_id: &str, pinned: bool) -> bool

Pin or unpin a message by id. Pinned messages always survive context pruning. Returns whether a message with that id was found.

pub fn set_template(&mut self, template: Arc<dyn ChatTemplate>)

Swap the chat template at runtime (e.g. after detecting the model family).

pub fn set_tools(&mut self, tools: Vec<Box<dyn Tool>>)

Register the tools the agent may invoke during a prompt.

Available only with the tools feature (enabled by default).

pub fn set_approval_hook(&mut self, hook: Arc<dyn ApprovalHook>)

Install a hook consulted before every tool call executes.

See ApprovalHook for the exact semantics. With no hook installed the tool loop behaves exactly as it did before - every call runs immediately.

Available only with the tools feature (enabled by default).

pub fn clear(&mut self)

Clear the conversation history.

pub fn replace_messages(&mut self, messages: Vec<Message>)

Replace the entire conversation (e.g. when restoring a saved session).

Advances the internal id counter past any restored msg-N ids so newly generated ids don’t collide with restored ones.

pub fn abort(&self)

Request cancellation of an in-flight generation.

pub fn abort_flag(&self) -> Arc<AtomicBool>

Clone of the shared abort flag, for wiring cancellation into a backend.

pub async fn prompt( &mut self, text: impl Into<String>, backend: impl Into<Backend>, tx: UnboundedSender<AgentEvent>, ) -> CoreResult<()>

Run a prompt through the agent loop.

Accepts an event sender so the caller can consume events concurrently while generation is in progress. This enables real-time token streaming to the UI.

Flow:

1. Adds the user message.
2. Generates an assistant response (prune + template + LLM call), streaming tokens via tx.
3. If tools are registered and the response contains tool calls, runs each tool, appends a tool-result message, and loops back to the LLM.
4. Repeats until the model returns a tool-free answer or the max_tool_iterations guard trips.
5. Emits lifecycle events for every step and updates conversation state.

pub fn prompt_stream( &mut self, text: impl Into<String>, backend: impl Into<Backend>, ) -> (UnboundedReceiver<AgentEvent>, impl Future<Output = CoreResult<()>> + '_)

Convenience wrapper over prompt that creates the event channel for you.

Returns the event receiver plus a future that drives generation. Poll the future (e.g. with tokio::join!) while draining the receiver - the two run concurrently so tokens stream as they’re produced:

let mut agent = Agent::new(AgentConfig::default());
let backend: Arc<dyn LlmBackend> = Arc::new(MockBackend);

let (mut rx, run) = agent.prompt_stream("Hello", backend);
let (result, reply) = tokio::join!(run, async move {
    let mut reply = String::new();
    while let Some(event) = rx.recv().await {
        if let AgentEvent::MessageDelta { delta, .. } = event {
            reply.push_str(&delta);
        }
    }
    reply
});
result.unwrap();
assert_eq!(reply, "Hi!");

pub async fn send( &mut self, text: impl Into<String>, backend: impl Into<Backend>, ) -> CoreResult<Message>

Run a turn and answer with the reply, for a caller that is not streaming.

prompt reports everything through events, which is what streaming wants and what a caller that only needs the answer has to unpick for itself. This runs the same turn, drains the events and hands back the assistant’s final message.

An error the agent reported as an event is returned as Err here: a caller with no event stream has nowhere else to see it, and answering Ok with no reply would be a failure that reads like a silence.