Struct Agent 

pub struct Agent { /* private fields */ }

The agent: manages conversation state, context pipeline, and the prompt → LLM → tool → LLM loop.

use orion_core::{Agent, AgentConfig, ContextConfig, InferenceParams};

let mut agent = Agent::new(AgentConfig {
    system_prompt: "You are a coding assistant.".into(),
    inference_params: InferenceParams {
        max_tokens: 4096,
        temperature: 0.4,
        context_size: 8192,
        n_threads: 6,
    },
    context_config: ContextConfig {
        max_context_tokens: 8192,
        max_response_tokens: 4096,
        ..Default::default()
    },
    ..Default::default()
});

// Change settings on the fly.
agent.set_system_prompt("You are a pirate.");
agent.set_inference_params(InferenceParams { temperature: 1.2, ..Default::default() });
agent.clear();

Implementations

impl Agent

pub fn new(config: AgentConfig) -> Self

Create an agent with the given config and the default ChatML template.

pub fn with_template( config: AgentConfig, template: Arc<dyn ChatTemplate>, ) -> Self

Create an agent with an explicit chat template.

pub fn messages(&self) -> &[Message]

The current conversation messages, in order.

pub fn config(&self) -> &AgentConfig

The agent’s current configuration.

pub fn template(&self) -> &dyn ChatTemplate

The chat template currently in use.

pub fn set_system_prompt(&mut self, prompt: impl Into<String>)

Replace the system prompt used for subsequent prompts.

pub fn set_inference_params(&mut self, params: InferenceParams)

Replace the inference parameters used for subsequent generations.

pub fn set_context_config(&mut self, config: ContextConfig)

Replace the context-management configuration.

pub fn set_prune_strategy(&mut self, strategy: PruneStrategy)

Select the strategy used when the conversation overflows the budget.

pub fn set_pinned(&mut self, message_id: &str, pinned: bool) -> bool

Pin or unpin a message by id. Pinned messages always survive context pruning. Returns whether a message with that id was found.

pub fn set_template(&mut self, template: Arc<dyn ChatTemplate>)

Swap the chat template at runtime (e.g. after detecting the model family).

pub fn set_tools(&mut self, tools: Vec<Box<dyn Tool>>)

Register the tools the agent may invoke during a prompt.

Available only with the tools feature (enabled by default).

pub fn clear(&mut self)

Clear the conversation history.

pub fn replace_messages(&mut self, messages: Vec<Message>)

Replace the entire conversation (e.g. when restoring a saved session).

Advances the internal id counter past any restored msg-N ids so newly generated ids don’t collide with restored ones.

pub fn abort(&self)

Request cancellation of an in-flight generation.

pub fn abort_flag(&self) -> Arc<AtomicBool>

Clone of the shared abort flag, for wiring cancellation into a backend.

pub async fn prompt( &mut self, text: impl Into<String>, backend: Arc<dyn LlmBackend>, tx: UnboundedSender<AgentEvent>, ) -> CoreResult<()>

Run a prompt through the agent loop.

Accepts an event sender so the caller can consume events concurrently while generation is in progress. This enables real-time token streaming to the UI.

Flow:

1. Adds the user message.
2. Generates an assistant response (prune + template + LLM call), streaming tokens via tx.
3. If tools are registered and the response contains tool calls, runs each tool, appends a tool-result message, and loops back to the LLM.
4. Repeats until the model returns a tool-free answer or the max_tool_iterations guard trips.
5. Emits lifecycle events for every step and updates conversation state.

pub fn prompt_stream( &mut self, text: impl Into<String>, backend: Arc<dyn LlmBackend>, ) -> (UnboundedReceiver<AgentEvent>, impl Future<Output = CoreResult<()>> + '_)

Convenience wrapper over prompt that creates the event channel for you.

Returns the event receiver plus a future that drives generation. Poll the future (e.g. with tokio::join!) while draining the receiver — the two run concurrently so tokens stream as they’re produced:

let mut agent = Agent::new(AgentConfig::default());
let backend: Arc<dyn LlmBackend> = Arc::new(MockBackend);

let (mut rx, run) = agent.prompt_stream("Hello", backend);
let (result, reply) = tokio::join!(run, async move {
    let mut reply = String::new();
    while let Some(event) = rx.recv().await {
        if let AgentEvent::MessageDelta { delta, .. } = event {
            reply.push_str(&delta);
        }
    }
    reply
});
result.unwrap();
assert_eq!(reply, "Hi!");