Crate soul_core 

soul-core

Async agentic runtime for Rust — the engine that powers autonomous agent loops with steerable execution, multi-provider LLM abstraction, and semantic context management.

Quick Start

use soul_core::semantic_recursion::{SemanticContextEngine, RetrievalQuery};

let mut engine = SemanticContextEngine::new();

// Ingest a conversation
let req = engine.ingest_user_request("Explain Rust async");
let resp = engine.ingest_llm_response("Rust async uses tokio...", req, Some("claude"));

// Symlink for compact references
let hash = engine.symlink_node(req).unwrap();
// hash is a 6-char hex like "A3F2B1"

// Fragment long messages into semantic clusters
let fragments = engine.symlink_fragmented(resp);
// Each cluster gets its own symlink hash

// Retrieve relevant context with token budget
let window = engine.retrieve(&RetrievalQuery {
    text: "async programming".into(),
    max_tokens: 4000,
    max_results: 5,
    include_graph_neighbors: true,
});

Architecture

The crate is organized into these core modules:

Module	Purpose
types	Core types: Message, Role, ContentBlock, ToolDefinition, AgentConfig
provider	Multi-provider LLM abstraction (Anthropic, OpenAI) with SSE streaming
agent	Steerable agent loop with tool execution, compaction triggers, interruption
context	Token-aware context window management with compaction and circuit breaker
session	JSONL-based session persistence with lane serialization
tool	Async tool trait and registry for agent tool execution
hook	Three-tier hook pipeline: modifying (sequential), void (parallel), persist (hot path)
subagent	Subagent spawner for parallel task delegation
memory	Hierarchical memory (MEMORY.md + topic files, bootstrap files)
error	Error types with thiserror: Provider, RateLimited, Auth, ContextOverflow, etc.
planner	Task graph with dependencies, status tracking, timing, and display rendering
rlm	Context-as-document engine — agent’s conversation history managed via DSL REPL (infinite context)
semantic_recursion	Knowledge graph context engine — symlinks, TF-IDF, semantic fragmentation (infinite context)

Context Management: The Core Innovation

The semantic_recursion module implements a “social graph for LLM context”:

• Graph nodes represent every piece of context: user requests, LLM responses, tool calls, external data, compaction summaries
• Graph edges encode relationships: RespondsTo, FollowsInSequence, CompactedInto, TriggeredTool, ProvidesData, DerivedFrom
• Nothing is ever deleted — compaction deactivates nodes but preserves them with CompactedInto edges. full_ancestry() traces through any depth back to originals
• Symlinks replace messages with 6-char hash references ([A3F2B1]: summary). LLMs can call resolve_symlink tool to expand any hash
• Semantic fragmentation splits messages into clusters via agglomerative clustering on TF-IDF embeddings. Close sentences group together, distant content gets separate symlinks
• Token-budgeted retrieval combines keyword relevance, vector similarity, and graph neighbor expansion to craft optimal context windows per task

Re-exports

pub use error::SoulError;

pub use types::*;

Modules

agent

context

cost

Cost tracking and budget enforcement for agent sessions.

error

executor

Executor registry — config-driven tool execution with pluggable backends.

gateway

harness

hook

lua

Lua sandbox — embedded Lua 5.4 runtime for skill execution and RLM REPL.

mcp

MCP (Model Context Protocol) client for integrating external tool servers.

memory

observation

Cross-session observation storage (L3 memory layer).

permission

Permission enforcement system for tool execution.

planner

Planner — task graph with dependencies, status tracking, and display formatting.

policy

provider

rlm

Recursive Language Model (RLM) — Infinite Context Engine

semantic_recursion

Semantic Recursion

session

skill

Skills system — config-driven tool definitions loaded from files.

snapshot

Versioned snapshot log — append-only history of arbitrary state.

soullog

Soul Log — unified event logging system.

subagent

tool

types

vexec

Virtual Executor — platform-agnostic command execution.

vfs

Virtual Filesystem — platform-agnostic storage abstraction.