Crate hope_agents

Expand description

§HOPE Agents

Hierarchical Optimizing Policy Engine for AIngle AI agents.

§Overview

HOPE Agents is a complete reinforcement learning framework for building autonomous AI agents that can:

🧠 Learn from experience using Q-Learning, SARSA, and TD algorithms
🎯 Plan hierarchically with goal decomposition and conflict resolution
🔮 Predict future states and detect anomalies
🤝 Coordinate with other agents through message passing and shared memory
💾 Persist state for checkpointing and transfer learning

§Architecture

┌─────────────────────────────────────────────────────────────┐
│                      HOPE Agent                             │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  Observation → State → Decision → Action → Learning         │
│                                                              │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────────┐  │
│  │  Predictive  │  │ Hierarchical │  │    Learning      │  │
│  │    Model     │  │ Goal Solver  │  │     Engine       │  │
│  │              │  │              │  │                  │  │
│  │ • Anomaly    │  │ • Goals      │  │ • Q-Learning     │  │
│  │ • Forecast   │  │ • Planning   │  │ • SARSA          │  │
│  │ • Patterns   │  │ • Conflicts  │  │ • TD Learning    │  │
│  │              │  │              │  │ • Experience     │  │
│  └──────────────┘  └──────────────┘  └──────────────────┘  │
│                                                              │
└─────────────────────────────────────────────────────────────┘

§Features

§✅ Complete (100%)

§1. Learning Engine

Algorithms: Q-Learning, SARSA, TD(λ), Expected SARSA
Experience Replay: Prioritized replay buffer for efficient learning
Exploration: Epsilon-greedy and Boltzmann exploration
Value Functions: Tabular and linear function approximation

§2. Hierarchical Goal Solver

Goal Types: Achieve, Maintain, Avoid, Explore
Decomposition: Automatic goal breakdown into subgoals
Conflict Resolution: Detect and resolve goal conflicts
Priorities: Support for goal prioritization

§3. Predictive Model

State Prediction: Forecast next states given actions
Anomaly Detection: Statistical anomaly detection with z-scores
Trajectory Planning: Multi-step lookahead
Transition Model: Learn state transition dynamics

§4. Multi-Agent Coordination

Message Passing: Broadcast and direct messaging
Shared Memory: Global key-value store for coordination
Consensus: Voting-based group decision making
Agent Registry: Dynamic agent registration/unregistration

§5. State Persistence

Formats: JSON, Binary, MessagePack
Compression: Optional compression for efficient storage
Checkpointing: Automatic periodic checkpointing
Transfer Learning: Save and load trained agents

§Quick Start

§Simple Reactive Agent

use hope_agents::{Agent, SimpleAgent, Goal, Observation, Rule, Condition, Action};

// Create a simple reactive agent
let mut agent = SimpleAgent::new("sensor_monitor");

// Add a rule: if temperature > 30, alert
let rule = Rule::new(
    "high_temp",
    Condition::above("temperature", 30.0),
    Action::alert("Temperature too high!"),
);
agent.add_rule(rule);

// Process observations
let obs = Observation::sensor("temperature", 35.0);
agent.observe(obs.clone());
let action = agent.decide();
let result = agent.execute(action.clone());
agent.learn(&obs, &action, &result);

§HOPE Agent with Learning

use hope_agents::{HopeAgent, Observation, Goal, Priority, Outcome, ActionResult};

// Create a HOPE agent with learning, prediction, and hierarchical goals
let mut agent = HopeAgent::with_default_config();

// Set a goal
let goal = Goal::maintain("temperature", 20.0..25.0)
    .with_priority(Priority::High);
agent.set_goal(goal);

// Agent loop with reinforcement learning
for episode in 0..100 {
    let obs = Observation::sensor("temperature", 22.0);
    let action = agent.step(obs.clone());

    // Execute action in environment and get reward
    let reward = 1.0;
    let next_obs = Observation::sensor("temperature", 21.0);
    let result = ActionResult::success(&action.id);

    let outcome = Outcome::new(action, result, reward, next_obs, false);
    agent.learn(outcome);
}

// Check statistics
let stats = agent.get_statistics();
println!("Episodes: {}", stats.episodes_completed);
println!("Success rate: {:.2}%", stats.success_rate * 100.0);

§Multi-Agent Coordination

use hope_agents::{AgentCoordinator, HopeAgent, Message, Observation};
use std::collections::HashMap;

// Create coordinator
let mut coordinator = AgentCoordinator::new();

// Register agents
let agent1 = HopeAgent::with_default_config();
let agent2 = HopeAgent::with_default_config();

let id1 = coordinator.register_agent(agent1);
let id2 = coordinator.register_agent(agent2);

// Broadcast message
coordinator.broadcast(Message::new("update", "System status changed"));

// Step all agents
let mut observations = HashMap::new();
observations.insert(id1, Observation::sensor("temp", 20.0));
observations.insert(id2, Observation::sensor("humidity", 60.0));

let actions = coordinator.step_all(observations);

§State Persistence

use hope_agents::{HopeAgent, AgentPersistence, CheckpointManager};
use std::path::Path;

let mut agent = HopeAgent::with_default_config();

// Train the agent...

// Save agent state
agent.save_to_file(Path::new("agent_state.json")).unwrap();

// Later, load agent state
let loaded_agent = HopeAgent::load_from_file(Path::new("agent_state.json")).unwrap();

// Or use checkpoint manager for automatic checkpointing
let mut manager = CheckpointManager::new(Path::new("checkpoints"), 5)
    .with_interval(1000);

// During training
for step in 0..10000 {
    // ... train agent ...

    if manager.should_checkpoint(step) {
        manager.save_checkpoint(&agent, step).unwrap();
    }
}

§Operation Modes

HOPE agents support multiple operation modes:

Exploration: High exploration rate for discovering new strategies
Exploitation: Use learned knowledge for optimal performance
GoalDriven: Balance exploration with goal achievement
Adaptive: Automatically switch modes based on performance

agent.set_mode(OperationMode::Exploration);  // High exploration
agent.set_mode(OperationMode::Exploitation); // Pure exploitation
agent.set_mode(OperationMode::Adaptive);     // Auto-adjust

§Goal Management

§Goal Types

// Achieve a target value
let goal = Goal::achieve("temperature", 25.0);

// Maintain value in range
let goal = Goal::maintain("humidity", 40.0..60.0);

// Avoid certain values
let goal = Goal::avoid("pressure", 100.0);

// Explore and discover
let goal = Goal::explore("new_area");

§Hierarchical Goals

Goals are automatically decomposed into subgoals:

let parent = Goal::achieve("optimize_system", 1.0)
    .with_priority(Priority::High);

let goal_id = agent.set_goal(parent);

// Automatically creates subgoals for different aspects
let active_goals = agent.active_goals();

§Consensus and Coordination

Agents can make group decisions through voting:

let mut coordinator = AgentCoordinator::new();

// Create proposal
let proposal_id = coordinator.create_proposal(
    "new_policy",
    "Should we adopt the new temperature policy?"
);

// Agents vote
// ... (voting happens through message passing) ...

// Check consensus
match coordinator.get_consensus(&proposal_id) {
    Some(ConsensusResult::Decided { approved, votes_for, votes_against, .. }) => {
        println!("Decision: {}", if approved { "Approved" } else { "Rejected" });
        println!("Votes: {} for, {} against", votes_for, votes_against);
    }
    _ => println!("Voting in progress..."),
}

§Configuration

§IoT Mode

Optimized for resource-constrained devices:

use hope_agents::AgentConfig;

let config = AgentConfig::iot_mode();
let agent = SimpleAgent::with_config("iot_agent", config);

// Features:
// - Limited memory (128KB)
// - Disabled learning
// - Reduced buffer sizes

§Custom Configuration

use hope_agents::{HopeConfig, LearningConfig, PredictiveConfig, LearningAlgorithm};

let config = HopeConfig {
    learning: LearningConfig {
        learning_rate: 0.1,
        discount_factor: 0.95,
        algorithm: LearningAlgorithm::QLearning,
        epsilon: 0.2,
        ..Default::default()
    },
    predictive: PredictiveConfig {
        history_size: 500,
        ..Default::default()
    },
    anomaly_sensitivity: 0.8,
    auto_decompose_goals: true,
    ..Default::default()
};

let agent = HopeAgent::new(config);

§Performance

HOPE Agents is designed for high performance:

1000+ steps/second on modern hardware
Efficient memory usage with configurable limits
Incremental learning with no batch processing required
Lock-free coordination for multi-agent scenarios

§Benchmarks

Run benchmarks with:

cargo bench

§Testing

Comprehensive test suite with 133 tests covering:

Unit tests for all components
Integration tests for complete workflows
Multi-agent coordination scenarios
Persistence roundtrip tests
Performance tests

Run tests:

# All tests
cargo test

# Specific module
cargo test coordination
cargo test persistence

# Integration tests
cargo test --test integration_test

# With output
cargo test -- --nocapture

§Examples

See the tests/integration_test.rs file for comprehensive examples demonstrating:

Simple agent workflows
Learning cycles with multiple episodes
Multi-agent coordination
Consensus mechanisms
State persistence and checkpointing
Anomaly detection
Operation mode switching

§API Documentation

Full API documentation is available at docs.rs/hope_agents.

Generate local documentation:

cargo doc --open

§Integration with AIngle

HOPE Agents integrates seamlessly with the AIngle network:

// Observe network events
let obs = Observation::network_event("node_joined", node_id);

// Execute actions on the network
let action = Action::send_message("Hello, network!");

// Store data in AIngle
let action = Action::store_data(serde_json::to_string(&data).unwrap());

§Roadmap

Future enhancements (beyond 100%):

Deep Q-Networks (DQN) support
Policy gradient methods (PPO, A3C)
Multi-objective optimization
Distributed training
Web assembly support
Python bindings

§Contributing

Contributions are welcome! Please see CONTRIBUTING.md for guidelines.

§License

Licensed under the Apache License, Version 2.0. See LICENSE for details.

§Citation

If you use HOPE Agents in your research, please cite:

@software{hope_agents,
  title = {HOPE Agents: Hierarchical Optimizing Policy Engine},
  author = {Apilium Technologies},
  year = {2025},
  url = {https://github.com/ApiliumCode/aingle}
}

§Support

Documentation: https://docs.rs/hope_agents
Issues: https://github.com/ApiliumCode/aingle/issues

Status: ✅ 100% Complete

All core features implemented, tested, and documented.

§HOPE Agents - Hierarchical Optimizing Policy Engine

Autonomous AI agents framework for AIngle semantic networks.

§Overview

HOPE Agents provides a complete framework for building autonomous AI agents that can:

Observe their environment (IoT sensors, network events, user inputs)
Decide based on learned policies and hierarchical goals
Execute actions in the AIngle network
Learn and adapt over time using reinforcement learning

This crate is designed for use cases ranging from simple reactive agents to complex multi-agent systems with learning capabilities

§Architecture

┌─────────────────────────────────────────────────────────────┐
│                       HOPE Agent                            │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────────┐  │
│  │   Sensors    │  │   Policy     │  │    Actuators     │  │
│  │              │  │   Engine     │  │                  │  │
│  │ • IoT data   │─►│              │─►│ • Network calls  │  │
│  │ • Events     │  │ • Goals      │  │ • State changes  │  │
│  │ • Messages   │  │ • Rules      │  │ • Messages       │  │
│  └──────────────┘  │ • Learning   │  └──────────────────┘  │
│                    └──────┬───────┘                         │
│                           │                                 │
│                    ┌──────▼───────┐                         │
│                    │   Memory     │                         │
│                    │ (Titans)     │                         │
│                    │              │                         │
│                    │ STM ◄──► LTM │                         │
│                    └──────────────┘                         │
│                                                              │
└─────────────────────────────────────────────────────────────┘

§Quick Start

§Simple Reactive Agent

use hope_agents::{Agent, SimpleAgent, Goal, Observation, Rule, Condition, Action};

// Create a simple reactive agent
let mut agent = SimpleAgent::new("sensor_monitor");

// Add a rule: if temperature > 30, alert
let rule = Rule::new(
    "high_temp",
    Condition::above("temperature", 30.0),
    Action::alert("Temperature too high!"),
);
agent.add_rule(rule);

// Process observations
let obs = Observation::sensor("temperature", 35.0);
agent.observe(obs.clone());
let action = agent.decide();
let result = agent.execute(action.clone());
agent.learn(&obs, &action, &result);

§HOPE Agent with Learning

use hope_agents::{HopeAgent, HopeConfig, Observation, Goal, Priority, Outcome};

// Create a HOPE agent with learning, prediction, and hierarchical goals
let mut agent = HopeAgent::with_default_config();

// Set a goal
let goal = Goal::maintain("temperature", 20.0..25.0)
    .with_priority(Priority::High);
agent.set_goal(goal);

// Agent loop with reinforcement learning
for episode in 0..100 {
    let obs = Observation::sensor("temperature", 22.0);
    let action = agent.step(obs.clone());

    // Execute action in environment and get reward
    let reward = 1.0; // Example reward
    let next_obs = Observation::sensor("temperature", 21.0);

    let outcome = Outcome::new(action, result, reward, next_obs, false);
    agent.learn(outcome);
}

§Multi-Agent Coordination

use hope_agents::{AgentCoordinator, HopeAgent, Message, Observation};
use std::collections::HashMap;

// Create coordinator
let mut coordinator = AgentCoordinator::new();

// Register agents
let agent1 = HopeAgent::with_default_config();
let agent2 = HopeAgent::with_default_config();

let id1 = coordinator.register_agent(agent1);
let id2 = coordinator.register_agent(agent2);

// Broadcast message
coordinator.broadcast(Message::new("update", "System status changed"));

// Step all agents
let mut observations = HashMap::new();
observations.insert(id1, Observation::sensor("temp", 20.0));
observations.insert(id2, Observation::sensor("humidity", 60.0));

let actions = coordinator.step_all(observations);

§State Persistence

use hope_agents::{HopeAgent, AgentPersistence};
use std::path::Path;

let mut agent = HopeAgent::with_default_config();

// Train the agent...

// Save agent state
agent.save_to_file(Path::new("agent_state.json")).unwrap();

// Later, load agent state
let loaded_agent = HopeAgent::load_from_file(Path::new("agent_state.json")).unwrap();

§Agent Types

ReactiveAgent: Simple stimulus-response behavior
GoalBasedAgent: Works toward explicit goals
LearningAgent: Adapts behavior over time
CooperativeAgent: Coordinates with other agents

Re-exports§

pub use action::Action;
pub use action::ActionResult;
pub use action::ActionType;
pub use agent::Agent;
pub use agent::AgentId;
pub use agent::AgentState;
pub use agent::SimpleAgent;
pub use config::AgentConfig;
pub use coordination::AgentCoordinator;
pub use coordination::ConsensusResult;
pub use coordination::CoordinationError;
pub use coordination::Message;
pub use coordination::MessageBus;
pub use coordination::MessageId;
pub use coordination::MessagePayload;
pub use coordination::MessagePriority;
pub use coordination::SharedMemory;
pub use error::Error;
pub use error::Result;
pub use goal::Goal;
pub use goal::GoalPriority;
pub use goal::GoalStatus;
pub use goal::GoalType;
pub use hierarchical::ConflictResolution;
pub use hierarchical::ConflictType;
pub use hierarchical::DecompositionResult;
pub use hierarchical::DecompositionRule;
pub use hierarchical::DecompositionStrategy;
pub use hierarchical::GoalConflict;
pub use hierarchical::GoalTree;
pub use hierarchical::GoalTypeFilter;
pub use hierarchical::HierarchicalGoalSolver;
pub use hierarchical::ParallelStrategy;
pub use hierarchical::SequentialStrategy;
pub use hierarchical::default_decomposition_rules;
pub use hope_agent::AgentStats;
pub use hope_agent::GoalSelectionStrategy;
pub use hope_agent::HopeAgent;
pub use hope_agent::HopeConfig;
pub use hope_agent::OperationMode;
pub use hope_agent::Outcome;
pub use hope_agent::SerializedState;
pub use learning::ActionId;
pub use learning::Experience;
pub use learning::LearningAlgorithm;
pub use learning::LearningConfig;
pub use learning::LearningEngine;
pub use learning::QValue;
pub use learning::StateActionPair;
pub use learning::StateId;
pub use observation::Observation;
pub use observation::ObservationType;
pub use observation::Sensor;
pub use persistence::AgentPersistence;
pub use persistence::CheckpointManager;
pub use persistence::LearningSnapshot;
pub use persistence::PersistenceError;
pub use persistence::PersistenceFormat;
pub use persistence::PersistenceOptions;
pub use policy::Condition;
pub use policy::Policy;
pub use policy::PolicyEngine;
pub use policy::Rule;
pub use predictive::AnomalyDetector;
pub use predictive::PredictedState;
pub use predictive::PredictiveConfig;
pub use predictive::PredictiveModel;
pub use predictive::StateEncoder;
pub use predictive::StateSnapshot;
pub use predictive::Trajectory;
pub use predictive::TransitionModel;
pub use types::*;

Modules§

action: Action types for HOPE Agents
agent: Agent implementations for HOPE framework
config: Configuration for HOPE Agents
coordination: Multi-Agent Coordination
error: Error types for HOPE Agents
goal: Goal types for HOPE Agents
hierarchical: Hierarchical goal decomposition and management for HOPE agents.
hope_agent: HOPE Agent Orchestrator
learning: Learning module for HOPE Agents
observation: Observation types for HOPE Agents
persistence: Agent State Persistence
policy: Policy engine for HOPE Agents
predictive: Predictive modeling for state and reward prediction in HOPE agents.
types: Core types for HOPE Agents

Constants§

VERSION: HOPE framework version

Functions§

create_agent: Create a simple agent with default configuration.
create_iot_agent: Create an IoT-optimized agent with reduced memory footprint.