learnwell

Easy reinforcement learning framework, allowing you to quickly create Environments and test them.
Aims to be simple
Minimal external dependencies

Currently only Q-Learning is implemented

The state of this project is in alpha. Use at your own risk.

Getting started

See the taxi example and walk through the comments You can also run the examples for hike, taxi, mouse with: cargo run --release --example taxi

Imports:

use learnwell::{
    runner::Runner, 
    agent::qlearning::QLearning, 
    environment::{Environment, EnvironmentDisplay}
    strategy::decliningrandom::DecliningRandom, 
    };

We then ask the Runner to run the agent for x number of epochs

Allows 2 modes:

• Runner::run for normal operation
• Runner::run_with_display to create a window and display image which gets updated as it runs

For example:

    Runner::run(
        QLearning::new(0.1, 0.98, DecliningRandom::new(epochs, 0.01)), //Agent
        TaxiEnvironment::default(), //Environment
        400, //epochs
    );

or

Runner::run_with_display(
        QLearning::new(0.2, 0.99,DecliningRandom::new(epochs, 0.005) ), //Agent
        Hike::new(), //Environment
        700_000, //epochs
        10 //frames per second to refresh image
    );

We need:

• Environment - this is the game/scenario we want to learn
• Agent - this is what interacts with the environment

We implement a few things to run

Environment

1. State Struct - this is what we base our actions on
2. Action (normally enum) - these are the actions we perform
3. Environment Struct that implements the Environment<S,A> trait and depends on the State and Action. The Environment struct should hold the state, because we will refer to it later

Agent

4. the Agent algorithm (e.g. QLearning),

Implementation:

Note we derive Hash, Eq, PartialEq and Clone for both State and Action

State

#[derive(Hash, Eq, PartialEq, Clone)]
pub struct TaxiState {
    taxi: Point,
    dropoff: Point,
    passenger: Point, 
    in_taxi: bool,
}

Action

#[derive(Clone, Hash, PartialEq, Eq)]
pub enum TaxiAction {
    Up,
    Down,
    Left,
    Right,
    Dropoff,
    Pickup,
}

Environment

pub struct TaxiEnvironment {
    state: TaxiState, //this is the actual state that gets saved in the qtable
    found: usize, //just a helper. there could be a few other items you want to track in the environment
}

Status

• implement Qlearning
• implement deep qlearning
• move optional functionality to features (e.g. display, fxhasher)