madepro

A minimal Rust library crate for solving finite deterministic Markov decision processes (MDPs).

Limitations

This library is still in development and is not ready for production use. It only implements a few algorithms and one environment. It is also limited to deterministic MDPs. Originally, I only created this library in order to refactor my blog post on Markov Decision Processes. However, I decided to probably continue working on it when I have the time. In the future I would first like to make the library more generic and then add more algorithms and environments.

Features

The library currently supports the following algorithms:

• Policy Iteration
• Value Iteration
• SARSA
• Q-Learning

Roadmap

The following features are planned for the future:

• Stochastic MDPs
• Stochastic policies
• State-dependent action spaces
• More algorithms