Struct Ucb1 

pub struct Ucb1 { /* private fields */ }

Available on crate feature bandit only.

UCB1 multi-armed bandit.

Selects arms using the upper confidence bound formula, which automatically balances exploration and exploitation. Unpulled arms are always selected first. Rewards passed to Bandit::update must be normalized to [0, 1].

Examples

use rill_ml::bandit::{Bandit, Ucb1, Ucb1Config};
use rand::SeedableRng;
use rand_chacha::ChaCha8Rng;

let mut rng = ChaCha8Rng::seed_from_u64(0);
let mut bandit = Ucb1::new(3, Ucb1Config::default()).unwrap();

let arm = bandit.select(&mut rng).unwrap();
bandit.update(arm, 1.0).unwrap();
assert_eq!(bandit.samples_seen(), 1);

Implementations

impl Ucb1

pub fn new(arm_count: usize, config: Ucb1Config) -> Result<Self, RillError>

Create a new UCB1 bandit.

Errors

Returns RillError::InvalidArmCount if arm_count is zero. Returns RillError::InvalidParameter if exploration_constant is not finite and positive.

pub fn pulls(&self) -> &[u64]

Per-arm pull counts (diagnostic).

pub fn total_rewards(&self) -> &[f64]

Per-arm total rewards (diagnostic).

pub fn validate(&self) -> Result<(), RillError>

Validate all persisted state invariants.

This is also run automatically during deserialization.

Trait Implementations

impl Bandit for Ucb1

fn arm_count(&self) -> usize

The number of arms (actions) available.

fn samples_seen(&self) -> u64

How many total samples (arm pulls) the bandit has observed.

fn select(&self, rng: &mut impl Rng) -> Result<usize, RillError>

Select an arm using the provided RNG for exploration.

fn update(&mut self, arm: usize, reward: f64) -> Result<(), RillError>

Update the bandit with the observed reward for a previously selected arm.

fn reset(&mut self)

Reset the bandit to its initial (no-data) state.

fn arm_stats(&self, arm: usize) -> Result<ArmStats, RillError>

Per-arm diagnostic statistics.

impl Clone for Ucb1

fn clone(&self) -> Ucb1

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Ucb1

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.