Struct tilecoding::IHT

pub struct IHT { /* fields omitted */ }

An index-hash-table, or IHT. It will allow to collect tile indices up to a certain size, after which collisions will start to occur. The underlying storage is a HashMap

Methods

impl IHT

pub fn new(size: usize) -> IHT

Create a new IHT with the given size. The tiles function will never report an index >= this size.

pub fn full(&self) -> bool

Convenience function to determine if the IHT is full. If it is, new tilings will result in collisions rather than new indices.

pub fn count(&self) -> usize

Convenience function to determine how full the IHT is. The maximum value will be the IHT size

pub fn size(&self) -> usize

Convenience function get the size of the IHT, in case you forgot what it was

pub fn tiles(
    &mut self,
    num_tilings: usize,
    floats: &[f64],
    ints: Option<&[isize]>
) -> Vec<usize>

This function takes a series of floating point and integer values, and encodes them as tile indices using the underlying IHT to deal with collisions.

Arguments

• num_tilings—indicates the number of tile indices to be generated (i.e. the length of the returned Vec). This value hould be a power of two greater or equal to four times the number of floats according to the original implementation.
• floats—a list of floating-point numbers to be tiled
• ints—an optional list of integers that will also be tiled; all distinct integers will result in different tilings. In reinforcement learning, discrete actions are often provided here.

Return Value

The returned Vec<usize> is a vector containing exactly num_tilings elements, with each member being an index of a tile encoded by the function. Each member will always be >= 0 and <= size - 1.

Example

// initialize an index-hash-table with size `1024`
let mut iht = IHT::new(1024);
 
// find the indices of tiles for the point (x, y) = (3.6, 7.21) using 8 tilings:
let indices = iht.tiles(8, &[3.6, 7.21], None);
 
// this is the first time we've used the IHT, so we will get the starting tiles:
assert_eq!(indices, vec![0, 1, 2, 3, 4, 5, 6, 7]);
 
// a nearby point:
let indices = iht.tiles(8, &[3.7, 7.21], None);
 
// differs by one tile:
assert_eq!(indices, vec![0, 1, 2, 8, 4, 5, 6, 7]);
 
// and a point more than one away in any dim
let indices = iht.tiles(8, &[-37.2, 7.0], None);
 
// will have all different tiles
assert_eq!(indices, vec![9, 10, 11, 12, 13, 14, 15, 16]);

pub fn tiles_read_only(
    &self,
    num_tilings: usize,
    floats: &[f64],
    ints: Option<&[isize]>
) -> Vec<Option<usize>>

The same as the tiles function, except never insert or generate new indices. If an tiling calculate would result in a new tile, return None instead

pub fn tiles_wrap(
    &mut self,
    num_tilings: usize,
    floats: &[f64],
    wrap_widths: &[Option<isize>],
    ints: Option<&[isize]>
) -> Vec<usize>

A wrap-around version of tiles, described in the original implementation.

Arguments

• num_tilings—indicates the number of tile indices to be generated (i.e. the length of the returned Vec). This value hould be a power of two greater or equal to four times the number of floats according to the original implementation.
• floats—a list of floating-point numbers to be tiled
• wrap_widths—a list of optional integer wrapping points.
• ints—an optional list of integers that will also be tiled; all distinct integers will result in different tilings. In reinforcement learning, discrete actions are often provided here.

Return Value

The returned Vec<usize> is a vector containing exactly num_tilings elements, with each member being an index of a tile encoded by the function. Each member will always be >= 0 and <= size - 1.

Examples

From the original implementation:

The tilings we have discussed so far stretch out to infinity with no need to specify a range for them. This is cool, but sometimes not what you want. Sometimes you want the variables to wrap-around over some range. For example, you may have an angle variable that goes from 0 to 2π and then should wrap around, that is, you would like generalization to occur between angles just greater than 0 and angles that are nearly 2π. To accommodate this, we provide some alternative, wrap-around versions of the tiling routines.

These versions take an additional input, wrap_widths, which parallels the float structure (array or list), and which specifies for each float the width of the range over which it wraps. If you don't want a float to wrap, it's wrap_width should be None. The wrap_width is in the same units as the floats, but should be an integer. This can be confusing, so let's do a simple 1D example. Suppose you have one real input, an angle theta. Theta is originally in radians, that is, in [0,2π), which you would like to wrap around at 2π. Remember that these functions all have their tile boundaries at the integers, so if you passed in theta directly there would be tile boundaries at 0, 1, and 2, i.e., just a few tiles over the whole range, which is probably not what you want. So let's say what we want! Suppose we want tilings with 10 tiles over the [0,2π) range. Then we have to scale theta so that it goes from 0 to 10 (instead of 0 to 2π). One would do this by multiplying theta by 10/2π. With the new scaled theta, the wrapping is over [0,10), for a wrap_width of 10.

Here is the code for the above case, assuming we want 16 tilings over the original [0, 2π) range with a memory size of 512:

let mut iht = IHT::new(512);
iht.tiles_wrap(
    16,
    &[theta * 10. / (2.0 * std::f64::consts::PI)],
    &[Some(10)],
    None
);

Note that the code would be exactly the same if the original range of theta was [-π,+π]. Specifying the complete range of wrapping (rather than just the width) is not necessary for the same reason as we did not need to give a range at all in the previous routines.

As another example, suppose you wanted to cover the 2π x 3 rectangular area with 16 tilings, each with a width of generalization one-tenth of the space in each dimension, and with wrap-around in the second dimension but not the first. In rust you would do:

let mut iht = IHT::new(512);
iht.tiles_wrap(
    16,
    &[x / (3. * 0.1), y / (2.0 * std::f64::consts::PI * 0.1)],
    &[None, Some(10)],
    None
);

pub fn tiles_wrap_read_only(
    &self,
    num_tilings: usize,
    floats: &[f64],
    wrap_widths: &[Option<isize>],
    ints: Option<&[isize]>
) -> Vec<Option<usize>>

The read-only version of tiles_wrap