Struct radiate::models::neat::layers::lstm::LSTM

pub struct LSTM {
    pub input_size: u32,
    pub memory_size: u32,
    pub output_size: u32,
    pub activation: Activation,
    pub memory: Vec<f32>,
    pub hidden: Vec<f32>,
    pub states: LSTMState,
    pub g_gate: Arc<RwLock<Dense>>,
    pub i_gate: Arc<RwLock<Dense>>,
    pub f_gate: Arc<RwLock<Dense>>,
    pub o_gate: Arc<RwLock<Dense>>,
    pub v_gate: Arc<RwLock<Dense>>,
}

LSTM is a long-short term memory cell represented by a collection of Dense layers and two distinct memory vectors which get updated and travel 'through time'

Fields

input_size: u32

memory_size: u32

output_size: u32

activation: Activation

memory: Vec<f32>

hidden: Vec<f32>

states: LSTMState

g_gate: Arc<RwLock<Dense>>

i_gate: Arc<RwLock<Dense>>

f_gate: Arc<RwLock<Dense>>

o_gate: Arc<RwLock<Dense>>

v_gate: Arc<RwLock<Dense>>

Implementations

impl LSTM

pub fn new(
    input_size: u32,
    memory_size: u32,
    output_size: u32,
    activation: Activation
) -> Self

pub fn step_forward_async(&mut self, inputs: &[f32]) -> Option<Vec<f32>>

Feed forward with each forward propagation being executed in a separate thread to speed up the forward pass if the network is NOT being evolved. If it is, there are already so many threads working to optimize the entire population that extra threading is unnecessary and might actually slow it down

pub fn step_forward(&mut self, inputs: &[f32]) -> Option<Vec<f32>>

step forward synchronously

pub fn step_back(&mut self, errors: &Vec<f32>, l_rate: f32) -> Option<Vec<f32>>

Preform one step backwards for the layer. Set the tracer historical meta data to look at the current index, and use that data to compute the gradient steps for each weight in each gated network. If update is true, the gates will take the accumulated gradient steps, and add them to their respective weight values

Trait Implementations

impl Clone for LSTM

Implement clone for the neat neural network in order to facilitate proper crossover and mutation for the network (*self.environment.read().unwrap()).clone();

fn clone(&self) -> Self

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LSTM

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for LSTM

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error> where
    __D: Deserializer<'de>, 

Deserialize this value from the given Serde deserializer.

impl Display for LSTM

implement display for the LSTM layer of the network

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Genome<LSTM, NeatEnvironment> for LSTM where
    LSTM: Layer, 

in order for the lstm layer to be evolved along with the rest of the network, Genome must be implemented so that the layer can be crossed over and measured along with other lstm layers

fn crossover(
    child: &LSTM,
    parent_two: &LSTM,
    env: Arc<RwLock<NeatEnvironment>>,
    crossover_rate: f32
) -> Option<LSTM>

implement how to crossover two LSTM layers

fn distance(one: &LSTM, two: &LSTM, env: Arc<RwLock<NeatEnvironment>>) -> f32

get the distance between two LSTM layers of the network

fn base(_: &mut E) -> T where
    T: Sized, 

Genome needs to have a base implementation in order for one of the population options to be satisfied

impl Layer for LSTM

fn forward(&mut self, inputs: &Vec<f32>) -> Option<Vec<f32>>

forward propagate inputs, if the model is being evolved don't spawn extra threads because it slows down the process by about double the original time. If the model is being trained traditionally, step forward asynchronously by spawning a thread for each individual gate which results in speeds about double as a synchronous thread.

fn backward(
    &mut self,
    errors: &Vec<f32>,
    learning_rate: f32
) -> Option<Vec<f32>>

apply backpropagation through time asynchronously because this is not done during evolution

fn reset(&mut self)

reset the lstm network by clearing the tracer and the states as well as the memory and hidden state

fn add_tracer(&mut self)

add tracers to all the gate.write().unwrap()s in the layer

fn remove_tracer(&mut self)

remove the tracers from all the gate.write().unwrap()s in the layer

fn as_ref_any(&self) -> &dyn Any where
    Self: Sized + 'static, 

Get a reference to the underlying type without generics in order to downcast to a concrete type

fn as_mut_any(&mut self) -> &mut dyn Any where
    Self: Sized + 'static, 

Get a mutable reference to the underlying type without generics in order to downcast to a concrete type

fn shape(&self) -> (usize, usize)

Return the (input size, output size) of this layer - used to make specifying layer sizes easier so the user only needs the say the size of the output, not the input. That would be too redundant.

impl Serialize for LSTM

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error> where
    __S: Serializer, 

Serialize this value into the given Serde serializer.