Struct Step

pub struct Step<E: Env> {
    pub act: E::Act,
    pub obs: E::Obs,
    pub reward: Vec<f32>,
    pub is_terminated: Vec<i8>,
    pub is_truncated: Vec<i8>,
    pub info: E::Info,
    pub init_obs: Option<E::Obs>,
}

Represents a single step in the environment, containing the action taken, the resulting observation, reward, and episode status.

This struct encapsulates all the information produced by an environment during a single interaction step. It is used to create transitions of the form (o_t, a_t, o_t+1, r_t) for training reinforcement learning agents.

Type Parameters

• E - The environment type that produced this step

Fields

• act - The action taken by the agent
• obs - The observation received from the environment
• reward - The reward received for the action
• is_terminated - Flags indicating if the episode has terminated
• is_truncated - Flags indicating if the episode has been truncated
• info - Additional environment-specific information
• init_obs - The initial observation of the next episode (if applicable)

Examples

let step = Step::new(
    observation,
    action,
    vec![0.5],  // reward
    vec![0],    // not terminated
    vec![0],    // not truncated
    info,
    None,       // no initial observation
);

if step.is_done() {
    // Handle episode completion
}

Fields

act: E::Act

The action taken by the agent in this step.

obs: E::Obs

The observation received from the environment after taking the action.

reward: Vec<f32>

The reward received for taking the action.

is_terminated: Vec<i8>

Flags indicating if the episode has terminated. A value of 1 indicates termination.

is_truncated: Vec<i8>

Flags indicating if the episode has been truncated. A value of 1 indicates truncation.

info: E::Info

Additional environment-specific information.

init_obs: Option<E::Obs>

The initial observation of the next episode, if applicable. This is used when an episode ends and a new one begins.

Implementations

impl<E: Env> Step<E>

pub fn new( obs: E::Obs, act: E::Act, reward: Vec<f32>, is_terminated: Vec<i8>, is_truncated: Vec<i8>, info: E::Info, init_obs: Option<E::Obs>, ) -> Self

Constructs a new Step object with the given components.

Arguments

• obs - The observation received from the environment
• act - The action taken by the agent
• reward - The reward received for the action
• is_terminated - Flags indicating episode termination
• is_truncated - Flags indicating episode truncation
• info - Additional environment-specific information
• init_obs - The initial observation of the next episode

Returns

A new Step object containing all the provided information

pub fn is_done(&self) -> bool

Checks if the episode has ended, either through termination or truncation.

Returns

true if the episode has ended, false otherwise