Module grid_world 

GridWorld sparse-reward discrete navigation environment.

GridWorld is a small, pure-Rust, zero-external-dependency FrozenLake-style navigation task (issue #182, part of the more-environments epic #180). It fills the gap in Thrust’s discrete-action catalog (CartPole, Snake, Pong, …) for a sparse-reward navigation problem: the agent must reach a goal on a grid under a per-step penalty, with absorbing terminal states (goal/hazard) that are distinct from step-cap truncation.

The agent occupies one cell of a fixed 4x4 grid and moves between cells. The default layout mirrors the classic FrozenLake-v1 4x4 map (without slip): S start, F frozen/empty, H hole/hazard, G goal.

S F F F
F H F H
F F F H
H F F G

• Action: i64, four discrete moves: 0=Up, 1=Right, 2=Down, 3=Left. Out-of-range / invalid actions are treated as a no-op (the agent stays in place).
• Observation: one-hot over cells, a Vec<f32> of length GRID_ROWS * GRID_COLS (= 16). The entry at the agent’s flattened index (row * GRID_COLS + col) is 1.0; every other entry is 0.0. This keeps the Q-network input fixed-width and discrete.
• Reward: reaching the goal yields GOAL_REWARD (+1.0); falling into a hole yields HOLE_REWARD (-1.0); any other move (including bumping into a wall) yields STEP_PENALTY (-0.01). The goal reward dominates so an optimal shortest path is the highest-return policy.
• Termination: the goal and hole cells are absorbing — terminated = true. Otherwise the episode runs until the step cap.
• Truncation: if not already terminated, truncated = true once steps reaches max_steps (default DEFAULT_MAX_STEPS = 100). Truncation is not termination.

Determinism contract

The default layout has no stochastic dynamics: Environment::step is fully deterministic given the current cell and action. Environment::reset always returns the agent to the fixed start cell. A seeded StdRng is held to match the surface of the other envs (and is reserved for an optional later slip/random-layout mode), but it is not consulted on the default layout. Consequently Environment::restore_state followed by Environment::step reproduces every subsequent StepResult bit-for-bit, and two envs constructed with the same seed produce identical episodes. The GridWorldState snapshot captures the agent position and step counter (agent_row, agent_col, steps) but not the RNG.

Structs

GridWorld

Sparse-reward discrete navigation task on a fixed 4x4 grid.

GridWorldState

Snapshot of GridWorld’s simulation state.

Constants

DEFAULT_MAX_STEPS

Default episode length cap before truncation.

DEFAULT_SEED

Default seed used by GridWorld::new.

GOAL_REWARD

Reward for reaching the goal cell.

GRID_COLS

Number of columns in the grid.

GRID_ROWS

Number of rows in the grid.

HOLE_REWARD

Reward for falling into a hole cell.

NUM_ACTIONS

Number of discrete actions (Up, Right, Down, Left).

STEP_PENALTY

Per-step penalty for any non-terminal move (encourages short paths).