Struct NfspTrainer 

pub struct NfspTrainer<B, P, O, E, FP, FO, FE>
where
    B: AutodiffBackend,
    P: JointPolicy<B>,
    O: Optimizer<P, B>,
    E: JointEnv,
    FP: Fn(&B::Device, u64) -> P,
    FO: Fn() -> BurnOptimizer<B, P, O>,
    FE: Fn() -> E,
{ /* private fields */ }

NFSP outer-loop trainer.

Generic over the Burn backend B, policy module P, optimizer type O, env E, and the user-supplied policy/optimizer/env factories. The trainer owns:

• A JointMultiAgentTrainer holding the N best-response (BR) policies. BR updates go through JointMultiAgentTrainer::update_with_active_agents — the freeze-N-1 primitive PSRO also uses.
• N average-policy (AP) Burn modules and their independent optimizers. AP updates are plain supervised cross-entropy on reservoir samples.
• N reservoir buffers (one per agent) carrying (obs, action) pairs harvested from BR-sampled rollout steps.
• An internal StdRng (seeded from NfspConfig::seed) used for η-anticipatory coin flips, reservoir eviction, and AP minibatch sampling.

As of #119, joint_config.num_agents may be any value ≥ 2. The formal ε-Nash convergence guarantee from Heinrich & Silver 2016 §3 holds only for N = 2 zero-sum games; for N > 2 this trainer ships the same per-agent BR/AP/reservoir machinery as an “approximate NFSP” — see the module docstring for the caveat.

Single-policy-class assumption

All 2N policies (N BR + N AP) share the same module type P. For symmetric matching-pennies-style games this is what we want.

See also

• crate::multi_agent::psro::PsroTrainer — sibling meta-game trainer with the same JointMultiAgentTrainer-based freeze-N-1 plumbing.

Implementations

impl<B, P, O, E, FP, FO, FE> NfspTrainer<B, P, O, E, FP, FO, FE>

where B: AutodiffBackend, P: JointPolicy<B>, O: Optimizer<P, B>, E: JointEnv, FP: Fn(&B::Device, u64) -> P, FO: Fn() -> BurnOptimizer<B, P, O>, FE: Fn() -> E,

pub fn new( config: NfspConfig, joint_config: JointTrainerConfig, device: B::Device, policy_factory: FP, optimizer_factory: FO, env_factory: FE, ) -> Result<Self>

Construct an NFSP trainer with fresh BR and AP policies.

joint_config.num_agents must be ≥ 2. For N = 2 the trainer retains Heinrich & Silver §3’s ε-Nash convergence guarantee on zero-sum games; for N > 2 it ships as “approximate NFSP” with no formal guarantee beyond per-agent reservoir uniformity (see the module docstring).

pub fn br_policy(&self, i: usize) -> &P

Borrow agent i’s best-response (BR) policy.

pub fn avg_policy(&self, i: usize) -> &P

Borrow agent i’s average (AP) policy.

pub fn reservoir(&self, i: usize) -> &ReservoirBuffer<(Vec<f32>, Vec<i64>)>

Borrow agent i’s reservoir buffer.

pub fn cumulative_br_pushes(&self) -> usize

Cumulative BR-sampled push count across all reservoirs.

pub fn cumulative_rollout_steps(&self) -> usize

Cumulative rollout-step count across all agents.

pub fn config(&self) -> &NfspConfig

Trainer configuration.

pub fn run<F>(&mut self, on_iteration: F) -> Result<NfspStats>

where F: FnMut(&NfspIterationStats), P: Send + Sync, O: Send, B::Device: Sync,

Run the NFSP outer loop and return the per-iteration history.

pub fn run_silent(&mut self) -> Result<NfspStats>

where P: Send + Sync, O: Send, B::Device: Sync,

Convenience entry point: drives Self::run with a no-op iteration callback.

pub fn action_marginal_for(&mut self, policy: &P) -> Option<Vec<f32>>

Average-policy action distribution under a constant observation [0.0; obs_dim]. Returns None if the policy’s action_dims is unsupported (e.g. multi-dim multi-discrete). Used as a diagnostic on matching-pennies and as the load-bearing convergence assertion in tests/test_nfsp_matching_pennies.rs.

Takes &mut self because the 128-probe sampling loop consumes the trainer-owned StdRng via JointPolicy::get_action_host_seeded. This makes the diagnostic reproducible under NfspConfig::seed (issue #114). Callers that hold a &P from Self::avg_policy / Self::br_policy must .clone() the policy before calling this method to avoid a &self / &mut self aliasing conflict — the policy clone is cheap (Burn modules are Clone by design).