Module patching 

Activation Patching

Token-level activation patching for mechanistic interpretability.

Method

Run the model twice:

1. Clean run – original prompt, record logprobs at each token position.
2. Corrupted run – corrupted prompt (token substitutions), record logprobs.

“Patching” means restoring the clean activation at a specific position into the corrupted run and measuring how much the final output recovers. Because we operate over the API (no direct weight access), we approximate the patch effect via a logit-difference metric:

patch_effect(pos) = logprob_clean(target) - logprob_corrupted(target)

A large positive value at position pos means that token’s activation carries information critical to producing the target output.

Usage

use every_other_token::patching::{ActivationPatcher, PatchConfig, CorruptionStrategy};

let cfg = PatchConfig::default();
let patcher = ActivationPatcher::new(cfg);

let prompt = "The capital of France is";
let corrupted = patcher.corrupt(prompt, CorruptionStrategy::RandomTokenSwap);
assert_ne!(prompt, corrupted.as_str());

Structs

ActivationPatcher

Activation patcher – simulates the clean/corrupted dual-run experiment.

PatchConfig

Configuration for activation patching experiments.

PatchingResult

Full result set for one patching experiment.

PositionPatchResult

Per-position result of an activation patch experiment.

Enums

CorruptionStrategy

How to corrupt the original prompt for the noisy run.