Struct TokenEvent 

pub struct TokenEvent {
    pub text: String,
    pub original: String,
    pub index: usize,
    pub transformed: bool,
    pub importance: f64,
    pub chaos_label: Option<String>,
    pub provider: Option<String>,
    pub confidence: Option<f32>,
    pub perplexity: Option<f32>,
    pub alternatives: Vec<TokenAlternative>,
    pub is_error: bool,
    pub arrival_ms: Option<u64>,
}

A single processed token emitted by the streaming pipeline.

Every token the interceptor produces — whether transformed or not — is represented as a TokenEvent. Events are sent over the web_tx channel for SSE fan-out to the web UI, written as JSON lines in --json-stream mode, or recorded to a replay file.

Fields

text: String

The (possibly transformed) token text shown to the user.

original: String

The original token text before any transform was applied.

index: usize

Zero-based position of this token in the full response.

transformed: bool

Whether the active transform was applied to this token.

importance: f64

Scalar token importance in [0.0, 1.0] — derived from API confidence when available, otherwise computed by the heuristic importance scorer.

chaos_label: Option<String>

For Chaos transform: which sub-transform was applied. None for other transforms.

provider: Option<String>

For diff mode: which provider produced this token (“openai” or “anthropic”).

confidence: Option<f32>

Model confidence 0.0–1.0, derived from API logprob. None when unavailable.

perplexity: Option<f32>

Per-token perplexity (exp(-log_prob)). None when logprobs unavailable.

alternatives: Vec<TokenAlternative>

Top alternative tokens with their probabilities (from top_logprobs).

is_error: bool

When true, this event represents an error notification rather than a real token.

arrival_ms: Option<u64>

Milliseconds elapsed since stream start when this token arrived (for latency tracking).

Trait Implementations

impl Clone for TokenEvent

fn clone(&self) -> TokenEvent

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenEvent

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for TokenEvent

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for TokenEvent

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.