Struct TokenInterceptor 

pub struct TokenInterceptor {

    pub provider: Provider,
    pub transform: Transform,
    pub model: String,
    pub token_count: usize,
    pub transformed_count: usize,
    pub visual_mode: bool,
    pub heatmap_mode: bool,
    pub orchestrator: bool,
    pub orchestrator_url: String,
    pub web_tx: Option<UnboundedSender<TokenEvent>>,
    pub web_provider_label: Option<String>,
    pub system_prompt: Option<String>,
    pub rate: f64,
    pub top_logprobs: u8,
    pub recorder: Option<Recorder>,
    pub json_stream: bool,
    pub min_confidence: Option<f64>,
    pub max_retries: u32,
    pub anthropic_max_tokens: u32,
    pub timeout_secs: Option<u64>,
    /* private fields */
}

The core streaming engine that sits between the caller and the LLM.

TokenInterceptor manages the HTTP connection to the configured provider, iterates the server-sent-event (SSE) stream, applies the active Transform to every N-th token (controlled by rate), attaches per-token confidence and perplexity from API logprobs, and routes enriched TokenEvents to one of three output sinks:

• Terminal — ANSI-colored text written to stdout.
• Web UI — events sent over the web_tx unbounded channel for SSE fan-out.
• JSON stream — one JSON line per token written to stdout (json_stream = true).

Construct with TokenInterceptor::new then call TokenInterceptor::intercept_stream.

Fields

provider: Provider

transform: Transform

model: String

token_count: usize

transformed_count: usize

visual_mode: bool

heatmap_mode: bool

orchestrator: bool

orchestrator_url: String

web_tx: Option<UnboundedSender<TokenEvent>>

When set, token events are sent here instead of printed to stdout.

web_provider_label: Option<String>

When set, each emitted TokenEvent carries this provider label (for diff mode).

system_prompt: Option<String>

Optional system prompt prepended to the conversation.

rate: f64

Fraction of tokens to transform (0.0–1.0). Bresenham-spread so the distribution is deterministic and uniform rather than probabilistic.

top_logprobs: u8

Number of top alternative tokens to request per position (OpenAI only, 0–20).

recorder: Option<Recorder>

Optional replay recorder — records each emitted TokenEvent.

json_stream: bool

When true, print one JSON line per token instead of colored text.

min_confidence: Option<f64>

Minimum confidence threshold for transform gating. When set, only tokens with confidence at or below this value are transformed.

max_retries: u32

Maximum retry attempts for API calls on 429/5xx (configurable via –max-retries).

anthropic_max_tokens: u32

Maximum tokens in the Anthropic response (configurable via –anthropic-max-tokens).

timeout_secs: Option<u64>

Optional stream timeout in seconds. When set, intercept_stream will fail with a timeout error if the entire stream does not complete within this duration.

Implementations

impl TokenInterceptor

pub fn new( provider: Provider, transform: Transform, model: String, visual_mode: bool, heatmap_mode: bool, orchestrator: bool, ) -> Result<Self, Box<dyn Error>>

Construct a new TokenInterceptor.

Reads the API key from the environment (OPENAI_API_KEY or ANTHROPIC_API_KEY) and validates its format. The Mock provider does not require a key.

Errors

Returns an error if the required API key environment variable is not set.

pub fn with_rate(self, rate: f64) -> Self

Set the intercept rate (0.0–1.0). Clamped to [0.0, 1.0].

pub fn with_seed(self, seed: u64) -> Self

Seed the internal RNG for reproducible Noise/Chaos output.

pub fn with_web_tx(self, tx: UnboundedSender<TokenEvent>) -> Self

Set the channel used to fan out token events to the web UI.

Calling this completes the builder chain for web-mode construction so callers do not need to set web_tx as a bare field assignment.

pub fn with_provider_label(self, label: impl Into<String>) -> Self

Set an optional provider label attached to every emitted TokenEvent. Used in diff mode to tag events with "openai" or "anthropic".

pub fn with_system_prompt(self, prompt: impl Into<String>) -> Self

Prepend a system prompt to the conversation.

pub fn with_top_logprobs(self, n: u8) -> Self

Number of top alternative tokens to request per position (OpenAI only, 0–20).

pub fn with_json_stream(self, enabled: bool) -> Self

Enable JSON-stream mode: emit one JSON line per token instead of ANSI text.

pub fn with_orchestrator_url(self, url: impl Into<String>) -> Self

Override the MCP orchestrator base URL (default: http://localhost:3000).

pub fn with_max_retries(self, n: u32) -> Self

Maximum retry attempts on 429/5xx errors.

pub fn with_timeout(self, secs: u64) -> Self

Set a stream timeout in seconds. If the entire stream does not complete within this duration, intercept_stream returns a timeout error.

pub fn with_min_confidence(self, threshold: f64) -> Self

Only transform tokens whose API confidence is at or below this threshold.

pub async fn intercept_stream( &mut self, prompt: &str, ) -> Result<(), Box<dyn Error>>

Stream the given prompt through the configured provider, applying the active transform to every other token.

In terminal mode the tokens are printed to stdout; in web mode they are sent over the web_tx channel for SSE fan-out.

Errors

Returns an error if the prompt is empty, exceeds 512 KB, the API key is missing, the HTTP request fails after all retries, or JSON parsing fails.

pub fn process_content(&mut self, content: &str)

Process a content chunk without logprob data.

pub fn process_content_with_logprob( &mut self, content: &str, lp: Option<OpenAILogprobContent>, )

Process a content chunk with optional logprob data (research mode API).

pub fn process_content_logprob( &mut self, content: &str, log_prob: Option<f32>, top_alts: Vec<TokenAlternative>, )

Process a content chunk, optionally attaching logprob-derived fields to the first non-whitespace token produced.

• log_prob — natural-log probability of the leading API token, if known.
• top_alts — alternative tokens from top_logprobs, already converted to probabilities (exp(logprob)).

pub fn print_header(&self, prompt: &str)

Print a formatted session header to stdout.

Displays provider, transform, model, and prompt. When visual_mode or heatmap_mode is active, additional legend lines are printed. This method is a no-op when web_tx is set (web mode handles its own header).

pub fn print_footer(&self)

Print a summary footer to stdout after a streaming session completes.

Reports total token count and how many tokens were transformed.