every-other-token 4.3.1

every-other-token is a free LLM token stream viewer and interceptor for the command line and the browser. It sits on the live OpenAI or Anthropic stream, shows the confidence and perplexity of each token from the logprobs, and can rewrite every other token (or any fraction) as it arrives. A built-in mock provider lets you try all of it with no API key.

Who it's for: anyone curious how language models pick their words, plus people doing LLM interpretability research, red-teaming and prompt engineering.

How it works

1. Intercept. It opens the model's streaming connection (SSE) itself, so it sees each chunk the moment it arrives.
2. Score. Each token gets confidence = exp(logprob) and perplexity = exp(-logprob), plus the top alternatives the model considered.
3. Mutate. The chosen tokens (every other one by default, or only the ones the model was unsure about) go through a transform: reverse, uppercase, noise, delete, synonym and more.
4. Output. You watch it in the terminal or the web UI, or save it as JSON lines, CSV or an HTML heatmap.

Install

Linux (x86_64, Ubuntu 20.04+ / Debian 11+). One line, no dependencies, installs to ~/.local/bin:

mkdir -p ~/.local/bin && curl -fsSL https://gitlab.com/mattbusel/Every-Other-Token/-/releases/permalink/latest/downloads/every-other-token-linux-x86_64.tar.gz | tar xz --strip-components=1 -C ~/.local/bin --wildcards '*/every-other-token'

Double-click the Windows .exe and the web UI opens in your browser. Every release, with SHA-256 checksums: Releases.

Examples

All of these are real output from the mock provider (a fixed reply with fixed logprobs, run through the real pipeline), so you can reproduce them exactly without an API key.

1. Reverse every other token (the default)

$ every-other-token "Why is the sky blue?" --provider mock
The kciuq brown xof jumps revo the yzal dog. This si a kcom response rof prompt: Why si the yks blue?
24 tokens streamed, 12 transformed

2. Rewrite only the words the model was unsure about (confidence at or below 0.6)

$ every-other-token "Why is the sky blue?" uppercase --provider mock --rate 1 --min-confidence 0.6
The quick BROWN fox JUMPS over the LAZY dog. THIS is a mock response for PROMPT: WHY IS THE SKY BLUE?
24 tokens streamed, 11 transformed

"brown" (0.46), "jumps" (0.57), "lazy" (0.41), "This" (0.54) and "prompt" (0.59) were below the line. The echoed prompt at the end carries no logprobs in the mock, so it falls back to the plain rate.

3. Get the numbers as JSON, one line per token

$ every-other-token "Why is the sky blue?" uppercase --provider mock --json-stream
{"text":"The","original":"The","index":0,"transformed":false,"importance":0.8869204521179199,"confidence":0.88692045,"perplexity":1.1274968,"is_error":false,"arrival_ms":0}
{"text":" QUICK","original":" quick","index":1,"transformed":true,"importance":0.6376281380653381,"confidence":0.63762814,"perplexity":1.5683122,"is_error":false,"arrival_ms":0}
...

4. Watch it in the browser with every-other-token --web --provider mock (or just double-click the .exe). Split view: the original stream on the left, the rewritten one on the right, each token underlined by its confidence.

Use it in 3 steps

1. Get it. Use the Linux one-liner or the Windows .exe above (or cargo install).
2. Try it offline. Run every-other-token "Why is the sky blue?" --provider mock, or double-click the .exe and pick Mock (no API key) in the provider menu.
3. Point it at a real model. Set OPENAI_API_KEY or ANTHROPIC_API_KEY, then run every-other-token "Why is the sky blue?" --visual (terminal, colored by confidence) or every-other-token --web (browser). Real confidence numbers come from OpenAI logprobs; Anthropic's stream has none, so there the tool uses its own importance score.

Run every-other-token --help for every flag, with examples at the bottom.

Documentation

License

MIT, see LICENSE.

Hire the author

Need this kind of engineering on your product? I take on a small number of client builds: LLM features, iOS apps and performance work, fixed price. Services and pricing · Email · LinkedIn