Module holdout 

Deterministic output-savings holdout (#895 Track B).

To honestly measure how much our output-shaping (cache-safe effort control #834 + the optional verbosity steer) reduces output tokens, we A/B it against a control arm. The cohort is a pure function of conversation identity (system prompt + first user message), so:

• the SAME conversation is always in the SAME arm — every turn — which keeps the decision (and therefore the request body) stable across turns, and
• a fraction f of conversations land in the control arm, which is metered but NOT output-shaped, giving a clean paired comparison of average output tokens per arm.

output_holdout = 0 (default) puts everyone in Treatment → no behaviour change. The hash is content-addressed (blake3), not random, so it is stable across processes and machines.

The same machinery runs a second, independent experiment for the proxy’s input compression (#1905, compression_arm), whose control arm is sent uncompressed.

Structs

Cohorts

The holdout arms one request belongs to; None where that holdout is off.

Enums

Arm

Which experiment arm a conversation is assigned to.

Functions

anthropic_key

Cohort key for an Anthropic /v1/messages body: system + first user turn.

assign

Assign an Arm for key given the control fraction holdout in [0,1]. holdout <= 0 ⇒ everyone is Arm::Treatment (no control arm).

assign_holdout_by_task

Deterministic holdout assignment based on task_id. Returns true if the task should be held out (assigned to baseline/control group).

bucket

Deterministic bucket 0..BUCKETS from conversation key material.

compression_arm

Arm for the input-compression holdout (#1905). The control arm forwards the request without any of the proxy’s input compression, giving a real uncompressed baseline instead of the simulated one Shadow Mode reports.

compression_holdout_arm

Input-compression holdout arm (#1905) for a parsed request body, or None when [proxy] compression_holdout is off. The forward path calls this once, on the caller’s pristine body before any stage rewrites it, so every turn of a conversation lands in the same arm.

conversation_key

Conversation key for any supported request shape, for holdouts decided before the provider is known (#1905). Gemini bodies carry contents, Responses-API bodies input, Anthropic bodies a top-level system; the rest are Chat Completions.

google_key

Cohort key for a Gemini generateContent body: systemInstruction + first user contents turn.

openai_chat_key

Cohort key for an OpenAI Chat Completions body: first system/developer message + first user message.

openai_responses_key

Cohort key for an OpenAI Responses body: instructions + first user input.