Module holdout 

Deterministic output-savings holdout (#895 Track B).

To honestly measure how much our output-shaping (cache-safe effort control #834 + the optional verbosity steer) reduces output tokens, we A/B it against a control arm. The cohort is a pure function of conversation identity (system prompt + first user message), so:

• the SAME conversation is always in the SAME arm — every turn — which keeps the decision (and therefore the request body) stable across turns, and
• a fraction f of conversations land in the control arm, which is metered but NOT output-shaped, giving a clean paired comparison of average output tokens per arm.

output_holdout = 0 (default) puts everyone in Treatment → no behaviour change. The hash is content-addressed (blake3), not random, so it is stable across processes and machines.

Enums

Arm

Which experiment arm a conversation is assigned to.

Functions

anthropic_key

Cohort key for an Anthropic /v1/messages body: system + first user turn.

assign

Assign an Arm for key given the control fraction holdout in [0,1]. holdout <= 0 ⇒ everyone is Arm::Treatment (no control arm).

bucket

Deterministic bucket 0..BUCKETS from conversation key material.

google_key

Cohort key for a Gemini generateContent body: systemInstruction + first user contents turn.

openai_chat_key

Cohort key for an OpenAI Chat Completions body: first system/developer message + first user message.

openai_responses_key

Cohort key for an OpenAI Responses body: instructions + first user input.