Crate openrouter_batch

Expand description

OpenRouter normalization Batch API and synchronous embedding client.

BatchTransport owns normalization submission and polling. EmbeddingTransport owns synchronous array embedding requests.

API rules encoded here, verified against the live API on 2026-08-29 (.agents/skills/openrouter-api/ and the batch quickstart):

One batch carries one endpoint, one batch-level model, and one inline requests array. The body must serialize endpoint and model before requests: the API stream-parses and answers 400 otherwise. SubmitBatch fixes that order by field declaration; a test pins it.
Success is any 2xx with a parseable batch body: submit answers 202 Accepted with status: "validating" — queued, not complete.
Terminal statuses are completed, failed, expired, cancelled. Results arrive inline in the poll response; there is no download endpoint. A completed batch can still contain per-request failures: exactly one of response or error is populated per result line.
Result lines decode per line: a line that breaks the documented envelope arrives as BatchResultLine::Undecodable with the raw JSON preserved, so one bad line never fails a completed batch. The result-side helpers are BatchResultLine::chat_content and embedding_vectors.
The batch-level model must be a :batch variant slug; a base slug is rejected synchronously and fails the whole submission.
Server-side retention is 30 days: callers persist every result into Postgres at poll time.

The API root is configuration, not a constant (context/05): the default is the public endpoint; e2e points it at the local fake batch server.

Structs§

BatchItem: One request inside a batch: a deterministic join key plus the endpoint body. custom_id must be unique within the batch.
BatchObject: The batch object returned by both submit (202, validating) and poll.
BatchUsage: Batch usage. cost is dollars as a plain f64: the API emits scientific-notation floats (1.33e-05), so fixed-point types are wrong.
EmbeddingRequest: One synchronous embedding request with array input.
HttpTransport
Message: One chat-completions message.
RawChatResponse: One synchronous chat-completions response: the provider payload exactly as received, plus the request ID header when present. The eval harness decodes it with chat_completion_content — the same traversal the batch result lines use.
RawEmbeddingResponse: A successful HTTP response before typed result validation.
RequestCounts: request_counts.completed + failed == total is a completeness sanity check, not a success signal: failed requests stay in-band as result lines.
ResultLine: The documented result envelope: exactly one of response or error is populated.
ResultResponse: The per-request success envelope: an HTTP status plus the endpoint’s response body (chat.completion or embedding list).
StructuredParams: Parameters shared by every request in one structured batch. One schema per batch is mandatory on Google-backed routes (quickstart), so the schema lives here, at batch level, not per item.
SubmitBatch: A batch submission.

BatchEndpoint: The API shape every request in one normalization batch uses.
BatchLineError: A per-line failure of one completed request. The Display strings are the durable last_error texts operators see in Postgres.
BatchResultLine: One completed request’s result line, decoded per line: a line that does not match the documented envelope arrives as Undecodable with the raw JSON preserved, so one malformed line never fails a completed batch and callers can persist it and mark the matched request retryable (context/05). Unknown fields on a parsed line survive a decode/encode round trip (pinned by the decode tests).
BatchStatus: Batch status lifecycle: validating -> in_progress -> finalizing -> completed, with failed, expired, and cancelled as the other terminal outcomes.
EmbeddingDecodeError: A decode failure of one stored embedding payload. The Display strings are the durable last_error texts operators see in Postgres.
TransportError: Typed transport failure. Retry classification lives here so every caller shares one policy (context/11): timeouts, connect failures, 408/429, and 5xx are retryable; other 4xx and decode errors are not.

BatchTransport: The transport seam: submit and poll, nothing else. Implementations must treat any 2xx with a parseable batch body as success (submit answers 202).
EmbeddingTransport: The synchronous embedding I/O seam.

chat_completion_content: The assistant message content of one chat-completions response body. Shared by the batch result decode and the synchronous client, so the two paths cannot drift on where the content lives.
embedding_vectors: Decode one stored raw embedding payload and return the vectors ordered to match the request inputs: vectors[i] belongs to input i. Validates the envelope, the count, the index range, duplicates, and completeness.
structured_body: One structured chat-completions request body: the exact body the Batch API wraps per item and the synchronous chat-completions endpoint accepts. Shared by both paths so their wire shapes cannot drift.
submit_structured: Submit one structured chat-completions batch: every item carries the same response_format json_schema with strict: true.