Expand description
The provider-neutral attempt loop every inference route runs (Go
BaseAPIHandler.Execute*WithAuthManager plus Manager.Execute*).
Model to providers through the registry, credential selection across all of them,
per-credential alias pools, request-scoped rules, retry rounds and stream bootstrap.
Routes only parse their request and render the Done or Failure.
Structs§
- Call
- One client request, already parsed by its route.
- Media
- How a media route runs through
run: which executor operation serves it, and the same credential binding aSessionTurnhas (WithPinnedAuthID,WithSelectedAuthIDCallback), which the video routes use to keep a job on the credential that created it. - RunError
- A terminal failure of
run.bootstrapmarks a stream that failed before its first payload (GostreamBootstrapError). - Session
Turn - How a session transport runs one turn through
run(GoExecuteStreamWithAuthManagerwithWithPinnedAuthID,WithSelectedAuthIDCallbackand the execution session). - SseError
- The
event: errorframe a route attaches to an error response (as a response extension), forserveto write instead when the event stream already committed (writeImagesStreamErrorEvent). - Trace
- The trace ID of the last credential selected for a request.
Enums§
Functions§
- jitter
- Go
jitteredCooldownWait: up to a quarter of the wait (at most 2s) extra, never past the configured maximum, so synchronized clients spread out. - peer
- request_
id - A UUIDv7 request ID (Go
logging.GenerateRequestID). - route_
path - Go
RequestPathMetadataKey(ginFullPath()): the matched route template in gin’s spelling (/v1beta/models/*action), or the URI path when no route matched. - run
- Runs one request through selection, execution and retry rounds (Go
Manager.Execute*). Selection failures take part in retry rounds too. - run_
with_ bootstrap_ retries - Go handlers_stream.go: a stream that failed before its first payload is retried as a whole request when the status is statusless, auth, quota, timeout or 5xx.
- serve
- Runs
calland renders the result, adding Go’sX-CPA-TRACE-ID(selection time, the selected credential’s auth index and a request ID) once a credential was picked. - upstream_
summary - Go
ExtractUpstreamErrorSummary: the upstream error’s code and message, sanitized.
Type Aliases§
- OnSelected
- The
SessionTurn::on_selectedcallback. - Peer
- The downstream peer address, as the listener records it (
ConnectInfo).