Module openai 

Constants

ANNOTATION_REQUEST_ID

Dynamo Annotation for the request ID

Functions

completion_response_stream

OpenAI Completions Request Handler

grpc_monitor_for_disconnects

This method will consume an AsyncEngineStream and monitor for disconnects or context cancellation. This is gRPC variant of monitor_for_disconnects as that implementation has SSE specific handling. Should decouple and reuse monitor_for_disconnects