Module tensor 

Constants

ANNOTATION_REQUEST_ID

Dynamo Annotation for the request ID

Functions

grpc_monitor_for_disconnects

This method will consume an AsyncEngineStream and monitor for disconnects or context cancellation. This is gRPC variant of monitor_for_disconnects as that implementation has SSE specific handling. Should decouple and reuse monitor_for_disconnects

tensor_response_stream

Tensor Request Handler