Struct ModelClient 

pub struct ModelClient { /* private fields */ }

Client for a ModelService server’s Join bidi stream.

Concurrency: demux by request_id

Responses are demultiplexed by request_id through a shared pending map, so multiple requests can be in flight on one connection at once. A response pump routes each response to the matching waiter; a response with no pending waiter (a late one from an abandoned request, or an unknown id) is logged and dropped.

The public per-request methods (predict, configure_route, close_route, close) take &mut self and await their own response, so used alone they behave exactly as before (one request at a time, response matched by id). To actually overlap predicts on one connection, use predict_concurrent, which takes &self and may be called from multiple tasks concurrently. The matching server advertises the rlmesh.model.concurrent_predict.v1 capability when it pipelines.

Implementations

impl ModelClient

pub async fn connect(address: &str, token: &str) -> Result<Self, GrpcError>

pub async fn connect_with_retry( address: &str, token: &str, options: &ConnectOptions, ) -> Result<Self, GrpcError>

Connect to a ModelService server, retrying until the server accepts the connection (or the deadline/cancellation in options fires).

Only the transport connect is retried; perform the handshake explicitly on the returned client.

pub fn address(&self) -> &str

pub fn take_last_telemetry(&mut self) -> Option<OperationTelemetry>

pub fn server_pipelines_predict(&self) -> bool

Whether the server advertised that it pipelines Join-stream predicts (rlmesh.model.concurrent_predict.v1). Advisory: overlapping predicts via predict_concurrent work either way, but serialize behind the handler when this is false.

pub async fn handshake(&mut self) -> Result<(), GrpcError>

pub async fn configure_route( &mut self, request: ConfigureRouteRequest, ) -> Result<(), GrpcError>

pub async fn predict( &mut self, request: PredictRequest, ) -> Result<PredictResponse, GrpcError>

pub async fn close_route( &mut self, request: CloseRouteRequest, ) -> Result<(), GrpcError>

pub async fn close( &mut self, reason: impl Into<String>, ) -> Result<(), GrpcError>

pub async fn close_with_timeout( &mut self, reason: impl Into<String>, timeout: Duration, ) -> Result<(), GrpcError>

pub async fn shutdown( &mut self, reason: impl Into<String>, ) -> Result<ShutdownResponse, GrpcError>

pub async fn predict_concurrent( &self, request: PredictRequest, ) -> Result<PredictResponse, GrpcError>

Issue a predict that may overlap other in-flight requests on the same connection. Takes &self, so it can be called from multiple tasks concurrently; responses are demuxed by request_id.

Unlike predict, this does not record last_telemetry (that field is single-threaded &mut self state); read per-call telemetry from the returned response if needed. The server only pipelines these when it advertises rlmesh.model.concurrent_predict.v1; against a serial server they still complete correctly, just serialized.