gRPC clients for vLLM, TensorRT-LLM, MLX, TokenSpeed, and SGLang backends.
This crate provides gRPC client implementations for communicating with the vLLM engine, TensorRT-LLM engine, MLX engine, TokenSpeed scheduler, and SGLang scheduler backends.
gRPC clients for vLLM, TensorRT-LLM, MLX, TokenSpeed, and SGLang backends.
This crate provides gRPC client implementations for communicating with the vLLM engine, TensorRT-LLM engine, MLX engine, TokenSpeed scheduler, and SGLang scheduler backends.