Skip to main content

Module async_session

Module async_session 

Source
Expand description

Async inference API for non-blocking, high-throughput inference.

This module provides async/await support for RONN inference sessions, enabling efficient concurrent request handling for production workloads.

Structsยง

AsyncBatchProcessor
Batch processor for async inference.
AsyncSession
Async wrapper for inference sessions.