Skip to main content

Module queue

oxillama_server::threads

Module queue 

Source
Expand description

Unbounded MPSC work queue for the Assistants API run processor.

Route handlers submit RunWorkItems through the RunQueueSender when a new run is created. The spawn_run_worker background task drains RunQueueReceiver and executes each run against the inference engine.

Structs§

RunQueueReceiver
Receiver half of the run work queue.
RunQueueSender
Sender half of the run work queue.
RunWorkItem
A single item submitted to the run worker queue.

Functions§

new_run_queue
Create an unbounded run work queue.