Skip to main content

Module concurrent

Module concurrent 

Source
Expand description

Concurrent model execution support.

This module provides infrastructure for running multiple model requests concurrently across different threads, allowing the TUI to remain responsive while models are generating responses.

Architecture:

  • Each model request runs in its own spawned task
  • Tasks send frames back via an mpsc channel
  • The main loop selects between client messages and model responses
  • Thread switching is allowed while models are running

Structs§

ActiveTasks
Tracks active model tasks per thread.
ChannelSink
A sink that sends WebSocket messages through a channel.

Enums§

ModelTaskMessage
A message from a spawned model task back to the main connection handler.

Functions§

channel
Create a new model task channel.

Type Aliases§

ModelTaskRx
Receiver for model task messages.
ModelTaskTx
Sender for model task messages.