Expand description
Concurrent model execution support.
This module provides infrastructure for running multiple model requests concurrently across different threads, allowing the TUI to remain responsive while models are generating responses.
Architecture:
- Each model request runs in its own spawned task
- Tasks send frames back via an mpsc channel
- The main loop selects between client messages and model responses
- Thread switching is allowed while models are running
Structs§
- Active
Tasks - Tracks active model tasks per thread.
- Channel
Sink - A sink that sends WebSocket messages through a channel.
Enums§
- Model
Task Message - A message from a spawned model task back to the main connection handler.
Functions§
- channel
- Create a new model task channel.
Type Aliases§
- Model
Task Rx - Receiver for model task messages.
- Model
Task Tx - Sender for model task messages.