Enum WorkerMsg 

pub enum WorkerMsg {
    Execute(ExecuteBatch, Sender<Result<TokenChunk, InferenceError>>),
    ContextPoisoned(String),
    RebuildSession {
        cause: SessionRebuildCause,
        reply: Sender<Result<(), InferenceError>>,
    },
}

Variants

Execute(ExecuteBatch, Sender<Result<TokenChunk, InferenceError>>)

ContextPoisoned(String)

Forwarded from the runner when a sticky CUDA error is detected. Triggers a child restart.

RebuildSession

Operator-triggered rebuild.

Fields

cause: SessionRebuildCause

reply: Sender<Result<(), InferenceError>>