Trait llm_daemon::LlmDaemon

source · 
pub trait LlmDaemon {
    type Config: LlmConfig;

    // Required methods
    fn fork_daemon(&self) -> Result<()>;
    fn heartbeat(&self) -> impl Future<Output = Result<()>> + Send + 'static;
}
Expand description

Represents a generic daemon capable of performing background tasks, including spawning itself, maintaining a heartbeat, and generating responses based on prompts.

Required Associated Types§

source

type Config: LlmConfig

Required Methods§

source

fn fork_daemon(&self) -> Result<()>

Spawns the daemon, initializing any necessary resources or processes. This method is expected to be called before creation of tokio runtime, mostly due to the use of the fork. User is free to use async runtime after calling this.

source

fn heartbeat(&self) -> impl Future<Output = Result<()>> + Send + 'static

Creates a task which maintains a periodic heartbeat to the daemon. Daemon is expected to terminate if there’s no heartbeat for a certain period of time. Keeping this task within async runtime will ensure that the daemon is kept running during the application.

Object Safety§

This trait is not object safe.

Implementors§