Trait LlmDaemon 

pub trait LlmDaemon {
    type Config: LlmConfig;

    // Required methods
    fn config(&self) -> &Self::Config;
    fn fork_daemon(&self) -> Result<()>;
    fn heartbeat<'a, 'b>(
        &'b self,
    ) -> impl Future<Output = Result<()>> + Send + 'a
       where 'a: 'b;
    fn ping(&self) -> Result<()>;

    // Provided method
    fn ready<'a>(&self) -> impl Future<Output = ()> + Send + 'a { ... }
}

Represents a generic daemon capable of performing background tasks, including spawning itself, maintaining a heartbeat, and generating responses based on prompts.

Required Associated Types

type Config: LlmConfig

Required Methods

fn config(&self) -> &Self::Config

fn fork_daemon(&self) -> Result<()>

Spawns the daemon, initializing any necessary resources or processes. This method is expected to be called before creation of tokio runtime, mostly due to the use of the fork. User is free to use async runtime after calling this.

fn heartbeat<'a, 'b>(&'b self) -> impl Future<Output = Result<()>> + Send + 'a

where 'a: 'b,

Creates a task which maintains a periodic heartbeat to the daemon. Daemon is expected to terminate if there’s no heartbeat for a certain period of time. Keeping this task within async runtime will ensure that the daemon is kept running during the application. FIXME: Extract heartbeat / ready to Async util?

fn ping(&self) -> Result<()>

Send a single heartbeat to let daemon know there is a client. Can be useful if you don’t want to use heartbeat.

Provided Methods

fn ready<'a>(&self) -> impl Future<Output = ()> + Send + 'a

FIXME: Extract heartbeat / ready to Async util?

Dyn Compatibility

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety", so this trait is not object safe.

Implementors

impl LlmDaemon for llm_daemon::Daemon2

type Config = LlamaConfig

impl LlmDaemon for Daemon3

type Config = LlamaConfig

impl LlmDaemon for llm_daemon::LlamaDaemon

type Config = LlamaConfig

impl<D: LlmDaemon + Debug> LlmDaemon for Proxy<D>

type Config = ProxyConfig