Struct llm_daemon::Llamafile

pub struct Llamafile { /* private fields */ }

Implementations

impl Llamafile

pub fn from(path: PathBuf, name: &str, port: u16) -> Self

Trait Implementations

impl LlmDaemon for Llamafile

fn fork_daemon(&self) -> Result<()>

Spawns the daemon, initializing any necessary resources or processes. This method is expected to be called before creation of tokio runtime, mostly due to the use of the fork. User is free to use async runtime after calling this.

type Config = Config

fn config(&self) -> &Self::Config

fn heartbeat<'a, 'b>(&'b self) -> impl Future<Output = Result<()>> + Send + 'a

where 'a: 'b,

Creates a task which maintains a periodic heartbeat to the daemon. Daemon is expected to terminate if there’s no heartbeat for a certain period of time. Keeping this task within async runtime will ensure that the daemon is kept running during the application.

fn ready<'a>(&self) -> impl Future<Output = ()> + Send + 'a