Struct Master 

pub struct Master<TG, IG> {
    pub ctx: Context,
    pub llm_model: Option<Box<TG>>,
    pub sd_model: Option<Box<IG>>,
}

A master connects to, communicates with and orchestrates the workers.

Fields

ctx: Context

llm_model: Option<Box<TG>>

sd_model: Option<Box<IG>>

Implementations

impl<TG: TextGenerator + Send + Sync + 'static, IG: ImageGenerator + Send + Sync + 'static> Master<TG, IG>

pub async fn new(ctx: Context) -> Result<Self>

Create a new instance.

pub async fn run(self) -> Result<()>

pub fn reset(&mut self) -> Result<()>

Reset the master state for a new inference.

pub async fn goodbye(&mut self) -> Result<()>

clear worker kv cache

pub async fn generate_text<S>( &mut self, max_tokens: Option<usize>, stream: S, ) -> Result<()>

where S: FnMut(&str),

Start the generation loop and call the stream function for every token. max_tokens overrides the default sample length if provided.

pub async fn generate_image<F>( &mut self, args: ImageGenerationArgs, callback: F, ) -> Result<()>

where F: FnMut(Vec<ImageBuffer<Rgb<u8>, Vec<u8>>>) + Send + 'static,