Struct GContextBuilder

pub struct GContextBuilder { /* private fields */ }

GGML context builder structure used to build a GContext.

Implementations

impl GContextBuilder

pub fn new() -> Self

Create a new GContextBuilder.

pub fn mem_size(self, mem_size: usize) -> Self

Set the GGML context size.

pub fn no_alloc(self, no_alloc: bool) -> Self

When a context is in no_alloc mode, apparently this means that data for tensors is not allocated at all so all requests will succeed (althoug the context needs to have at least enough memory for the GGML objects).

Naturally, you can’t actually execute the graph when this is turned on, so the main use is to probe how much memory building a graph will require and then use that value to create another context with the expected memory size and then populate that with the actual tensor data.

pub fn build(self) -> Result<GContext>

Build a GGML context (GContext) based on the builder’s configuration.

Trait Implementations

impl Default for GContextBuilder

fn default() -> GContextBuilder

Returns the “default value” for a type.