Struct GContext

pub struct GContext { /* private fields */ }

Implementations

impl GContext

pub fn estimate_tensor_size<const DIMS: usize>( &self, typ: GType, shape: [usize; DIMS], ) -> Result<GMemoryRequest>

pub fn tensor<const DIMS: usize>( &self, typ: GType, shape: [usize; DIMS], ) -> Result<GTensor<DIMS>>

where Dim<DIMS>: DimValid, DimPair<DIMS, 4>: DimLt,

Create a new tensor with the specified type and shape.

This uses const generics to determine the new tensor’s dimensions. The tensor dimensions will be equal to the number of items in the shape array.

pub fn register_scratch_buffer(&mut self, buf: ScratchBuffer) -> Result<usize>

Register a scratch buffer. The return value is the scratch buffer id which can be used with Self::set_scratch_buffer.

pub fn set_scratch_buffer(&self, maybebufid: Option<usize>) -> Result<()>

Set or clear the current scratch buffer. A valid id as returned by Self::register_scratch_buffer must be supplied.

Note: Scratch buffers cannot be removed directly and are only freed when the GContext structure is dropped.

pub fn compute(&self, graph: &mut GGraph) -> Result<()>

Runs the supplied graph using this context.

pub fn used_mem(&self) -> Result<usize>

Returns the amount of memory GGML is currently using.

Trait Implementations

impl Clone for GContext

fn clone(&self) -> GContext

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.