pub struct Model { /* private fields */ }Implementations§
Source§impl Model
impl Model
pub fn load(path: &PathBuf) -> Result<Self>
pub fn load_with_mmap(path: &PathBuf) -> Result<(Mmap, Self)>
Sourcepub fn prefetch_mmap(_mmap: &Mmap)
pub fn prefetch_mmap(_mmap: &Mmap)
No-op. madvise hints are now applied inside load_with_mmap() immediately
after the mmap is created and before tensor data is read, which is the only
point where they have effect. Calling this after load returns is useless.
pub fn metadata(&self) -> &GgufMetadata
pub fn clear_kv_cache(&mut self)
pub fn forward(&mut self, tokens: &[u32], pos: usize) -> Result<Tensor>
Auto Trait Implementations§
impl Freeze for Model
impl !RefUnwindSafe for Model
impl Send for Model
impl Sync for Model
impl Unpin for Model
impl UnsafeUnpin for Model
impl !UnwindSafe for Model
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
Source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more