Expand description
Utilities for interacting with LLMs and loading them.
Structs§
- Token
Utf8 Buffer - Used to buffer incoming tokens until they produce a valid string of UTF-8 text.
Enums§
- Find
AllModel Files Error - Errors encountered during the loading process.
Functions§
- find_
all_ model_ files - Find all the files related to a model.
- has_
data_ left - Check if there is any data left in the reader.
- mmap_
populate - mmap with MAP_POPULATE
- read_
bytes - Read a fixed-size array of bytes from a reader.
- read_
bytes_ with_ len - Read a variable-length array of bytes from a reader.
- read_
f32 - Read a
f32from a reader. - read_
i32 - Read a
i32from a reader. - read_
u32 - Read a
u32from a reader. - write_
f32 - Write a
f32from a writer. - write_
i32 - Write a
i32from a writer. - write_
u32 - Write a
u32from a writer.