kopitiam_loader/lib.rs
1//! Kopitiam Runtime: native model file loaders.
2//!
3//! This crate turns a GGUF or SafeTensors file on disk into a
4//! format-agnostic description of what it contains: model-level
5//! hyperparameters ([`ModelMetadata`]) and a directory of tensors, each
6//! described by name, [`kopitiam_core::DType`], [`kopitiam_core::Shape`],
7//! and its raw on-disk bytes ([`TensorEntry`], served through
8//! [`LoadedModel::tensor_bytes`]).
9//!
10//! # Why this crate never constructs a `Tensor`
11//!
12//! `kopitiam-tensor` — the crate that owns the `Tensor` type — is being
13//! developed independently of this one. Depending on it here would couple
14//! two crates whose APIs are each still settling, for a feature this crate
15//! does not need: nothing a loader does requires an actual `Tensor`, only
16//! the bytes, dtype and shape needed to build one. So this crate stops one
17//! step short: [`LoadedModel::tensor_bytes`] hands back exactly those
18//! bytes, and whoever *does* depend on `kopitiam-tensor` combines them with
19//! the dtype and shape from [`LoadedModel::tensor`] to build the tensor on
20//! their side of the boundary. Neither crate has to know the other's
21//! internals for this to work.
22//!
23//! # Supported formats
24//!
25//! * **GGUF** ([`GgufLoader`], module `gguf`) — the `llama.cpp`/`ggml`
26//! format. Note its on-disk dimension order is the *reverse* of this
27//! crate's [`kopitiam_core::Shape`] convention; see the `gguf` module
28//! docs before touching anything shape-related there.
29//! * **SafeTensors** ([`SafeTensorsLoader`], module `safetensors`) —
30//! Hugging Face's format. Its dimension order already matches
31//! `Shape`'s convention directly.
32//!
33//! [`load_model`] picks between the two by sniffing file content (GGUF has
34//! a magic number; SafeTensors does not, so it is the fallback), so most
35//! callers only need that one function.
36//!
37//! # Memory strategy
38//!
39//! Model files are memory-mapped where possible rather than read fully
40//! into a `Vec<u8>` — see the internal `byte_source` module's doc for why
41//! that matters at multi-gigabyte model sizes and when this crate falls
42//! back to a plain read instead.
43//!
44//! # Malformed input
45//!
46//! Every parser in this crate treats its input as untrusted: truncated
47//! files, offsets that point past end-of-file, and absurd counts that
48//! would otherwise justify a huge allocation all become
49//! [`kopitiam_core::Error::MalformedModel`] rather than a panic. A tensor
50//! type or dtype this crate cannot represent becomes
51//! [`kopitiam_core::Error::UnsupportedModelFeature`] instead of being
52//! silently misdecoded as something else.
53
54mod byte_source;
55mod gguf;
56mod metadata;
57mod model;
58mod safetensors;
59
60pub use gguf::GgufLoader;
61pub use metadata::{GgufMetadata, GgufValue, ModelMetadata};
62pub use model::{LoadedModel, ModelLoader, TensorEntry, load_model};
63pub use safetensors::SafeTensorsLoader;
64
65// Re-exported for ergonomics: every public signature in this crate already
66// names these types (`TensorEntry::dtype`, `TensorEntry::shape`, fallible
67// methods returning `Result`), so callers otherwise need a second `use`
68// line from `kopitiam_core` just to name what this crate hands them.
69pub use kopitiam_core::{DType, Error, Result, Shape};