Crate nuvec_io 

Reading and writing arrays: safetensors (MLX’s checkpoint format) and NumPy’s .npy, loaded without a copy by memory-mapping the file, and the IDX format (MNIST); a minimal json parser for model files.

Status: pre-alpha.

Modules

json

A minimal JSON parser (RFC 8259) for safetensors headers and model files (configs, tokenizers), so that no serialization framework is needed.

Structs

SafeTensors

The tensors and metadata of a safetensors file.

Constants

VERSION

The version of this crate.

Functions

load_npy

Loads a .npy file as a view of the memory-mapped file.

load_safetensors

Loads a safetensors file. The tensors are views of the memory-mapped file (or of one copy of its data, if they are not aligned to their element sizes).

parse_idx

Parses an IDX file’s bytes. Element types map to uint8, int8, int16, int32, float32 and float64.

read_idx

Reads an (uncompressed) IDX file.

save_npy

Saves a (evaluating it) as a C-order .npy file (version 1). bfloat16 has no NumPy type and is rejected.

save_safetensors

Saves tensors (evaluating them) and metadata as a safetensors file.