Crate pictor 

Pictor

Pure Rust 1-bit LLM inference engine for PrismML Bonsai models.

Pictor is a high-performance inference engine designed for 1-bit quantized large language models in GGUF format. It provides a complete pipeline from model loading through token generation, with optional RAG, tokenization, evaluation, and HTTP serving capabilities.

Quick Start

use pictor::core::GgufStreamParser;

// Parse a GGUF model file via the streaming parser
let _parser = GgufStreamParser::new();

Crate Organization

Crate	Description
pictor-core	GGUF loader, tensor types, quantization, configuration
pictor-kernels	Optimized compute kernels (SIMD, matmul, softmax)
pictor-model	Transformer model definitions, KV cache, attention
pictor-runtime	Inference engine, sampling, speculative decoding
pictor-tokenizer	HuggingFace tokenizer integration
pictor-rag	Retrieval-augmented generation pipeline
pictor-eval	Model evaluation and benchmarking
pictor-serve	OpenAI-compatible HTTP server

Feature Flags

Feature	Description
server	HTTP server support via pictor-serve
rag	Retrieval-augmented generation
native-tokenizer	HuggingFace tokenizer support
eval	Model evaluation framework
full	Enable all optional features
simd-avx2	AVX2 SIMD kernels (x86_64)
simd-avx512	AVX-512 SIMD kernels (x86_64)
simd-neon	NEON SIMD kernels (AArch64)
wasm	WebAssembly target support

License

Apache-2.0 — derived from oxibonsai (COOLJAPAN OU). See LICENSE and NOTICE.

Re-exports

pub use pictor_core as core;

pub use pictor_kernels as kernels;

pub use pictor_model as model;

pub use pictor_runtime as runtime;