unsloth_rs/lib.rs
1// SPDX-License-Identifier: MIT
2// Copyright 2026 Tyler Zervas
3
4//! # unsloth-rs
5//!
6//! Rust implementations of transformer building blocks for LLM inference and fine-tuning.
7//!
8//! ## What This Crate Provides
9//!
10//! This crate provides common transformer operations built on [Candle](https://github.com/huggingface/candle):
11//!
12//! - **Multi-head attention**: Core attention mechanism with grouped-query attention (GQA) support
13//! - **Rotary position embeddings (`RoPE`)**: Position encoding used in modern LLMs
14//! - **RMS normalization**: Efficient normalization layer used in LLaMA-style models
15//! - **`SwiGLU` activation**: Gated activation function for transformer MLPs
16//! - **Memory estimation utilities**: Tools for tracking and estimating memory usage
17//!
18//! ## Why This Crate?
19//!
20//! This crate provides a Rust-native implementation of transformer components,
21//! offering type safety and memory safety guarantees. The implementations are
22//! designed to be clear and maintainable, serving as reference implementations
23//! that can be extended with optimized GPU kernels.
24//!
25//! ## Current Status
26//!
27//! Current implementations are CPU reference implementations with GPU dispatch
28//! via Candle's CUDA backend. Fused GPU kernels using `CubeCL` are planned for
29//! future versions.
30//!
31//! ## Quick Start
32//!
33//! ```rust,ignore
34//! use unsloth_rs::kernels::{FusedAttention, FusedAttentionConfig};
35//! use candle_core::Device;
36//!
37//! let config = FusedAttentionConfig {
38//! hidden_size: 768,
39//! num_heads: 12,
40//! head_dim: 64,
41//! ..Default::default()
42//! };
43//! let attention = FusedAttention::new(config, &Device::Cpu)?;
44//! ```
45
46#![warn(missing_docs)]
47#![warn(clippy::pedantic)]
48#![allow(clippy::doc_markdown)]
49#![allow(clippy::missing_errors_doc)]
50#![allow(clippy::missing_panics_doc)]
51#![allow(clippy::cast_precision_loss)]
52#![allow(clippy::cast_possible_truncation)]
53#![allow(clippy::cast_sign_loss)]
54#![allow(clippy::implicit_hasher)]
55#![allow(clippy::needless_pass_by_value)]
56#![allow(clippy::uninlined_format_args)]
57#![allow(clippy::struct_field_names)]
58#![allow(clippy::doc_overindented_list_items)]
59#![allow(clippy::module_name_repetitions)]
60#![allow(clippy::manual_div_ceil)]
61#![allow(clippy::type_complexity)]
62
63pub mod error;
64pub mod kernels;
65pub mod memory;
66pub mod training;
67
68pub use error::{Result, UnslothError};