Expand description
ML framework for Rust
use candle_core::{Tensor, DType, Device};
let a = Tensor::arange(0f32, 6f32, &Device::Cpu)?.reshape((2, 3))?;
let b = Tensor::arange(0f32, 12f32, &Device::Cpu)?.reshape((3, 4))?;
let c = a.matmul(&b)?;§Features
- Simple syntax (looks and like PyTorch)
- CPU and Cuda backends (and M1 support)
- Enable serverless (CPU) small and fast deployments
- Model training
- Distributed computing (NCCL).
- Models out of the box (Llama, Whisper, Falcon, …)
§FAQ
- Why Candle?
Candle stems from the need to reduce binary size in order to enable serverless possible by making the whole engine smaller than PyTorch very large library volume
And simply removing Python from production workloads. Python can really add overhead in more complex workflows and the GIL is a notorious source of headaches.
Rust is cool, and a lot of the HF ecosystem already has Rust crates safetensors and tokenizers
Re-exports§
pub use cpu_backend::CpuStorage;pub use error::Error;pub use error::Result;pub use layout::Layout;pub use shape::Shape;pub use shape::D;
Modules§
- backend
- backprop
- cpu
- cpu_
backend - display
- error
- layout
- npy
- Numpy support for tensors.
- pickle
- quantized
- safetensors
- scalar
- shape
- The shape of a tensor is a tuple with the size of each of its dimensions.
- test_
utils - utils
Macros§
Structs§
- Cuda
Device - Cuda
Storage - Strided
Index - An iterator over offset position for items of an N-dimensional arrays stored in a flat buffer using some potential strides.
- Tensor
- The core struct for manipulating tensors.
- Tensor
Id - Unique identifier for tensors.
- Var
- A variable is a wrapper around a tensor, however variables can have their content modified whereas tensors are immutable.
Enums§
- DType
- The different types of elements allowed in tensors.
- Device
- Device
Location - A
DeviceLocationrepresents a physical device whereas multipleDevicecan live on the same location (typically for cuda devices). - Storage
- Strided
Blocks