Crate candle_core_temp

ML framework for Rust

use candle_core::{Tensor, DType, Device};

let a = Tensor::arange(0f32, 6f32, &Device::Cpu)?.reshape((2, 3))?;
let b = Tensor::arange(0f32, 12f32, &Device::Cpu)?.reshape((3, 4))?;

let c = a.matmul(&b)?;

Features

• Simple syntax (looks and like PyTorch)
• CPU and Cuda backends (and M1 support)
• Enable serverless (CPU) small and fast deployments
• Model training
• Distributed computing (NCCL).
• Models out of the box (Llama, Whisper, Falcon, …)

FAQ

• Why Candle?

Candle stems from the need to reduce binary size in order to enable serverless possible by making the whole engine smaller than PyTorch very large library volume

And simply removing Python from production workloads. Python can really add overhead in more complex workflows and the GIL is a notorious source of headaches.

Rust is cool, and a lot of the HF ecosystem already has Rust crates safetensors and tokenizers

Re-exports

• pub use cpu_backend::CpuStorage;
• pub use error::Error;
• pub use error::Result;
• pub use layout::Layout;
• pub use shape::Shape;
• pub use shape::D;

Modules

• backend
• backprop
• cpu
• cpu_backend
• display
• error
• layout
• npy
  Numpy support for tensors.
• pickle
• quantized
• safetensors
• scalar
• shape
  The shape of a tensor is a tuple with the size of each of its dimensions.
• test_utils
• utils

Macros

• bail
• map_dtype
• test_device

Structs

• CudaDevice
• CudaStorage
• StridedIndex
  An iterator over offset position for items of an N-dimensional arrays stored in a flat buffer using some potential strides.
• Tensor
  The core struct for manipulating tensors.
• TensorId
  Unique identifier for tensors.
• Var
  A variable is a wrapper around a tensor, however variables can have their content modified whereas tensors are immutable.

Enums

• DType
  The different types of elements allowed in tensors.
• Device
• DeviceLocation
  A DeviceLocation represents a physical device whereas multiple Device can live on the same location (typically for cuda devices).
• Storage
• StridedBlocks

Traits

• CustomOp1
  Unary ops that can be defined in user-land.
• CustomOp2
• CustomOp3
• FloatDType
• IndexOp
  Trait used to implement multiple signatures for ease of use of the slicing of a tensor
• IntDType
• Module
• ModuleT
• ToUsize2
• WithDType