Skip to main content

Crate oxidelm

Crate oxidelm 

Source
Expand description

A standard-library CPU tensor and reverse-mode automatic differentiation core.

Data is stored in row-major order behind shared immutable CPU storage (Rc<Vec<f32>>). Contiguous reshape, permute/transpose, slice, and narrow create views (shared storage + offset/strides). Kernels that require packed row-major data call Tensor::contiguous to materialize. Mutation paths clone-on-write when storage is shared. Element-wise binary operations support NumPy-style trailing-dimension broadcasting. Matrix multiplication supports 2D matrices and equal leading batch dimensions.

§Concurrency model

The autograd graph is single-threaded by design. Tensor and IndexTensor use Rc<RefCell<_>>, so they are neither Send nor Sync: graph construction, forward, backward, and optimizer steps that touch tensor nodes must run on one thread.

Parallelism is confined to kernels and host helpers that operate on plain buffers without sharing the graph across threads:

  • blocked CPU matmul (Rayon over output rows)
  • GPU kernel dispatch
  • tokenizer pair counting / batch encode
  • data-loader prefetch worker (token IDs only)
  • CPU data-parallel replicas (each worker owns a private graph; only weight/gradient Vec<f32> buffers cross threads)

Re-exports§

pub use grad::enable_grad;
pub use grad::is_grad_enabled;
pub use grad::no_grad;
pub use grad::set_grad_enabled;
pub use grad::with_no_grad;
pub use grad::GradModeGuard;

Modules§

act_quant
Activation quantization for mixed-precision inference (W8A8 / W4A8 / W4A16).
advanced_quant
Calibration-light and high-compression weight quantization.
amp
Mixed-precision training helpers.
analyze
Machine-consumable model analysis (oxidelm --analyze).
anomaly
Optional early detection of NaN and infinity in tensor operations.
attention
calibration
Calibration data loading for activation-aware PTQ (GPTQ / AWQ / SmoothQuant).
checkpoint
Binary, named, atomic training checkpoints.
config
Training configuration from JSON files and CLI overrides.
corpus
Training corpus discovery: single .txt, JSONL ({"text":…}), or a directory of those files concatenated in deterministic sorted order.
data_loader
Memory-mapped token shards and deterministic background batch prefetch.
data_parallel
Data-parallel training via per-thread model replicas.
generate
gguf
GGUF import/export for OxideLM checkpoints.
gpu
grad
Global gradient-tracking mode for inference and evaluation.
initialization
inspect
Offline checkpoint and model inspection without training data.
kv_quant
KV-cache quantization (int8 / FP8 / NF4) with per-head or per-layer scales.
layers
log
Structured logging with quiet, human-readable, and JSON output modes.
memory
Static FP32 memory estimates and device-budget enforcement.
model
norm
optimizer
peft
LoRA / DoRA parameter-efficient adapters for Limited-RAM training.
perf_regression
Performance regression thresholds against committed baselines.
profile
Named timing spans for profiling hot paths.
ptq
Calibration-based post-training quantization (GPTQ, AWQ, SmoothQuant).
qat
Quantization-aware training (QAT) configuration and fake-quant helpers.
quality
Standard model quality evaluation.
quantize
Production quantization surface (oxidelm --quantize).
rope
state_dict
Named state dictionaries and SafeTensors import/export.
tokenizer
Byte-level BPE training, encoding, decoding, and persistence.
tokenizer_quality
Tokenizer quality metrics on held-out text.
trainer
transformer
weight_quant
Weight-only quantization schemes for Linear packs.

Structs§

CpuMemoryStats
Logical CPU tensor payload memory tracked by the tensor engine.
IndexTensor
Row-major integer tensor for token IDs and class indices.
Tensor
Differentiable tensor node.
TensorError
Recoverable shape and argument errors from public tensor APIs.

Enums§

CpuMatmulBackend
Device

Functions§

checkpoint
Runs function without retaining intermediate activations.
checkpoint_gpu
GPU counterpart of [checkpoint].
cpu_matmul_backend
cpu_memory_stats
Returns current and peak logical CPU tensor payload bytes.
reset_cpu_peak_memory
Starts a new peak-memory measurement from the currently live tensor bytes.
set_cpu_matmul_backend