Crate kopitiam_tensor 

Kopitiam Runtime: tensor.

The CPU tensor type and the inference-time op set (matmul, broadcasting arithmetic, softmax, reductions, normalization, activations, embedding gather) that every layer of a transformer forward pass is built from, plus decoders for the f16/bf16/GGUF-quantized formats model weights ship in.

Scope: Phase 1, inference only

This crate has no autograd, no gradient tape, and no training ops. That is a scope decision, not an oversight: the Kopitiam Runtime’s stated purpose (see docs/ai-decisions/) is local inference, and every op here is chosen because a forward pass needs it. Concretely, this means:

• Every general-purpose op (arithmetic, softmax, reductions, normalization, activations, Tensor::matmul) works on f32 and rejects other dtypes with Error::DTypeMismatch (see Tensor::require_dtype). The one exception is Tensor::quantized_matmul, which computes directly on Q4_0/Q8_0 weight blocks — see that method’s docs for why a fused quantized matmul earns an exception to “everything is f32” (memory: a dequantized 7B Q4_0 model is ~28GB of f32 vs ~4GB quantized) and for the correctness gate that keeps it honest against the plain to_dtype(F32) + Tensor::matmul reference path, which remains this crate’s default and stays permanently, both as the general fallback for every other op and as that oracle.
• There is no backward pass, so there is nothing here resembling a requires_grad flag or a computation graph — Tensor is a plain value type.
• Tensor::gather_rows implements embedding lookup specifically (indices select whole rows), not PyTorch’s general per-element gather; see that method’s docs for why the general form is out of scope.
• There is still no general f32 -> quantized encoder (no Tensor::to_dtype(DType::Q4_0)) — see Tensor::to_dtype’s docs for why requantizing weights is a model-export concern out of this crate’s scope. The quantized module’s quantize_row_q8_0 is a narrower thing: an activation-only Q8_0 encoder, private to Tensor::quantized_matmul’s implementation, not a public Tensor -> Tensor conversion.

Layering

This crate depends only on kopitiam-core for the shared vocabulary (DType, Shape, Device, Error) and re-exports it so downstream crates (kopitiam-loader, kopitiam-runtime) need not add a direct kopitiam-core dependency just to name a dtype:

kopitiam-runtime -> { kopitiam-loader, kopitiam-tokenizer } -> kopitiam-tensor -> kopitiam-core

Module map

• Tensor — the shape + strided-view + shared-storage tensor type, and every op, split across tensor/*.rs by concern (see that module’s docs for why the split is by Rust module rather than by flat file, and how that lets internals stay module-private instead of crate-wide pub(crate)).
• Storage — the owned CPU buffer Tensor is a view into.
• half — f16/bf16 <-> f32 conversion (re-exported as free functions; useful on their own, e.g. for a loader converting a whole weight file up front).
• quant (private) — GGUF block-quantized format decoders, used only through Tensor::to_dtype.

Structs

Shape

The dimensions of a tensor, outermost first.

Tensor

A CPU tensor: a Shape plus a strided view into a shared, ref-counted Storage buffer.

Enums

DType

The element type of a tensor.

Device

Where a tensor’s storage lives and where kernels run on it.

Error

Every way a Kopitiam Runtime operation can fail.

Storage

An owned CPU buffer of tensor elements.

Functions

bf16_to_f32

Decodes a bfloat16 bit pattern to f32.

f16_to_f32

Decodes an IEEE 754 binary16 bit pattern to f32.

f32_to_bf16

Encodes an f32 as the nearest bfloat16 value, rounding ties to even.

f32_to_f16

Encodes an f32 as the nearest IEEE 754 binary16 value, rounding ties to even (the IEEE default), with overflow saturating to infinity and underflow flushing to a signed zero or subnormal.

Type Aliases

Result

The runtime’s result alias.