Struct Tensor 

pub struct Tensor { /* private fields */ }

A lazy tensor that builds a fuel_graph::Graph as its methods are called. Cheap to clone — the underlying fuel_graph::NodeHandle is a cheap handle pair (Rc<RefCell<Graph>>, NodeId), so cloning just bumps the Rc and copies the id.

Tensors are graph-affine — read this before building anything

Every tensor in one computation must descend from a common root. Each from_* constructor (from_f32, from_bf16, …) — and zeros / full, which delegate to them — mints a brand-new graph. Two independently-constructed tensors therefore live on two different graphs and cannot be combined: every binary op asserts "… must live on the same graph".

So the obvious shape does not work:

let a = Tensor::from_f32(a_data, [2, 3], &device);
let w = Tensor::from_f32(w_data, [3, 2], &device);  // ← a SECOND graph
let y = a.matmul(&w);                                   // ← panics

There are two ways to put a second tensor on an existing graph.

1. from_*_on — pass the graph. Use this when you have the graph, or simply want the construction site to say which graph it targets:

let a = Tensor::from_f32(a_data, [2, 3], &device);          // the root
let w = Tensor::from_f32_on(a.graph(), w_data, [3, 2], &device);
let y = a.matmul(&w);                                           // ✓

2. const_*_like — pass an anchor tensor. Identical effect; use it when what you have to hand is a sibling tensor rather than the graph:

let a = Tensor::from_f32(a_data, [2, 3], &device);  // the root
let w = a.const_f32_like(w_data, [3, 2].into());        // same graph ✓
let y = a.matmul(&w);                                   // ✓

Every tensor also reports its graph_id, so if two do end up on different graphs the panic names which is which.

In a model this means the activation tensor is the root and the weights are const_*_like off it. const_bf16_like exists for precisely the mixed-precision case — f32 activations with bf16 weight matrices sharing one graph.

Implementations

impl Tensor

pub fn from_f32( data: impl Into<Arc<[f32]>>, shape: impl Into<Shape>, device: &Device, ) -> Result<Self>

Build an f32 lazy tensor from flat data, a shape, and a device.

data takes impl Into<Arc<[f32]>> so both Vec<f32> and Arc<[f32]> callers work without conversion. Pass an Arc when you already have one (e.g. model weights loaded once at startup) to avoid any copy.

Phase 7.5 G2: device selects where the realized Storage is allocated. The graph’s storage_map slot for the new node is populated and Op::Const(None) is emitted — no host-side ConstData payload rides on the graph node.

Mints a NEW graph

This starts a fresh graph. A tensor built here cannot be combined with one built by another from_* call — ops assert both operands share a graph. To add a second tensor to this one’s graph, use const_f32_like (or a sibling const_*_like). See graph affinity.

pub fn graph_id(&self) -> GraphId

This tensor’s graph identity — see fuel_graph::GraphId. Two tensors can be combined iff their graph_ids match.

pub fn graph(&self) -> &SharedGraph

The shared graph this tensor belongs to — hand it to a from_*_on constructor to build a sibling on the same graph.

pub fn from_f32_on( graph: &SharedGraph, data: impl Into<Arc<[f32]>>, shape: impl Into<Shape>, device: &Device, ) -> Result<Self>

Build an f32 lazy tensor on the graph you pass in, not a fresh one.

The non-anchor route for adding a tensor to an existing graph. from_f32 mints a NEW graph, so two from_* tensors can never be combined; this puts the new leaf on graph. const_f32_like does the same when what you have to hand is a sibling tensor rather than the graph itself.

let a = Tensor::from_f32(a_data, [2, 3], &device);          // root
let w = Tensor::from_f32_on(a.graph(), w_data, [3, 2], &device);
let y = a.matmul(&w)?;                                          // ✓

pub fn from_f64_on( graph: &SharedGraph, data: impl Into<Arc<[f64]>>, shape: impl Into<Shape>, device: &Device, ) -> Result<Self>

f64 sibling of from_f32_on.

pub fn from_bf16_on( graph: &SharedGraph, data: impl Into<Arc<[bf16]>>, shape: impl Into<Shape>, device: &Device, ) -> Result<Self>

bf16 sibling of from_f32_on — the mixed-precision shape (f32 activations, bf16 weights) is this plus from_f32_on against one shared graph.

pub fn from_f16_on( graph: &SharedGraph, data: impl Into<Arc<[f16]>>, shape: impl Into<Shape>, device: &Device, ) -> Result<Self>

f16 sibling of from_f32_on.

pub fn from_u32_on( graph: &SharedGraph, data: impl Into<Arc<[u32]>>, shape: impl Into<Shape>, device: &Device, ) -> Result<Self>

u32 sibling of from_f32_on — index tensors built straight onto the graph they will be gathered on.

pub fn from_f64( data: impl Into<Arc<[f64]>>, shape: impl Into<Shape>, device: &Device, ) -> Result<Self>

Build an f64 lazy tensor. device selects where the realized Storage is allocated.

Mints a NEW graph — to build on an existing tensor’s graph use const_f64_like. See graph affinity.

pub fn from_bf16( data: impl Into<Arc<[bf16]>>, shape: impl Into<Shape>, device: &Device, ) -> Result<Self>

Build a bf16 lazy tensor. device selects where the realized Storage is allocated.

Mints a NEW graph. For the common case — bf16 weights alongside f32 activations — build the weights with const_bf16_like off the activation tensor so both share one graph. See graph affinity.

pub fn from_f16( data: impl Into<Arc<[f16]>>, shape: impl Into<Shape>, device: &Device, ) -> Result<Self>

Build an f16 lazy tensor. device selects where the realized Storage is allocated.

pub fn from_u32( data: impl Into<Arc<[u32]>>, shape: impl Into<Shape>, device: &Device, ) -> Result<Self>

Build a u32 (index) lazy tensor. Used for gather/scatter/ index_select and similar discrete operations. device selects where the realized Storage is allocated.

pub fn const_f32_like( &self, data: impl Into<Arc<[f32]>>, shape: impl Into<Shape>, ) -> Result<Self, Error>

Build a const tensor of the same dtype and graph as self. This is the most convenient way to attach new input data to an existing computation.

Phase 7.5 G2: the realized Storage is allocated on the device derived from self’s graph (any existing slot’s device — the graph always has at least one slot-bearing leaf by the time const_*_like is called). Use Self::from_f32 with an explicit &Device when you need a const on a different device than self.

pub fn const_f16_like( &self, data: impl Into<Arc<[f16]>>, shape: impl Into<Shape>, ) -> Result<Self, Error>

Build a const f16 tensor on the same graph as self.

pub fn const_bf16_like( &self, data: impl Into<Arc<[bf16]>>, shape: impl Into<Shape>, ) -> Result<Self, Error>

Build a const bf16 tensor on the same graph as self. Used for bf16-on-device weights in the mixed-precision matmul path — activations stay f32, weight matrices live as bf16.

pub fn const_like_dtype( &self, data: &[f32], shape: impl Into<Shape>, dtype: DType, ) -> Result<Self, Error>

Build a const tensor on the same graph as self, holding data’s values encoded in dtype. Only F32 (identity copy) and BF16 (host half::bf16::from_f32 conversion) are supported today — the two activation dtypes the BF16-throughout decode seam (Phase D increment A: LlamaModel’s cached decode path) threads through small per-model host constants (norm gains, biases, the causal mask) that must track the running activation dtype instead of being hardcoded f32. Any other dtype is a caller bug (not a data-dependent runtime possibility), so it surfaces a typed error rather than silently emitting wrong bytes.

pub fn into_graph_tensor(self) -> NodeHandle

Unwrap the underlying fuel_graph::NodeHandle. Used by callers that need to drop down to the graph layer for operations the bridge does not yet expose.

pub fn graph_tensor(&self) -> &NodeHandle

Borrow the underlying fuel_graph::NodeHandle.

pub fn from_graph_tensor(t: NodeHandle) -> Self

Wrap an existing fuel_graph::NodeHandle in a Tensor. Useful when you have code that already builds a graph and want to present its outputs through this API.

pub fn shape(&self) -> Shape

The tensor’s shape.

pub fn dtype(&self) -> DType

The tensor’s dtype.

pub fn graph_handle(&self) -> &SharedGraph

This tensor’s underlying graph handle. Exposed for call sites outside this module that need to drive InferenceContext directly — e.g. a persistent cross-graph decode cache implemented in a sibling model module (lazy_deepseek2.rs’s forward_with_latent_kv_context). Most callers should prefer Self::realize_f32 and friends instead.

pub fn node_id(&self) -> NodeId

This tensor’s NodeId on its graph. See Self::graph_handle.

pub fn rank(&self) -> usize

The tensor’s rank (number of dimensions).

pub fn elem_count(&self) -> usize

Total element count.

pub fn numel(&self) -> usize

PyTorch-convention alias of Self::elem_count.

pub fn dim<D: Dim>(&self, dim: D) -> Result<usize, Error>

Size of the tensor along dimension dim. Returns a typed error rather than panicking on out-of-range — matches eager’s crate::Tensor::dim signature.

pub fn add(&self, other: &Self) -> Result<Self, Error>

Element-wise addition. Shapes and dtypes must match — mismatches surface as typed errors at build time.

pub fn sub(&self, other: &Self) -> Result<Self, Error>

Element-wise subtraction.

pub fn mul(&self, other: &Self) -> Result<Self, Error>

Element-wise multiplication.

pub fn div(&self, other: &Self) -> Result<Self, Error>

Element-wise division.

pub fn maximum(&self, other: &Self) -> Result<Self, Error>

Element-wise maximum.

pub fn minimum(&self, other: &Self) -> Result<Self, Error>

Element-wise minimum.

pub fn fmax_ieee(&self, other: &Self) -> Result<Self, Error>

IEEE-754 NaN-suppressing maximum (fmax_ieee, KISS-OPS §6.15-0001) — DISTINCT from maximum (NaN-propagating). Resolves through the spec-pinned §6.13 decomposition; does not substitute maximum. GAP-048.

pub fn fmin_ieee(&self, other: &Self) -> Result<Self, Error>

IEEE-754 NaN-suppressing minimum (fmin_ieee, KISS-OPS §6.15-0001) — DISTINCT from minimum (NaN-propagating). GAP-048.

pub fn rem_trunc(&self, other: &Self) -> Result<Self, Error>

Truncated remainder (rem_trunc, KISS-OPS §6.15-0003) — sign of the DIVIDEND (C99 fmod), DISTINCT from rem (floored, sign of the divisor). Resolves through the spec-pinned §6.13 decomposition. GAP-048.

pub fn trunc(&self) -> Self

Round toward zero (trunc, KISS-OPS §6.3 floor op; GAP-300). Fuel’s own expansion (q>=0 ? floor(q) : ceil(q)), NOT spec-pinned — its ±0/NaN/inf correctness is Fuel’s liability. Integer dtypes are identity.

pub fn eq(&self, other: &Self) -> Result<Self, Error>

Element-wise equality (self == other) producing a U8 mask: 1 where equal, 0 otherwise. Both operands must share dtype and shape. NaN follows IEEE-754 (NaN == NaN is false). The resulting tensor’s dtype is DType::U8. Non-differentiable.

pub fn ne(&self, other: &Self) -> Result<Self, Error>

Element-wise inequality (self != other) producing a U8 mask. NaN follows IEEE-754 (NaN != NaN is true → 1). Non-differentiable.

pub fn lt(&self, other: &Self) -> Result<Self, Error>

Element-wise strictly-less (self < other) producing a U8 mask. NaN-on-either-side is 0 (IEEE-754 unordered). Non-differentiable.

pub fn le(&self, other: &Self) -> Result<Self, Error>

Element-wise less-or-equal (self <= other) producing a U8 mask. NaN-on-either-side is 0. Non-differentiable.

pub fn gt(&self, other: &Self) -> Result<Self, Error>

Element-wise strictly-greater (self > other) producing a U8 mask. NaN-on-either-side is 0. Non-differentiable.

pub fn ge(&self, other: &Self) -> Result<Self, Error>

Element-wise greater-or-equal (self >= other) producing a U8 mask. NaN-on-either-side is 0. Non-differentiable. Final variant of the comparison family (eq / ne / lt / le / gt / ge).

pub fn where_cond(&self, a: &Self, b: &Self) -> Result<Self, Error>

Ternary select (typically used to consume a comparison-op mask): result[i] = if self[i] { a[i] } else { b[i] }. self is the cond mask (must be DType::Bool, GAP-168(c)); a and b share dtype + shape with self. Output dtype matches a/b, shape matches self.

Differentiable through a and b only.

pub fn broadcast_add(&self, other: &Self) -> Result<Self, Error>

Element-wise addition with auto-broadcasting.

pub fn broadcast_sub(&self, other: &Self) -> Result<Self, Error>

Element-wise subtraction with auto-broadcasting.

pub fn broadcast_mul(&self, other: &Self) -> Result<Self, Error>

Element-wise multiplication with auto-broadcasting.

pub fn broadcast_div(&self, other: &Self) -> Result<Self, Error>

Element-wise division with auto-broadcasting.

pub fn neg(&self) -> Self

Element-wise negation.

pub fn sqr(&self) -> Self

Element-wise square.

pub fn sqrt(&self) -> Self

Element-wise square root.

pub fn exp(&self) -> Self

Element-wise exponential.

pub fn log(&self) -> Self

Element-wise natural logarithm.

pub fn relu(&self) -> Self

Rectified linear unit.

pub fn silu(&self) -> Self

SiLU / Swish activation.

pub fn gelu_tanh(&self) -> Self

GELU activation (tanh approximation).

pub fn sigmoid(&self) -> Self

Logistic sigmoid.

pub fn tanh(&self) -> Self

Hyperbolic tangent.

pub fn sin(&self) -> Self

Element-wise sine.

pub fn cos(&self) -> Self

Element-wise cosine.

pub fn step(&self) -> Self

Heaviside step (1 where x > 0, else 0) — the derivative of Self::relu.

pub fn recip(&self) -> Self

Element-wise reciprocal (1 / x).

pub fn abs(&self) -> Self

Element-wise absolute value (|x|).

pub fn floor(&self) -> Self

Element-wise floor (⌊x⌋). Same dtype as input. Backward is silently zero (non-differentiable almost everywhere).

pub fn ceil(&self) -> Self

Element-wise ceiling (⌈x⌉). Same dtype as input. Backward is silently zero.

pub fn round(&self) -> Self

Element-wise round-to-nearest with banker’s rounding (round-half-to-even, IEEE 754 roundeven). Backward is silently zero. Differs from C99 round() at exact halves: 0.5 → 0, 2.5 → 2, etc.

pub fn sign(&self) -> Self

Element-wise sign (-1 / 0 / 1); sign(0) = 0 by subgradient convention. Same dtype as input. Backward is silently zero.

pub fn erf(&self) -> Self

Element-wise Gauss error function (erf(x)). Same dtype as input. Differentiable: d/dx erf(x) = (2/√π) * exp(-x²).

pub fn gelu(&self) -> Self

GELU activation, exact erf form (0.5 * x * (1 + erf(x/√2))). Distinct from Self::gelu_tanh (tanh approximation). Same dtype as input. Differentiable.

pub fn pow(&self, other: &Self) -> Result<Self, Error>

Element-wise binary power pow(self, other) (real exponent). Both operands must share dtype + shape. Distinct from Self::powi (scalar i32 exponent). Differentiable. Returns Result: dtype/shape mismatch surfaces as a typed error.

pub fn rsqrt(&self) -> Self

Element-wise reciprocal square root (1 / sqrt(x)). Same dtype as input. One op rather than sqrt(x).recip() — saves a kernel launch and matches the RMSNorm shape. Differentiable.

pub fn rem(&self, other: &Self) -> Result<Self, Error>

Element-wise remainder, PyTorch convention: a - floor(a/b) * b (sign of result matches divisor; matches torch.remainder, not C99 fmod). Differentiable through a and b. Returns Result: dtype/shape mismatch surfaces as a typed error.

pub fn flip<D: Dim>(&self, dim: D) -> Result<Self, Error>

Reverse element order along dim. Materializing op (real byte shuffle). Differentiable; backward is itself. Accepts any Dim (usize, D::Minus1, etc.).

pub fn roll<D: Dim>(&self, dim: D, shift: i64) -> Result<Self, Error>

Cyclic shift along dim by shift positions (positive → higher indices, wrapping). Differentiable; backward is roll(dim, -shift).

pub fn cumsum<D: Dim>(&self, dim: D) -> Result<Self, Error>

Running cumulative sum along dim. Same shape as input. Differentiable; backward is reverse cumsum (flip → cumsum → flip).

pub fn pad( &self, padding: Vec<(usize, usize)>, mode: PadMode, value: f64, ) -> Result<Self, Error>

Multi-dim Pad: padding[i] = (before, after) for axis i, length must equal tensor rank. Output shape: out[i] = in[i] + padding[i].0 + padding[i].1. Only Constant mode is implemented; Reflect / Replicate exist as enum stubs that error at realize time. Differentiable for Constant. Returns Result: rank mismatch surfaces as a typed error.

pub fn powi(&self, n: i32) -> Self

Element-wise integer power (x.powi(n)).

pub fn matmul(&self, other: &Self) -> Result<Self, Error>

N-D batched matrix multiply with automatic rank-2 broadcasting. Shape incompatibility (rank < 2 or contracting-dim mismatch) surfaces as a typed error at build time.

pub fn matmul_dyn_m( &self, other: &Self, row_count: DynScalar, ) -> Result<Self, Error>

Data-determined-M matmul (sparse-MoE / capacity-buffer): like Self::matmul, but computes only row_count rows of the self.shape[-2]-row capacity buffer, the rest left zeroed. row_count is a fuel_ir::DynScalar resolved at compile if input-determined, else at execute from the producer-bound SymEnv (e.g. Op::NonZeroIndices’s per-expert count). F32-only today.

pub fn qmatmul( &self, weight_bytes: &Self, quant_type: QuantType, k: usize, n: usize, ) -> Result<Self, Error>

Quantized matmul: C = self @ dequant(W_Q). See fuel_graph::NodeHandle::qmatmul for details. The weight bytes tensor must be a flat U32 const holding the raw Q-block byte stream (length = n_bytes / 4).

Dtype / rank / k / block-alignment / byte-count mismatches surface as typed errors at build time rather than panicking inside the inner fuel_graph call.

pub fn transpose(&self) -> Result<Self, Error>

Transpose the last two dims. Returns a typed error on rank < 2 rather than panicking — build-time validation surfaces a useful diagnostic.

pub fn permute<D: Dims>(&self, axes: D) -> Result<Self, Error>

Permute axes by the given ordering. Accepts any Dims implementer — (0, 2, 1), [0, 2, 1], &[0, 2, 1], etc. Validates rank match + dim bounds + duplicate check at build time.

pub fn reshape(&self, shape: impl Into<Shape>) -> Result<Self, Error>

Reshape to a new shape with matching element count. Element-count mismatch surfaces as a typed error at build time.

pub fn squeeze<D: Dim>(&self, dim: D) -> Result<Self, Error>

Drop the size-1 dimension at position dim (range 0..rank). Metadata-only view; bytes shared with self. Returns Result rather than panicking — bad dim (out of bounds or shape[dim] != 1) surfaces as a typed error.

Accepts any Dim implementer — usize, D::Minus1, D::Minus2, D::Minus(n).

pub fn broadcast_to(&self, shape: impl Into<Shape>) -> Result<Self, Error>

Broadcast to a larger shape. Shape-incompatibility surfaces as a typed error at build time.

pub fn layer_norm_affine( &self, gain: Arc<[f32]>, bias: Arc<[f32]>, eps: f64, ) -> Result<Self, Error>

Apply LayerNorm along the last dim with an affine gain · x + bias post-step. Both gain and bias are length-hidden vectors materialized fresh on self’s graph; they’re broadcast across all leading dims of the output.

Equivalent to the per-port apply_layer_norm(x, ln, hidden, eps) helpers that several ports inlined — promoted here so the call sites stop drifting.

pub fn l2_normalize<D: Dim>(&self, dim: D, eps: f64) -> Result<Self, Error>

L2-normalize along dim: x / sqrt(sum(x²) + eps). Output shape equals input shape; the normalization divisor is broadcast across dim after a keepdim reduction.

Common values: eps = 1e-12 (PyTorch default), eps = 1e-6 (some retrieval pipelines), eps = 0.0 (no epsilon — caller guarantees no all-zero rows).

pub fn repeat_interleave<D: Dim>( &self, dim: D, repeats: usize, ) -> Result<Self, Error>

Equivalent to torch.repeat_interleave(x, repeats, dim). Replaces each element along dim with repeats consecutive copies of itself, expanding that dim by a factor of repeats. Implemented via reshape+broadcast+reshape — no new graph op.

repeats == 1 is a no-op clone. repeats == 0 returns an error at build time.

pub fn slice<D: Dim>( &self, dim: D, start: usize, len: usize, ) -> Result<Self, Error>

Slice (narrow) along dim: take elements [start, start+len). Bad dim / out-of-range slice surfaces as a typed error at build time. Accepts any Dim.

pub fn concat<D: Dim>(&self, other: &Self, dim: D) -> Result<Self, Error>

Concatenate two tensors along dim. Shape mismatch or bad dim surfaces as a typed error at build time. Accepts any Dim.

pub fn add_scalar(&self, c: f64) -> Self

Add a scalar to every element.

pub fn mul_scalar(&self, c: f64) -> Self

Multiply every element by a scalar.

pub fn argmax_dim<D: Dim>(&self, dim: D) -> Result<Self, Error>

Argmax along a dim, returning a U32 tensor with the reduced dim removed. Non-differentiable. Bad dim surfaces as a typed error at build time. Accepts any Dim.

pub fn realize_u32(&self) -> Vec<u32>

Realize as a u32 (index) Vec.

Routes through the fuel_dispatch::pipelined::PipelinedExecutor like Self::realize_f32 — the legacy fuel-reference-backend executor predates U8-output ops (comparison masks feeding argmin/argmax) and rejects them.

pub fn realize_u8(&self) -> Vec<u8>

Realize as raw bytes (u8). A BYTE VIEW: reads a Bool mask (0/1 per byte) or a U8 tensor back to host (GAP-168(c)), and by design does NOT guard the root dtype — it routes through crate::pipelined_bridge::realize_one_bytes (the raw byte entry, GAP-327), so a Bool root (dtype != U8) is read as its bytes rather than rejected.

PANICS on a realize failure; for a fallible byte view use crate::pipelined_bridge::realize_one_bytes. See Self::realize_f32 for the documented-not-enforced rationale (GAP-186).

pub fn sum_all(&self) -> Self

Sum of all elements, producing a scalar.

pub fn mean_all(&self) -> Self

Arithmetic mean of all elements, producing a scalar.

pub fn max_all(&self) -> Self

Maximum of every element, producing a scalar.

pub fn min_all(&self) -> Self

Minimum of every element, producing a scalar.

pub fn sum_dim<D: Dim>(&self, dim: D) -> Result<Self, Error>

Sum along a single dimension (dim removed from output). Bad dim surfaces as a typed error at build time. Accepts any Dim.

pub fn max_dim<D: Dim>(&self, dim: D) -> Result<Self, Error>

Max along a single dimension (dim removed from output).

pub fn min_dim<D: Dim>(&self, dim: D) -> Result<Self, Error>

Min along a single dimension (dim removed from output).

pub fn clamp(&self, min: f64, max: f64) -> Self

Element-wise clamp to [min, max].

pub fn mean_dim<D: Dim>(&self, dim: D) -> Result<Self, Error>

Mean along a single dimension.

pub fn reduce_sum_to(&self, target: impl Into<Shape>) -> Self

Sum-reduce to a smaller broadcast-compatible shape. Inverse of Self::broadcast_to; reduces over any dim where the source was broadcast against the target.

pub fn reduce_max_to(&self, target: impl Into<Shape>) -> Self

Max-reduce to a smaller broadcast-compatible shape — the max-symmetric counterpart of Self::reduce_sum_to.

pub fn softmax_last_dim(&self) -> Result<Self, Error>

Softmax along the last dim. Rank-0 input surfaces as a typed error at build time rather than panicking inside fuel_graph.

pub fn nf4_matmul( &self, w_packed: &Self, absmax: &Self, block_size: usize, ) -> Self

bitsandbytes-style 4-bit NormalFloat quantized matrix multiply. See fuel_graph::NodeHandle::nf4_matmul for the full shape contract. v1 covers F32/F16/BF16 activations.

pub fn ssd_chunk_scan( &self, dt: &Self, a: &Self, b: &Self, c: &Self, chunk_size: usize, ) -> Self

Mamba-2’s State-Space Duality chunked scan (forward). See fuel_graph::NodeHandle::ssd_chunk_scan for the full shape contract. chunk_size is a GPU-parallelism granularity knob; the CPU kernel runs sequential regardless.

pub fn selective_scan( &self, delta: &Self, a: &Self, b: &Self, c: &Self, delta_softplus: bool, ) -> Self

Mamba-1’s selective state-space scan (forward). See fuel_graph::NodeHandle::selective_scan for the full shape contract. Returns just y — for the bundled (y, last_state) form needed by autoregressive resumption use Self::selective_scan_bundled.

pub fn selective_scan_bundled( &self, delta: &Self, a: &Self, b: &Self, c: &Self, delta_softplus: bool, ) -> Result<(Self, Self), Error>

Multi-output Mamba-1 SSM scan: returns (y, last_state). y matches the single-output Self::selective_scan result; last_state is the final hidden state [batch, dim, dstate] used by autoregressive callers to resume from a prefill snapshot. Both Tensors are Op::View projections of the same bundled producer Storage — realizing them in the same pass shares the bundle.

pub fn ssd_chunk_scan_bundled( &self, dt: &Self, a: &Self, b: &Self, c: &Self, chunk_size: usize, ) -> Result<(Self, Self), Error>

Multi-output Mamba-2 SSD scan: returns (y, last_state). Mirrors Self::selective_scan_bundled. last_state has shape [batch, heads, head_dim, state_dim].

pub fn nonzero_indices_bundled( &self, count_sym: SymId, ) -> Result<(Self, Self), Error>

Data-determined nonzero-index extraction — the keystone primitive for data-dependent dynamic shapes. Returns (indices, count): indices is [capacity] U32 (capacity == self.elem_count()), the first count entries being the ascending flat indices of self’s nonzero elements; count is [1] U32, the runtime nonzero count. Both are Op::View projections of one bundled producer. The executor also publishes count’s realized value into the per-pass SymEnv under count_sym, so downstream ops can consume it as a dynamic extent (the KV-cache cached_len pattern, generalized to a data-determined count). count_sym is allocated by the caller (from a fuel_ir::SymGen).

pub fn fresh_data_determined_sym(&self) -> SymId

Allocate a data-determined fuel_ir::SymId not yet used by any producer on this tensor’s graph — see fuel_graph::Graph::next_data_determined_sym. Lets a builder that emits several Self::nonzero_indices_bundled producers on one graph (per-expert MoE dispatch, stacked layers) claim non-colliding count syms without threading a fuel_ir::SymGen: each call reflects the producers already added.

pub fn causal_conv1d(&self, weight: &Self, bias: &Self, use_silu: bool) -> Self

Depthwise 1-D causal convolution + bias + optional fused SiLU — the Mamba-1 / Mamba-2 prefill convolution fusion. See fuel_graph::NodeHandle::causal_conv1d for the full shape contract (caller must left-pad x with kernel - 1 zeros).

pub fn fused_softmax_cross_entropy( &self, targets: &Self, reduction: Reduction, ignore_index: i64, ) -> Self

Fused softmax + cross-entropy with integer (class-index) targets — the standard PyTorch CE loss. See fuel_graph::NodeHandle::fused_softmax_cross_entropy for the full shape contract.

pub fn layer_norm_last_dim(&self, eps: f64) -> Result<Self, Error>

LayerNorm along the last dim with the given epsilon. Rank-0 or zero-last-dim input surfaces as a typed error at build time.

pub fn rms_norm_last_dim(&self, eps: f64) -> Result<Self, Error>

RmsNorm along the last dim (LLaMA’s normalization). Rank-0 or zero-last-dim input surfaces as a typed error at build time.

pub fn rope(&self, base: f64, start_pos: usize) -> Result<Self, Error>

Apply rotary position embeddings. See fuel_graph::NodeHandle::rope. Rank < 2 surfaces as a typed error at build time.

pub fn rope_with_tables(&self, cos: &Self, sin: &Self) -> Result<Self, Error>

Apply RoPE using caller-supplied cos and sin tables so they can be shared across many layers. See fuel_graph::NodeHandle::rope_with_tables.

Rank / dtype / table-shape mismatches surface as typed errors at build time rather than panicking inside fuel_graph.

pub fn rope_with_tables_decomposed( &self, cos: &Self, sin: &Self, ) -> Result<Self, Error>

Apply RoPE using caller-supplied cos/sin tables, emitting the PRIMITIVE decomposition (slice + neg + concat + mul + add — the rotate-half recipe) instead of a single fused FusedOps::ROPE node. See fuel_graph::NodeHandle::rope_with_tables_decomposed.

CapturedRun 4b-resume: Fuel’s FusedOps::ROPE is rotate-half (Llama/HF) but has NO correct (rotate-half) CUDA kernel — baracuda’s is interleaved (see the rope-convention note). The fused op therefore places on CPU, which breaks CUDA-graph capture. Emitting the decomposition makes every step a primitive with a capture-safe CUDA kernel, so the decode runs entirely on CUDA. Forward-compatible: once a rotate-half fused CUDA rope kernel exists, the optimizer’s re-fusion pattern will fold this subgraph back into the fused op automatically (recipe principle).

Byte-identical semantics to Self::rope_with_tables (same rotate-half math); same validation.

pub fn index_select<D: Dim>( &self, dim: D, indices: &Self, ) -> Result<Self, Error>

Pick slices along dim using a 1-D U32 index tensor. Accepts any Dim. Dim bounds / index dtype / index rank mismatches surface as typed errors at build time.

pub fn gather<D: Dim>(&self, dim: D, indices: &Self) -> Result<Self, Error>

N-D gather along dim using a U32 index tensor with the same rank as self; output shape equals the index shape. Accepts any Dim. Dim bounds / index dtype / rank mismatches surface as typed errors at build time.

pub fn to_dtype(&self, dtype: DType) -> Result<Self, Error>

Convert to a different dtype. Same-dtype is a fast-path no-op (returns a clone) rather than emitting a redundant graph node.

The name follows the eager-API convention: users shouldn’t need to care whether the underlying bytes are reinterpreted (e.g. integer widening) or transcoded (e.g. f32 → bf16). Build-time validation is currently minimal — Cast itself is unfailing in fuel_graph; the Result return is reserved for future kernel-registry checks (Phase A.8c-extension).

pub fn detach(&self) -> Self

Detach this tensor from autograd. On lazy, autograd is structural (every graph edge participates in backward unless explicitly cut by a non-differentiable op), so there’s no per-tensor toggle — detach() is the identity function. Provided for eager-API parity so consumer code that calls .detach() compiles unchanged.

pub fn track_op(&self) -> bool

Whether autograd is tracking this tensor. On lazy, every tensor participates in autograd structurally; track_op returns true unconditionally for API parity with eager.

pub fn realize_f32(&self) -> Vec<f32>

Realize this tensor as an f32 Vec.

Routes unconditionally through the pipelined bridge: walk the graph, pre-realize Consts onto CPU, plan + dispatch through PipelinedExecutor, read back the root’s bytes.

Judge profile data still shapes dispatch — on this same path. When a profile is cached (crate::judge::populate_dispatch_table ran this process, or a prior run persisted one for this hardware), crate::judge::cached_oracle feeds the picker: compile_plan’s Layer-2 cost refinement and the JudgeAware runtime selector both rank alternatives (portable CPU vs AOCL/MKL kernel-source siblings included) by measured latency. Executor-unification Session 3 (2026-06-11) deleted the legacy judge::cached() branch that swapped in a Router-backed GraphExecutor instead — the picker consumes the same Judge data without leaving the production executor.

Convenience accessor — PANICS on a realize failure (.expect()). That is a deliberate choice for the test / example / notebook callers who want .realize_f32()[0] without error-handling clutter, NOT a trap: a caller that must PROPAGATE a realize failure (any serving / production path) has a documented fallible sibling — crate::pipelined_bridge::realize_one_as (with ::<f32>) — which returns Result. GAP-186: this is DOCUMENTED, NOT ENFORCED. realize_* is a genuine public API (13 fuel-core/tests/ integration crates plus fuel-examples call it), so it cannot be made test-only; the never-panic obligation lives on the production callers, which use the fallible sibling directly (as train.rs::param_to_host does).

pub fn realize_f32_reference(&self) -> Vec<f32>

Realize on CPU as an independent correctness oracle: like Self::realize_f32 but with cost-based cross-device placement suppressed, so the whole graph runs on the CPU backend’s bit-stable kernels and is never relocated to a GPU by the optimizer.

Self::realize_f32 pins CPU only as a soft host anchor — since the Step-E cost-based auto-placement, its optimizer may price model nodes onto a present GPU and insert an H2D Op::Copy, which both defeats the oracle’s independence (it would validate a backend against itself) and, on a single-device realize, crashes for lack of a seeded GPU handle. This entry hard-pins CPU (allow_cost_placement = false); by the always-built coverage commitment the CPU backend supplies a kernel for every primitive op, so nothing is ever stranded. This is the pairwise-consensus oracle that replaces the retiring fuel-reference-backend.

pub fn realize_f64(&self) -> Vec<f64>

Realize as an f64 Vec.

Routes through the fuel_dispatch::pipelined::PipelinedExecutor like Self::realize_f32 — executor-unification Session 1 (re-audit gap 8) retires the typed fuel_graph_cpu recursive evaluator from the public API. The root must already be F64-dtype (insert Self::to_dtype otherwise). The dtype guard lives at the single realize funnel crate::pipelined_bridge::realize_one_as (GAP-327): a mismatch returns fuel_ir::Error::UnexpectedDType, which the .expect below turns into a panic here — so the byte reinterpretation never silently returns garbage.

pub fn realize_bf16(&self) -> Vec<bf16>

Realize as a bf16 Vec. See Self::realize_f64 for the routing + dtype-guard rationale.

pub fn realize_f16(&self) -> Vec<f16>

Realize as an f16 Vec. See Self::realize_f64 for the routing + dtype-guard rationale.

impl Tensor

pub fn const_u32_like( &self, data: impl Into<Arc<[u32]>>, shape: impl Into<Shape>, ) -> Result<Self, Error>

Build a second const U32 (index) tensor on the same graph.

pub fn const_placeholder_like( &self, shape: impl Into<Shape>, dtype: DType, ) -> Self

Push a fuel_graph::Op::Const node on the same graph as self without populating the graph’s storage_map. The caller binds the storage Arc into the realize call via InferenceContext::insert.

Used by the Phase E.3.3 forward path to bind pre-allocated KV-cache storage Arcs (Arc<RwLock<fuel_memory::Storage>>) into a per-step graph — the graph’s legacy storage_map only holds fuel_backend_contract::Storage, so direct binding isn’t possible without a type conversion.

pub fn write_slice( &self, source: &Self, ranges: Vec<(usize, usize)>, ) -> Result<Self>

Append an fuel_graph::Op::WriteSlice node. Copies source’s bytes into self at the rectangular slab defined by ranges and returns a tensor whose Storage Arc is self’s — i.e. the post-write reference to the same underlying buffer.

Destructive on self: after the write, downstream consumers must read the bytes through the returned tensor’s NodeId, not self’s.

Returns Result: rank/shape/range mismatches surface as a typed error.

pub fn write_slice_dyn( &self, source: &Self, ranges: Vec<(usize, usize)>, dyn_axis: usize, offset: DynScalar, ) -> Result<Self>

Append an fuel_graph::Op::WriteSlice whose start on dyn_axis is a runtime value resolved through the per-pass SymEnv at realize (Phase D symbolic extents). ranges[dyn_axis].0 is ignored (the start is dynamic); the slab width ranges[dyn_axis].1 - ranges[dyn_axis].0 must equal source’s dyn_axis dim and not exceed the destination capacity. Backs the persistent decode KV-cache write at the per-token cached_len.

pub fn write_slice_rotating( &self, source: &Self, position: &Self, axis: usize, modulus: usize, ranges: Vec<(usize, usize)>, ) -> Result<Self>

Append an fuel_graph::Op::WriteSliceRotating node — like Self::write_slice but the axis axis wraps modulo modulus. position is a rank-0 U32 tensor whose value (read at realize time) is wrapped modulo modulus to determine the dynamic write start on axis. ranges[axis].0 is ignored (the rotating-axis start is dynamic); the slab width ranges[axis].1 - ranges[axis].0 must equal source’s axis dim and must not exceed modulus.

Destructive on self: same scheduling as write_slice. Backs sliding-window KV caches (Mistral / Phi-3 sliding- window). Returns Result: rank / dtype / axis-bound / modulus / range mismatches surface as typed errors at build time.

pub fn write_slice_doff( &self, source: &Self, offset: &Self, axis: usize, ranges: Vec<(usize, usize)>, ) -> Result<Self>

Append a fuel_graph::Op::WriteSliceDoff node — write source into a slab of self whose start on axis is read from offset (a rank-0 I64 tensor) device-side at kernel launch. No modulo wrap: offset is the raw start on axis. ranges[axis].0 is ignored; ranges[axis].1 - ranges[axis].0 is the write width on axis and must equal source’s axis dim. Bounds (offset + width <= dest_dims[axis]) are the caller’s contract (device-only at build). Destructive on self; same scheduling as write_slice. Backs the CUDA-graph- capturable KV-cache append (CapturedRun / DecodeSession). Returns Result: rank / dtype / axis-bound / static-width / offset-dtype mismatches surface as typed errors at build time.

pub fn conv2d( &self, weight: &Self, bias: Option<&Self>, stride: (usize, usize), padding: (usize, usize), groups: usize, ) -> Result<Self, Error>

Append a fuel_graph::Op::Fused node carrying fuel_graph::registry::FusedOpParams::Conv2D. See fuel_graph’s Tensor::conv2d for the full shape contract: self must be [N, Cin, H, W]; weight must be [Cout, Cin/groups, Kh, Kw]; bias is optional and must be [Cout] when provided. Returns a rank-4 lazy tensor [N, Cout, Hout, Wout].

Rank / channel / groups / stride mismatches surface as typed errors at build time rather than panicking inside the inner fuel_graph call.

pub fn flash_attn( &self, k: &Self, v: &Self, alibi_slopes: Option<&Self>, softmax_scale: f32, causal: bool, window_size_left: Option<usize>, window_size_right: Option<usize>, softcap: Option<f32>, ) -> Result<Self, Error>

Append a fuel_graph::Op::Fused node carrying fuel_graph::registry::FusedOpParams::FlashAttn. self is q of shape [B, Hq, Sq, D]; k and v are [B, Hkv, Sk, D] with Hq a multiple of Hkv (GQA). alibi_slopes (optional) is [Hq]. Returns the attention output, shape [B, Hq, Sq, D].

Rank / batch / GQA-divisibility / head-dim mismatches surface as typed errors at build time rather than panicking inside the inner fuel_graph call.

pub fn paged_attn( &self, k_cache: &Self, v_cache: &Self, block_table: &Self, context_lens: &Self, alibi_slopes: Option<&Self>, softmax_scale: f32, block_size: usize, softcap: Option<f32>, ) -> Result<Self, Error>

Append a fuel_graph::Op::Fused node carrying fuel_graph::registry::FusedOpParams::PagedAttn. self is the Q tensor [B, Hq, Sq, D]. k_cache / v_cache are paged caches [num_blocks, block_size, Hkv, D]. block_table is [B, max_blocks] u32; context_lens is [B] u32.

Rank / batch / GQA-divisibility / block-size / dtype mismatches surface as typed errors at build time rather than panicking inside the inner fuel_graph call.

pub fn conv_transpose2d( &self, weight: &Self, stride: (usize, usize), padding: (usize, usize), output_padding: (usize, usize), dilation: (usize, usize), groups: usize, ) -> Result<Self, Error>

Append a fuel_graph::Op::Fused node carrying fuel_graph::registry::FusedOpParams::ConvTranspose2D. self must be [N, Cin, H, W]; weight must be [Cin, Cout/groups, Kh, Kw] (note transposed channel order vs conv2d). Returns a rank-4 lazy tensor [N, Cout, Hout, Wout].

Rank / channel / groups / stride / dilation mismatches surface as typed errors at build time rather than panicking inside the inner fuel_graph call.

pub fn conv_transpose1d( &self, weight: &Self, stride: usize, padding: usize, output_padding: usize, dilation: usize, groups: usize, ) -> Result<Self, Error>

Append a transposed 1D convolution. self is [N, Cin, Lin]; weight is [Cin, Cout/groups, K] (PyTorch channel order). Returns [N, Cout, Lout].

Internally lifts to rank-4 and dispatches through conv_transpose2d — there is no separate 1D op in the IR; the lift is transparent to the executor (which sees the same Op::Fused(CONV_TRANSPOSE2D, _) it already dispatches CPU kernels for).

Unblocks audio codec decoders (DAC, EnCodec, SNAC, Mimi, Parler-TTS, MetaVoice, CSM) which all upsample quantized latents to waveform via strided transposed convs.

impl Tensor

pub fn unsqueeze<D: Dim>(&self, dim: D) -> Result<Self, Error>

Append a size-1 dimension at position dim. Inverse of Self::squeeze. Accepts any Dim (usize, D::Minus1, etc.). Bad dim surfaces as a typed error at build time.

pub fn triu(&self, diagonal: i64) -> Result<Self, Error>

Upper-triangular mask along the last two dims. diagonal = 0 keeps the main diagonal and above; positive shifts higher.

pub fn tril(&self, diagonal: i64) -> Result<Self, Error>

Lower-triangular mask along the last two dims. tril(0) is the canonical causal-attention mask.

pub fn log_softmax_last_dim(&self) -> Result<Self, Error>

log(softmax(self)) along the last dim, fused into one op.

pub fn softmax<D: Dim>(&self, dim: D) -> Result<Self, Error>

Numerically-stable softmax along an arbitrary axis. Accepts any Dim. Decomposes into max_keepdim / broadcast_sub / exp / sum_keepdim / broadcast_div, all of which already accept D: Dim, so this is a pure composition with no new graph op.

When dim resolves to the last axis, prefer Self::softmax_last_dim, which dispatches to the fused SoftmaxLastDim op (single kernel rather than five graph nodes).

pub fn log_softmax<D: Dim>(&self, dim: D) -> Result<Self, Error>

Numerically-stable log(softmax(self)) along an arbitrary axis. Accepts any Dim. Computes x - max - log(sum(exp(x - max))) — the standard log-sum-exp form, which avoids the explicit softmax-then-log underflow path. Pure composition over existing primitives.

When dim resolves to the last axis, prefer Self::log_softmax_last_dim, which dispatches to the fused LogSoftmaxLastDim op.

pub fn argmin_dim<D: Dim>(&self, dim: D) -> Result<Self, Error>

Argmin along dim, returning a U32 tensor with the reduced dim removed. Non-differentiable. Bad dim surfaces as a typed error at build time. Accepts any Dim.

pub fn masked_fill(&self, mask: &Self, value: Scalar) -> Result<Self, Error>

Fill every position where mask != 0 with value; pass self through everywhere mask == 0. mask must be U8 with the same shape as self; value’s dtype must match self.

pub fn index_add<D: Dim>( &self, dim: D, indices: &Self, src: &Self, ) -> Result<Self, Error>

self + scatter(indices, src, dim=dim) — accumulate src rows at positions named by indices along dim. indices is rank-1 U32 with length equal to src.dims()[dim]. Accepts any Dim. Dim bounds / index dtype / shape / dtype-parity mismatches surface as typed errors at build time.

pub fn scatter_add<D: Dim>( &self, dim: D, indices: &Self, src: &Self, ) -> Result<Self, Error>

Functional inverse of Self::gather. Accumulates src into self at positions given by indices (substituted at dim). Accepts any Dim. Dim bounds / index dtype / shape / dtype- parity mismatches surface as typed errors at build time.

pub fn relu_inplace(&self) -> Self

In-place max(0, self). See Self::relu for the functional variant.

pub fn silu_inplace(&self) -> Self

In-place self * sigmoid(self). See Self::silu for the functional variant.

pub fn gelu_tanh_inplace(&self) -> Self

In-place tanh-approximation GELU. See Self::gelu_tanh for the functional variant.

pub fn tanh_inplace(&self) -> Self

In-place tanh(self). See Self::tanh for the functional variant.

pub fn sigmoid_inplace(&self) -> Self

In-place sigmoid(self). See Self::sigmoid for the functional variant.

pub fn affine_inplace(&self, mul: f64, add: f64) -> Self

In-place self = mul · self + add. Single fused-op equivalent of self.mul_scalar(mul).add_scalar(add) plus reassignment.

pub fn const_f64_like( &self, data: impl Into<Arc<[f64]>>, shape: impl Into<Shape>, ) -> Result<Self, Error>

Build a sibling F64 Const on the same graph as self.

pub fn const_i64_like( &self, data: impl Into<Arc<[i64]>>, shape: impl Into<Shape>, ) -> Result<Self, Error>

Build a sibling I64 Const on the same graph. Used by integer- target ops (e.g. cross-entropy with PyTorch-convention class indices).

pub fn on_device(self, device: &Device) -> Self

Pin this tensor’s realized storage to device. Consumes self because the placement is a graph-level annotation tied to the node id rather than a side-effecting operation.

pub fn release(&self) -> Self

Append an Op::Release node — explicitly drop this tensor’s device-resident storage once the ordering pass has scheduled every reader before it.

pub fn move_to_device(&self, device: &Device) -> Self

Move bytes to device, destroying the source. Use when the source is genuinely dead after the transfer.

pub fn copy_to_device(&self, device: &Device) -> Self

Copy bytes to device, leaving the source resident. Use when other ops still need the source.

pub fn backward(&self) -> GradMap

Run reverse-mode autograd from this tensor as the loss, returning a fuel_graph::GradMap keyed by every input tensor reached. The gradient nodes extend the same graph; realizing a gradient re-executes the forward dependencies.

impl Tensor

pub fn transpose_last_two(&self) -> Result<Self, Error>

Transpose the last two dims as a Result-returning convenience — rank < 2 surfaces as an error rather than the panic the no-arg Self::transpose would produce. Alias for the eager transpose_last_two.

pub fn t(&self) -> Result<Self, Error>

Eager-API alias of Self::transpose_last_two. Matches PyTorch’s .t() short form and the existing eager Tensor::t method.

pub fn transpose_dims<D1: Dim, D2: Dim>( &self, dim1: D1, dim2: D2, ) -> Result<Self, Error>

Two-argument transpose: swap dims dim1 and dim2, leaving the rest in place. Implemented via fuel_graph::NodeHandle::try_permute; matches the eager transpose(d1, d2) two-arg form. Accepts any Dim (usize, D::Minus1, etc.).

pub fn flatten<D1: Dim, D2: Dim>( &self, start_dim: D1, end_dim: D2, ) -> Result<Self, Error>

Collapse dims [start_dim, end_dim] (inclusive) into a single dimension. Returns Result so out-of-bounds surfaces as a typed error rather than a panic. Accepts any Dim for either arg (D::Minus1 for the last axis works).

pub fn flatten_to<D: Dim>(&self, end_dim: D) -> Result<Self, Error>

Flatten dims [0, end_dim] (inclusive) into one.

pub fn flatten_from<D: Dim>(&self, start_dim: D) -> Result<Self, Error>

Flatten dims [start_dim, rank-1] into one.

pub fn flatten_all(&self) -> Result<Self, Error>

Flatten the tensor to rank-1 (single dim containing every element).

pub fn stack<D: Dim>(args: &[&Self], dim: D) -> Result<Self, Error>

Stack tensors along a new dim at position dim. Each input is unsqueezed at dim then concatenated. All inputs must have identical shape; dim may equal rank (append a new trailing dim). Accepts any Dim.

pub fn sum_keepdim<D: Dim>(&self, dim: D) -> Result<Self, Error>

Sum along dim, keeping the reduced dim as size 1. Accepts any Dim. Returns Result because of the cascade from Self::unsqueeze.

pub fn mean_keepdim<D: Dim>(&self, dim: D) -> Result<Self, Error>

Mean along dim, keeping the reduced dim as size 1.

pub fn max_keepdim<D: Dim>(&self, dim: D) -> Result<Self, Error>

Max along dim, keeping the reduced dim as size 1.

pub fn min_keepdim<D: Dim>(&self, dim: D) -> Result<Self, Error>

Min along dim, keeping the reduced dim as size 1.

pub fn var_keepdim<D: Dim>( &self, dim: D, correction: f64, ) -> Result<Self, Error>

Variance along dim, keeping the reduced dim as size 1. Divides squared deviations by n - correction (numpy/PyTorch-2.x correction convention: 0.0 is the population variance — KISS-Ops §6.13’s default, and numpy’s own default — 1.0 is Bessel’s (sample/unbiased) correction, matching PyTorch’s older default and the eager Tensor::var_keepdim’s prior behavior.

Not a default-argument convenience: KISS-Ops §6.13 requires Bessel’s correction to be an EXPLICIT attribute, not an implicit torch-style default silently baked into the divisor — this parameter exists so a caller cannot get Bessel’s correction by omission. correction >= n produces a non-finite result (same shape as any other degenerate reduction — never a panic).

pub fn var<D: Dim>(&self, dim: D, correction: f64) -> Result<Self, Error>

Variance along dim, squeezing the reduced dim. See Self::var_keepdim for the correction parameter’s meaning.

pub fn affine(&self, mul: f64, add: f64) -> Self

y = mul * self + add element-wise. Two-op composite of Self::mul_scalar then Self::add_scalar; a fused Op::Affine can collapse this into a single op later (see the in-place counterpart Self::affine_inplace).

pub fn scale_and_shift(&self, scale: f64, shift: f64) -> Self

y = scale * self + shift. Alias of Self::affine with descriptive parameter names; matches eager’s Tensor::scale_and_shift.

pub fn elu(&self, alpha: f64) -> Self

Exponential Linear Unit: self where self > 0, alpha * (exp(self) - 1) otherwise. Composite of where_cond, gt, exp, affine.

pub fn dot(&self, rhs: &Self) -> Result<Self, Error>

Inner product of two rank-1 tensors. Composite of mul + sum_all; matches eager’s Tensor::dot.

pub fn mv(&self, rhs: &Self) -> Result<Self, Error>

Matrix × vector: [m, n] · [n] -> [m]. No broadcasting. Composite of unsqueeze + matmul + squeeze. Matches eager’s Tensor::mv.

pub fn matvec(&self, rhs: &Self) -> Result<Self, Error>

Alias of Self::mv with a more descriptive name. Matches eager’s Tensor::matvec.

pub fn broadcast_matmul(&self, rhs: &Self) -> Result<Self, Error>

Broadcast-aware matmul. Lazy’s Self::matmul already accepts broadcast-compatible operands; this method is exposed for signature compatibility with eager’s Tensor::broadcast_matmul.

pub fn ones_like(&self) -> Result<Self, Error>

New tensor with the same shape, dtype, and graph as self, filled with ones. Returns Err for unsupported dtypes (anything outside F32/F64/BF16/F16/U32/I64) — matches eager Tensor::ones_like parity.

pub fn zeros_like(&self) -> Result<Self, Error>

New tensor with the same shape, dtype, and graph as self, filled with zeros. Returns Err for unsupported dtypes (anything outside F32/F64/BF16/F16/U32/I64) — matches eager Tensor::zeros_like parity.

pub fn ones( shape: impl Into<Shape>, dtype: DType, device: &Device, ) -> Result<Self, Error>

New tensor with shape/dtype/device, every element set to 1. Static factory equivalent of eager’s Tensor::ones. Returns Err for dtypes outside F32/F64/BF16/F16/U32.

pub fn zeros( shape: impl Into<Shape>, dtype: DType, device: &Device, ) -> Result<Self, Error>

New tensor with shape/dtype/device, every element set to 0. Static factory equivalent of eager’s Tensor::zeros. Returns Err for dtypes outside F32/F64/BF16/F16/U32.

Mints a NEW graph — it delegates to the from_* constructors, so the tensor it returns cannot be combined with one built elsewhere. To add a zero tensor to an existing graph, use zeros_like or a const_*_like builder. See graph affinity.

pub fn full( shape: impl Into<Shape>, value: Scalar, device: &Device, ) -> Result<Self, Error>

New tensor of shape/device filled with value. The scalar’s dtype determines the tensor’s dtype. Returns Err for scalar dtypes outside F32/F64/BF16/F16/U32.

Mints a NEW graph — it delegates to the from_* constructors, so the tensor it returns cannot be combined with one built elsewhere. To add a filled tensor to an existing graph, use a const_*_like builder. See graph affinity.

pub fn eye(n: usize, dtype: DType, device: &Device) -> Self

Identity matrix [n, n] with the given dtype on the given device. Built host-side as a flat Vec; no graph-layer arange dependency.

pub fn split_heads( &self, num_heads: usize, head_dim: usize, ) -> Result<Self, Error>

Split a (B, N, num_heads * head_dim) projection into the multi-head attention layout (B, num_heads, N, head_dim). Equivalent to reshape(B, N, num_heads, head_dim).permute([0, 2, 1, 3]) — promoted to a method to retire the per-port reimplementations of this same composite.

pub fn merge_heads(&self) -> Result<Self, Error>

Merge a (B, num_heads, N, head_dim) attention result back into the projection layout (B, N, num_heads * head_dim). Inverse of Self::split_heads.

pub fn add_trailing_bias(&self, bias: Arc<[f32]>) -> Result<Self, Error>

Add a length-bias.len() bias vector to the trailing dim of self, broadcasting across all leading dims. The bias is materialized fresh on self’s graph from the supplied Arc<[f32]>.

Common pattern after WeightStorage::apply_linear when the linear has a bias term but the activation tensor is on a different anchor than where the bias was originally allocated. Several lazy ports inlined this same 3-line helper as bias_add — promoted here to a method.

pub fn embed_tokens( embed_table: Arc<[f32]>, vocab_size: usize, hidden: usize, tokens: &[u32], device: &Device, ) -> Result<Self, Error>

Embed tokens against an [vocab_size, hidden] embedding table held as Arc<[f32]>. Returns (1, seq, hidden) rank-3 hidden states ready to feed into a decoder backbone.

Bootstraps a fresh graph anchored on a new const-f32 node. For composition with an already-built tensor (e.g., multimodal models that need text embeddings on the audio graph), use Self::embed_tokens_anchored instead.

Retires the 7-line `from_f32 + const_u32_like + index_select

• reshape` ceremony every LLM port carried.

pub fn embed_tokens_anchored( &self, embed_table: Arc<[f32]>, vocab_size: usize, hidden: usize, tokens: &[u32], ) -> Result<Self, Error>

Variant of Self::embed_tokens that anchors the embedding table and token-id constants on the receiver’s graph, so the resulting embeddings can compose with self and other tensors already on that graph. Used by multimodal models (vision + text, audio + text) where the text embeddings must live on the modality encoder’s graph for cross-substitution to work.

pub fn rope_tables_const( &self, theta: f64, start_pos: usize, seq: usize, head_dim: usize, ) -> (Self, Self)

Build the standard (non-interleaved) RoPE cos/sin tables for seq positions starting at start_pos, anchored on the receiver’s graph. Returns (cos, sin), each with shape [seq, head_dim].

Delegates the actual (theta, position) → (cos, sin) host computation to fuel_graph::build_rope_tables (the canonical reference); only the const-tensor materialization is folded into one method to retire the per-port 4-line ceremony every LLM port did before calling rope_with_tables.

pub fn rope_delta_tables_const( &self, theta: f64, delta: usize, rows: usize, head_dim: usize, ) -> (Self, Self)

Constant-position RoPE cos/sin tables [rows, head_dim] where EVERY row is the same θ·delta rotation — the rung-2 delta-rotation tables. A cached prefix’s keys, already rotated for their original positions, are shifted to +delta by applying this uniform rotation (RoPE composes additively per pair: R(delta)·R(p) = R(p+delta)).

This is the position-delta row repeated rows times — NOT rope_tables_const(theta, delta, rows, head_dim), which would give the incrementing progression delta, delta+1, … (the standard RoPE sweep). Routed through the canonical fuel_graph::build_rope_tables so it inherits the model’s inv-freq/scaling exactly.

pub fn rope_delta_rotate_block_f32( dev: &Device, k_block: &[f32], theta: f64, delta: usize, block_size: usize, n_kv_heads: usize, head_dim: usize, ) -> Vec<f32>

Uniformly delta-rotate one cached (post-RoPE) K block by θ·delta, returning the shifted block in the SAME pool layout. Reuses the model’s exact rope_with_tables_decomposed (rotate-half) rather than a hand-rolled rotation, so it inherits the model’s RoPE convention + scaling and stays byte-exact with a direct rope-at-shifted-position. A single uniform rotation is correct for the whole block even though it holds block_size distinct original positions — the delta is the same for every position.

k_block is the pool block layout, row-major [block_size, n_kv_heads, head_dim] (positions-major, as DeviceKvPool::read_block returns). RoPE wants (position, dim) as the last two axes, so heads are brought forward to the project_qkv_roped layout [1, n_kv_heads, block_size, head_dim], rotated, and returned to [block_size, n_kv_heads, head_dim]. The rung-2 numeric core, realized on dev.

pub fn rope_tables_const_batched( &self, theta: f64, positions: &[usize], head_dim: usize, ) -> (Self, Self)

[B, 1, 1, head_dim] RoPE cos/sin tables, row b at absolute position positions[b] — the per-row generalization of Self::rope_tables_const (which carries one shared position). Built as a single const from B calls to fuel_graph::build_rope_tables (the canonical (theta, position) → (cos, sin) math), concatenated host-side — no Op::Concat chain. At B == 1 the emitted data is bit-identical to rope_tables_const’s [1, head_dim] const (same builder call), so Self::rope_batched reduces to the single-position decomposed RoPE.

This is the mechanism that lets a decode batch carry sessions at different positions (continuous batching) rather than requiring equal-length rows. anchor (self) supplies the graph.

pub fn rope_batched(&self, cos: &Self, sin: &Self) -> Result<Self, Error>

Rotary position embedding with per-batch-row tables.

self is [B, H, 1, D]; cos/sin are [B, 1, 1, D] (row b holds the table for sequence b’s own position — see Self::rope_tables_const_batched). Broadcasting the [B, 1, 1, D] tables over the head axis is what lets one table row serve all of that sequence’s heads.

This is fuel_graph::NodeHandle::rope_with_tables_decomposed’s body (fuel-graph/src/lib.rs ~6931-6954) with the table shape generalized from one shared position to one-per-row — same ops, same order, same associativity (y = x·cos + concat(-x[…,D/2..], x[…,..D/2])·sin), so at B == 1 it is bit-identical to rope_with_tables_decomposed. Built on the public Tensor surface (not a new fuel-graph op) deliberately: the single-position Self::rope_with_tables_decomposed hard-requires cos.dims() == [seq, d], so it cannot carry per-row tables, and rebuilding the lowering here avoids re-entering the fuel-graph rope op.

(Adopted from the Lightbulb consumer’s verified rope_batched; the mechanism belongs in Fuel, not the consumer.)

pub fn rope_partial( &self, rope_cos: &Self, rope_sin: &Self, rope_dim: usize, ) -> Result<Self, Error>

Apply RoPE to the first rope_dim entries of each head and pass the remaining head_dim - rope_dim features through unchanged. head_dim is derived from the receiver’s last-dim size. When rope_dim == head_dim this reduces to Self::rope_with_tables.

Implements the partial-rotary convention used by StableLM, Phi, Persimmon, MixFormer, RecurrentGemma, and Gemma-4 text — all the ports that carried an identical 5-line slice + rope + concat helper before this method.

pub fn add_optional_trailing_bias( &self, bias: Option<&Arc<[f32]>>, ) -> Result<Self, Error>

Option<Arc<[f32]>> variant of Self::add_trailing_bias: if bias.is_none(), return self unchanged; else apply add_trailing_bias. Models the linear_b / linear_no_bias branch every per-port optional_bias / opt_bias helper does.

pub fn softcap(&self, cap: f64) -> Self

Logit-softcap: cap · tanh(self / cap). Used by the Gemma-2 / Gemma-3 attention-logit and final-logit softcap branches. The math is identical regardless of where the cap is applied; retired the two per-port softcap / apply_softcap helpers in favor of this method.

pub fn softcap_optional(&self, cap: Option<f64>) -> Self

Option<f64> variant of Self::softcap: when cap.is_none() or cap <= 0.0, return self unchanged; else apply Self::softcap. Mirrors the optional-bias pattern.

pub fn rms_norm_affine_with_offset( &self, gain: &[f32], offset: f32, eps: f64, ) -> Result<Self, Error>

Apply RMSNorm along the last dim with (gain + offset) · x. Equivalent to Self::rms_norm_affine after adding a scalar to every gain element — used by Gemma-family ports where the stored gain represents gain - 1 and the runtime path must reconstruct gain + 1.

Materializes the shifted gain on the receiver’s graph; one allocation per call.

pub fn rms_norm_affine(&self, gain: Arc<[f32]>, eps: f64) -> Result<Self, Error>

Apply RMSNorm along the last dim with an affine gain · x post-step (no bias — RMSNorm has no β term). gain is a length-gain.len() vector materialized fresh on the receiver’s graph and broadcast across all leading dims.

pub fn global_avg_pool_2d(&self) -> Result<Self, Error>

Global average pool over the spatial dims of a rank-4 (B, C, H, W) tensor: reduces dims 2 and 3, returning (B, C). For the keepdim variant ((B, C, 1, 1), used by SE blocks), follow with .reshape(Shape::from_dims(&[B, C, 1, 1])).

Backs the classification heads of every conv vision port (ResNet, EfficientNet, ConvMixer, FastViT, MobileNetV4, MobileOne, RepVGG, ConvNeXt, EfficientViT, etc.) plus the pre-projection pool inside each squeeze-excite block.

pub fn channel_affine_4d( &self, gain: Arc<[f32]>, bias: Arc<[f32]>, ) -> Result<Self, Error>

Apply a per-channel affine gain · x + bias to a rank-4 (B, C, H, W) tensor. Both gain and bias are length-C vectors materialized fresh on the receiver’s graph and broadcast across the spatial axes.

Equivalent to fused-affine BatchNorm at inference time: the running mean / running var / eps are absorbed at load time into gain = γ / sqrt(var + eps) and bias = β - μ · γ / sqrt(var + eps), so the runtime forward is just this multiply-add. Used by inference-only conv vision ports (ResNet, EfficientNet, FastViT, etc.).

pub fn additive_causal_mask_like(anchor: &Tensor, seq_len: usize) -> Self

Build the strict additive causal mask (seq_len, seq_len) anchored on anchor’s graph: 0 on and below the diagonal, f32::NEG_INFINITY above it. Add to attention scores before softmax to enforce strict causality (position i cannot attend to position j > i).

Equivalent to the (T, T) mask several ports build inline — promoted here so call sites stop drifting.

pub fn tril2(n: usize, dtype: DType, device: &Device) -> Self

Lower-triangular ones matrix [n, n]. tril2(n).to_dtype(dtype) is the causal-attention-mask building block.

pub fn triu2(n: usize, dtype: DType, device: &Device) -> Self

Upper-triangular ones matrix [n, n].

pub fn narrow<D: Dim>( &self, dim: D, start: usize, len: usize, ) -> Result<Self, Error>

Eager-API alias of Self::slice (PyTorch / Candle naming). narrow(dim, start, len) is slice(dim, start, len) — produces a view of [start, start+len) along dim. Bad input surfaces as a typed error at build time. Accepts any Dim.

pub fn chunk<D: Dim>(&self, chunks: usize, dim: D) -> Result<Vec<Self>, Error>

Split into chunks views along dim. The split distributes the chunk_size = ceil(dim_size / chunks) extra slot to the leading chunks so every chunk’s size differs by at most 1. If dim_size < chunks, returns dim_size singleton chunks instead of chunks chunks (matches eager / PyTorch). Accepts any Dim.

pub fn get(&self, i: usize) -> Result<Self, Error>

Sub-tensor at index i along dim 0. Equivalent to self.slice(0, i, 1).unwrap().squeeze(0). Matches eager’s crate::Tensor::get.

pub fn get_on_dim<D: Dim>(&self, dim: D, index: usize) -> Result<Self, Error>

Sub-tensor at index along an arbitrary dim. Equivalent to self.slice(dim, index, 1).unwrap().squeeze(dim). Matches eager’s crate::Tensor::get_on_dim. Accepts any Dim.

pub fn sum_dims<D: Dims>(&self, dims: D) -> Result<Self, Error>

Multi-dim sum: reduce over every dim in dims, squeezing each. Reduces from the highest dim down so the lower dim indices stay valid throughout the reduction.

pub fn mean_dims<D: Dims>(&self, dims: D) -> Result<Self, Error>

Multi-dim mean: reduce over every dim in dims, squeezing each. Reduces from the highest dim down. Accepts any Dims.

pub fn sum_dims_keepdim(&self, dims: &[usize]) -> Result<Self, Error>

Multi-dim sum with keepdim: every named dim becomes size 1 instead of being squeezed out. Reduce-order-invariant (every keepdim preserves indices). Returns Result because of cascade from Self::sum_keepdim.

pub fn mean_dims_keepdim(&self, dims: &[usize]) -> Result<Self, Error>

Multi-dim mean with keepdim.

pub fn rand_like(&self, lo: f64, up: f64) -> Result<Self, Error>

Uniform random tensor in [lo, up) with shape/dtype/device matching self. Backed by rand::rng. Returns Err for unsupported dtypes.

pub fn randn_like(&self, mean: f64, stdev: f64) -> Result<Self, Error>

Normal random tensor with shape/dtype/device matching self. Returns Err for unsupported dtypes or invalid stdev.

pub fn rand( shape: impl Into<Shape>, lo: f64, up: f64, dtype: DType, device: &Device, ) -> Result<Self, Error>

Uniform random tensor in [lo, up). Static factory. Supported dtypes: F32, F64, BF16, F16. F32 is the typical initialization target. Returns Err for any other dtype.

pub fn randn( shape: impl Into<Shape>, mean: f64, stdev: f64, dtype: DType, device: &Device, ) -> Result<Self, Error>

Normal random tensor with given mean and stdev. Static factory. Supported dtypes: F32, F64, BF16, F16. Returns Err on any other dtype, or if stdev is not finite / not positive.

pub fn arange(start: f32, end: f32, device: &Device) -> Self

arange(start, end, device): a rank-1 tensor of [start, end) in step 1, dtype F32. Matches NumPy / PyTorch convention.

pub fn arange_step(start: f32, end: f32, step: f32, device: &Device) -> Self

arange_step(start, end, step, device): a rank-1 tensor of [start, end) with constant step. F32 only for the static factory; cast for other dtypes. Errors at runtime if step == 0.

pub fn linspace(start: f32, end: f32, n: usize, device: &Device) -> Self

Linearly-spaced 1D tensor with n points from start to end (inclusive on both ends). Matches NumPy’s linspace.

pub fn norm(&self) -> Self

Frobenius norm: sqrt(sum(self * self)). Returns a scalar tensor.

pub fn conv1d( &self, weight: &Self, bias: Option<&Self>, stride: usize, padding: usize, groups: usize, ) -> Result<Self, Error>

General 1-D cross-correlation. Shapes:

• self: [N, Cin, T]
• weight: [Cout, Cin/groups, K]
• bias (optional): [Cout]
• returns: [N, Cout, Tout] where Tout = (T + 2·padding - K) / stride + 1

Implemented as a composite via Conv2D: unsqueeze the spatial dim to make a unit H = 1, run Conv2D with Kh = 1, stride.0 = 1, padding.0 = 0, then squeeze the dim back out. Works through every backend’s Conv2D dispatch (CPU, CUDA via baracuda, Vulkan, AOCL, MKL). The future fused Op::Conv1D will collapse the unsqueeze/squeeze pair when a high-volume Conv1D consumer materializes.

pub fn conv1d_with_algo<A>( &self, weight: &Self, bias: Option<&Self>, stride: usize, padding: usize, groups: usize, _algo: A, ) -> Result<Self, Error>

Eager-API parity for conv1d_with_algo. The _algo selector is ignored on the lazy path — algorithm selection happens at backend dispatch time, not at graph construction. Reduces to Self::conv1d.

pub fn avg_pool2d( &self, kernel: (usize, usize), stride: (usize, usize), padding: (usize, usize), count_include_pad: bool, ) -> Result<Self, Error>

2-D average pooling. Input [N, C, H, W], output [N, C, H_out, W_out] where H_out = (H + 2·padding.0 - kernel.0) / stride.0 + 1.

Implemented as a depthwise Conv2D with a constant 1/(kh·kw) kernel: one graph node + the kernel const. Works through every backend’s Conv2D dispatch and inherits Conv2D’s gradient. Composite supports arbitrary kernel / stride / padding.

pub fn avg_pool2d_with_stride( &self, kernel: (usize, usize), stride: (usize, usize), ) -> Result<Self, Error>

Eager-API parity for avg_pool2d_with_stride. Same shape as Self::avg_pool2d but the stride is passed explicitly rather than inferred from kernel.

pub fn max_pool2d( &self, kernel: (usize, usize), stride: (usize, usize), padding: (usize, usize), ) -> Result<Self, Error>

2-D max pooling. Input [N, C, H, W], output [N, C, H_out, W_out] where H_out = (H + 2·padding.0 - kernel.0) / stride.0 + 1.

Composite via slice + maximum: pad the input, then for every (ky, kx) in [0..kh, 0..kw] slice the strided grid of taps (one tap per output position) and take the elementwise max. Produces kh·kw nodes per call plus padding — cheap for the common 2×2 / 3×3 cases.

Strided-slice trick: for stride sh, reshape the padded H from (H_out · sh) to (H_out, sh), then slice the inner sh-dim at the tap index. Requires H_padded == H_out · sh exactly. Inputs that don’t divide cleanly will be padded so they do.

pub fn max_pool2d_with_pad_value( &self, kernel: (usize, usize), stride: (usize, usize), padding: (usize, usize), pad_value: f32, ) -> Result<Self, Error>

max_pool2d variant where the boundary padding is filled with an explicit pad_value instead of 0.0. Pass f32::NEG_INFINITY for PyTorch-correct semantics (padded slots can never win the max). All other constraints match Self::max_pool2d; the only difference is the constant value the implicit boundary pad uses.

pub fn max_pool2d_with_stride( &self, kernel: (usize, usize), stride: (usize, usize), ) -> Result<Self, Error>

Eager-API parity for max_pool2d_with_stride.

pub fn upsample_nearest2d(&self, scale: usize) -> Result<Self, Error>

Nearest-neighbor upsample by integer factor scale along the last two spatial dims. Input [N, C, H, W] → output [N, C, H·scale, W·scale].

Composite via reshape + concat + reshape: insert a unit dim after each spatial dim, concat scale copies of the tensor on each new dim, then collapse the inflated dims back. Same shape as the upsample_nearest_2x helper in fuel_transformers::models::lazy_yolov8 and fuel_transformers::models::lazy_sd_unet, generalized to arbitrary scale.

pub fn upsample_nearest1d(&self, scale: usize) -> Result<Self, Error>

Nearest-neighbor upsample for 1-D signals [N, C, T] by integer scale. Reshape to insert a unit dim, concat scale copies, reshape back.

pub fn interpolate2d( &self, target_h: usize, target_w: usize, ) -> Result<Self, Error>

2-D nearest interpolation to an explicit target size. Arbitrary ratios (non-integer, non-uniform between H and W) supported via an index_select-based composite. The indexing convention matches PyTorch / the eager kernel: src_h[oi] = min(H - 1, floor(oi * H / H_out)).

Used by DepthAnythingV2’s DPT head and similar dense prediction heads that resize feature maps to arbitrary targets.

pub fn interpolate1d(&self, target_t: usize) -> Result<Self, Error>

1-D nearest interpolation to an explicit target size. Same constraints as Self::interpolate2d: integer-multiple targets only.

pub fn pad_with_zeros<D: Dim>( &self, dim: D, left: usize, right: usize, ) -> Result<Self, Error>

Pad with zeros along dim: left zeros before, right zeros after. Thin wrapper over Self::pad_with_value with value = 0.0. Composite — no new graph op. Accepts any Dim.

pub fn pad_with_value<D: Dim>( &self, dim: D, before: usize, after: usize, value: f32, ) -> Result<Self, Error>

Pad with a constant value along dim: before slots before, after slots after. Wraps Self::pad with PadMode::Constant for the named dim (other dims get (0, 0)); the f32 value is widened to the graph op’s f64 param. Useful for -inf padding around max-reductions (e.g. PyTorch-style max_pool2d, where padded positions must never win the max). Accepts any Dim.

pub fn meshgrid(args: &[&Self], xy_indexing: bool) -> Result<Vec<Self>, Error>

Coordinate grids from rank-1 inputs. Matches PyTorch’s torch.meshgrid and eager’s crate::Tensor::meshgrid:

• xy_indexing = true (Cartesian, NumPy default): the first two inputs are swapped before broadcasting and the resulting grids reversed back, so for (x, y) the grids carry shape [len(y), len(x)] (image-coordinate convention).
• xy_indexing = false (matrix / ij): grids carry the input cardinalities in input order.

Implemented as reshape + repeat. All inputs must share the same dtype and live on the same graph.

pub fn repeat(&self, repeats: impl Into<Shape>) -> Result<Self, Error>

Repeat the tensor along each dim repeats[i] times. If repeats has more dims than self, self is implicitly left-padded with size-1 dims to match. Matches PyTorch’s Tensor.repeat.

impl Tensor

pub fn from_safetensors_bytes( bytes: &[u8], dtype: Dtype, shape: &[usize], device: &Device, ) -> Result<Self>

Build a Tensor from raw little-endian bytes as they appear in a safetensors file, plus a dtype and shape. Row-major layout is assumed. The byte count must match shape.elem_count() * dtype_bytes.

This is the low-level loader. Prefer Self::from_safetensors_view if you already have a safetensors::TensorView in hand.

Supported dtypes today: F32, F64, BF16, F16, U32. Integer types other than U32 are rejected to keep the surface small; add them when a real model needs them.

pub fn from_safetensors_view( view: &TensorView<'_>, device: &Device, ) -> Result<Self>

Build a Tensor from a safetensors::TensorView. This is the most natural entry point when iterating over a fuel_loaders::safetensors::MmapedSafetensors or similar.

Trait Implementations

impl Clone for Tensor

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Tensor

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.