Crate mlas_sys 

Thin FFI wrapper around a vendored subset of ONNX Runtime’s MLAS single-precision GEMM (MlasGemmBatch).

The vendored MLAS is compiled in its standalone BUILD_MLAS_NO_ONNXRUNTIME mode, whose threading primitives normally serialize. This crate installs a Rayon-backed parallel-for backend (see [ensure_threading] and vendor/shim.cpp) so MLAS keeps its own cache-aware GEMM tile partitioning while executing the tiles across the current Rayon pool — the same pool the rest of onnx-runtime-ep-cpu uses, so there is no oversubscription. See docs/MLAS_SYS_SPIKE.md for the original single-thread feasibility spike.

Structs

ConvPlan

Prepared MLAS Float32 convolution parameters for one concrete NCHW shape.

NchwcActivation

MLAS activation applied inside (or immediately after) a convolution.

PackedB

MLAS’s packed layout is accessed with aligned AVX-512 loads/stores, so the backing allocation is 64-byte aligned (a plain Vec<u8> is not).

SQNBitPackedB

MLAS-packed blockwise-quantized B weight for sqnbit_gemm, mirroring how ORT pre-packs the constant MatMulNBits initializer once and reuses it.

Enums

PoolKind

MLAS Float32 pooling mode.

SQNBitComputeType

Blocked n-bit quantized GEMM compute type, mirroring MLAS’s MLAS_QNBIT_GEMM_COMPUTE_TYPE. Only the two x86 float-input variants used by the CPU MatMulNBits decode path are exposed.

Functions

compute_clip

Compute contiguous Float32 clipping with MLAS SIMD.

compute_logistic

Compute the elementwise logistic (sigmoid) output = 1 / (1 + exp(-input)) over equal-length contiguous f32 slices using MLAS’s SIMD sigmoid. Single threaded; callers shard across threads themselves when needed.

compute_relu

Compute contiguous Float32 ReLU with MLAS SIMD.

compute_silu

Compute elementwise SiLU output = input / (1 + exp(-input)) over equal-length contiguous f32 slices. MLAS runtime-dispatches to its fused one-pass AVX-512F kernel when available and uses a portable fallback elsewhere. Single threaded; callers shard across threads themselves.

eltwise_add

Compute contiguous Float32 elementwise addition with MLAS SIMD.

nchwc_block_size

SIMD channel-block width used by the MLAS NCHWc kernels (8 for AVX2, 16 for AVX-512). A value <= 1 means the host has no blocked-convolution kernel and callers must use the plain ConvPlan path instead.

nchwc_conv

Execute an NCHWc blocked 2-D convolution.

nchwc_pool

Execute an NCHWc blocked 2-D pool using MLAS.

nchwc_reorder_filter_bibo

Reorder an OIHW filter into the OIHWBiBo layout (both input and output channels blocked), padding partial blocks with zeros. dest must hold round_up(O, block) * round_up(I, block) * H * W elements.

nchwc_reorder_filter_bo

Reorder an OIHW filter into the OIHWBo layout (only output channels blocked), padding partial output blocks with zeros. Used for the NCHW-input (first-layer) and depthwise algorithms. dest must hold round_up(O, block) * I * H * W elements.

nchwc_reorder_input_nchw

Reorder an NCHW activation plane set into NCHWc. channels must be a multiple of 4; dest must hold round_up(channels, block) * input_size elements (partial trailing block is zero padded).

nchwc_reorder_output_nchw

Reorder an NCHWc output buffer back to dense NCHW, keeping only output_shape[1] channels.

pool

Execute an N-dimensional NCHW Float32 pool using MLAS.

selected_float_kernel

Runtime-selected f32 GEMM microkernel: 512 = AVX-512F, 3 = FMA3/AVX2, 1 = AVX, -1 = other/unknown, 0 = non-x86.

sgemm

General entry point mirroring the C shim, exposing transpose flags and alpha/beta. Leading dimensions default to the natural row-major strides.

sgemm_nn

Safe wrapper computing C = A * B for row-major matrices with no transpose.

sgemm_nn_packed

C = A * packed(B) for row-major A (m x k), reusing a pre-packed B.

sqnbit_gemm

Compute C = A * dequant(packed) + bias for row-major A (m x k) and C (m x n), reusing a pre-packed blockwise-quantized weight.

sqnbit_gemm_available

Returns whether MLAS has a blocked n-bit GEMM kernel for the current host and the given (bits, block_len, compute_type). Callers must gate every SQNBitPackedB / sqnbit_gemm use on this being true.

sqnbit_gemm_into^⚠

Compute one N-shard of C = A * dequant(packed) + bias into a caller-owned output whose leading dimension is ldc (columns per row), writing this shard’s packed.n columns starting at c for each of the m rows. This lets a weight partitioned along N (e.g. one shard per decode worker) write its columns into a shared [m, ldc] output without a scatter copy; for a single full-width shard ldc == packed.n and it matches sqnbit_gemm.