Crate rten_gemm

Expand description

This crate provides implementations of matrix multiplication for various architectures, for both floating point and int8 inputs.

Structs§

BlockQuantizedGemm: Performs matrix-multiplication between an un-quantized LHS / “A” matrix and a block-quantized RHS / “B” matrix.
BlockQuantizedMatrix: Matrix which is quantized into blocks along the K dimension.
ColOffsets: Maps columns of an Im2Col matrix to locations in the source image.
GemmExecutor: Executes matrix multiplication operations.
GemmOptions
GemmUninitOptions: Options for GemmExecutor::gemm_uninit and GemmExecutor::batched_gemm_uninit.
Im2Col: A matrix formed by unrolling patches of an image into columns.
PackedAMatrix: Left-hand or “A” GEMM input that has been pre-packed.
PackedBMatrix: Right-hand or “B” GEMM input that has been pre-packed.
QuantParams: Parameters required to perform matrix multiplication on quantized tensors.
RowOffsets: Maps rows of an Im2Col matrix to locations in the source image.

BiasVector: A bias to add to the output of a matrix multiplication.
BlockQuantizedError: Errors with block-quantized inputs.
ComputeMode: Specifies whether to quantize the LHS / “A” input to block-quantized matrix multiplication.
GemmError: Errors with matrix multiplication inputs.
GemmInputA: Left-hand or “A” input for a GEMM operation.
GemmInputB: Right-hand or “B” input for a GEMM operation.