Module types 

Type definitions Core types for UniLLM runtime

This module defines fundamental data structures used throughout the runtime.

Structs

GenerationStats

Generation statistics

InferenceInputs

Inference inputs

InferenceOutput

Inference output

MemoryRequirements

Memory requirements for a model

ModelConfig

Model configuration

ModelOutput

Model output structure

ModelWeightMetadata

Model weight metadata

ModelWeights

Container for loaded model weights

PreparedInputs

Prepared inputs for model execution

Tensor

Tensor structure with real data storage

WeightMetadata

Metadata about loaded weights

Enums

DataType

Data types supported by tensors

Device

Device types for tensor placement

ModelError

Error types for model operations

ModelFeature

Model features that can be supported

ModelFormat

Model format types

ModelInputs

Model inputs for forward pass

ModelOutputs

Model outputs from forward pass

ModelPrecision

Model precision types

NormalizationType

Normalization types

PositionEmbeddingType

Position embedding types

TensorData

Tensor data storage

WeightFormat

Supported weight file formats for model loading

Functions

calculate_strides

Calculate strides for a given shape (row-major order)

Type Aliases

ModelResult

Result type for model operations