Module inference 

Inference optimization utilities

This module provides optimizations for model inference including:

• Operator fusion
• Constant folding
• Memory optimization
• Batch inference

Structs

BatchInference

Batch inference helper

FusedOp

Fused operation

InferenceConfig

Inference mode configuration

InferenceOptimizer

Inference optimizer

InferenceSession

Inference session for optimized model execution

Functions

warmup_model

Warmup helper for inference