Module inference 

Basic inference implementation Complete inference pipeline for text generation

This module ties together tokenization, model inference, and text generation into a complete pipeline that can process text input and generate responses.

Structs

InferenceMetrics

Performance metrics for inference

InferencePipeline

Complete inference pipeline

InferencePipelineBuilder

Builder for creating inference pipelines

Sampler

Sampling strategy for text generation