Module logprobs 

Module for recording logprobs from a streaming response.

Logprobs are a bit easier than token counting and timing because they are fully self-contained in the response chunk.

In fact, if logprobs are given, they are a good way to count tokens; however, the emission of logprobs is also more costly and generally not available unless explicitly requested.

The primary reason to record logprobs is to analyze the possible outputs of a model as a function of sequence position.

Structs

ChoiceAnalysis

Analysis for a single choice

ClosePosition

A position where top candidates have close probabilities

MultipleCloseTokens

Information about multiple close tokens at a position

PositionCloseness

Closeness information for a position

SensitivityAnalysis

Analysis focused on detecting close logprobs indicating model uncertainty

TokenLogProbs

Represents logprob information for a single position with selected and alternative tokens

TokenLogprob

Represents a token with its logprob information

Enums

LogprobType

The type of logprobs observed in the response.

Traits

LogprobExtractor

Trait for extracting logprob information from various response types

Functions

analyze_logprob_sensitivity

Analyzes logprobs from a recorded stream focusing on token similarity/closeness

validate_and_flatten_choices

Validate and flatten choice logprobs HashMap to Vec Ensures all expected choice indices [0, max_choice) are present