Module utils 

Structs

AnalysisMetrics

Calculate various metrics from token analysis

Functions

create_mock_analysis

Create a basic token analysis from text with random confidence scores This is useful for testing when you don’t have actual LLM confidence data

detect_issues

Detect potential issues in token analysis

simple_tokenize

Simple tokenizer for demo purposes - splits on whitespace and punctuation