Skip to main content

Crate llm_token_visualizer

Crate llm_token_visualizer 

Source

Re-exports§

pub use data::ConfidenceLevel;
pub use data::FlagType;
pub use data::TokenAnalysis;
pub use data::TokenFlag;
pub use data::TokenInfo;
pub use data::VisualizationConfig;
pub use renderer::HtmlRenderer;
pub use renderer::MarkdownRenderer;
pub use renderer::Renderer;
pub use renderer::TerminalRenderer;
pub use utils::create_mock_analysis;
pub use utils::detect_issues;
pub use utils::simple_tokenize;
pub use utils::AnalysisMetrics;

Modules§

data
detect
Hallucination-risk detection from token log probabilities.
live
Fetch a completion with token logprobs from an OpenAI-compatible API.
renderer
report
Reports for detect mode: a confidence heatmap of the answer, the flagged spans, and the alternatives the model weighed at each weak token.
utils

Functions§

analyze_with_issues
Comprehensive analysis with issue detection
quick_analyze
Quick analysis function for testing - creates mock data and visualizes
visualize_tokens
Main visualization function that can be used by other applications