Modules§
- cache
- Cache for LLM answers to avoid redundant API calls Uses Blake3 hash of (query + context) as the key
Structs§
- Entity
Extraction Response - Response from entity extraction
- Extracted
Entity - Extracted entity from text
- Grounding
Result - Result of grounding/hallucination verification
- Model
Answer - Model
Context Fragment - Model
Inference - Prompt
Parts - Token
Usage - Token usage and cost information from LLM inference
Enums§
Constants§
- ENTITY_
EXTRACTION_ PROMPT - Default system prompt for entity extraction
Functions§
- calculate_
cost - Calculate cost for a given model based on token usage. Prices are per 1M tokens in USD (December 2025 pricing).
- extract_
entities - Extract entities from text using an LLM
- generate_
search_ query - Generate optimized search keywords from a question using LLM Returns the original question plus extracted search terms for better retrieval
- postprocess_
answer - Post-process the LLM answer for quality
- run_
model_ inference - verify_
grounding - Verify how well the answer is grounded in the provided context. Returns a GroundingResult with a score (0.0 to 1.0) indicating how well the answer is supported by the context.