Skip to main content

Module utils

Module utils 

Source
Expand description

Shared utilities for LLM request/response processing

This module provides common functions used across multiple providers to eliminate duplicate code and reduce allocations.

Functionsยง

estimate_token_count
Count the number of tokens in text using tiktoken BPE.
parse_stream_event_openai_format
Parse stream event from OpenAI-compatible format
truncate_to_token_limit
Truncate text to at most max_tokens tokens.