Skip to main content

count_passage_tokens

Function count_passage_tokens 

Source
pub fn count_passage_tokens(text: &str) -> Result<usize, AppError>
Expand description

Returns the approximate token count for text when prefixed with prefix (e.g. passage: for embed_passage).