Skip to main content

semantic_tokens

Function semantic_tokens 

Source
pub fn semantic_tokens(prose: &str) -> Vec<String>
Expand description

Splits prose into the semantic token stream used by FORGE prompt budgets.