get_model_chunk_config

Function get_model_chunk_config 

Source
pub fn get_model_chunk_config(model_name: Option<&str>) -> (usize, usize)
Expand description

Get model-specific chunk configuration (target_tokens, overlap_tokens) Balanced for precision vs context - larger models can handle bigger chunks but not too big