pub const EXTRACTION_MAX_TOKENS_DEFAULT: usize = 5_000;Expand description
Default cap on tokens fed to GLiNER NER per memory body.
v1.0.31: large markdown documents (>50 KB) tokenise into thousands of 512-token windows, each requiring a CPU forward pass that takes hundreds of milliseconds. A 68 KB document was observed taking 5+ minutes. Truncating the input before sliding-window construction caps the worst-case latency while preserving extraction quality for the leading body region.
Regex prefilter still runs on the full body, so URLs, emails, UUIDs, all-caps identifiers and CamelCase brand names are extracted regardless.