Struct ChunkerConfig 

pub struct ChunkerConfig {
    pub target_tokens: usize,
    pub min_tokens: usize,
    pub max_tokens: usize,
    pub overlap_messages: usize,
    pub noise_filter_enabled: bool,
}

Configuration for the chunker.

Fields

target_tokens: usize

Target tokens per chunk (default: 1500)

min_tokens: usize

Minimum tokens — don’t create tiny chunks unless it’s the last window (default: 500)

max_tokens: usize

Maximum tokens — force split if exceeded (default: 2500)

overlap_messages: usize

Number of messages to overlap between consecutive windows (default: 2)

noise_filter_enabled: bool

Whether to strip structural noise (line-numbered grep matches, tool echoes, stray YAML delimiters) before signal/highlight extraction. Default: true. Set to false for debugging or when raw upstream content must be preserved verbatim.

Trait Implementations

impl Clone for ChunkerConfig

fn clone(&self) -> ChunkerConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ChunkerConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ChunkerConfig

fn default() -> Self

Returns the “default value” for a type.