Struct FixedChunker 

pub struct FixedChunker { /* private fields */ }

Fixed-size chunker that splits text at character boundaries.

This is the simplest chunking strategy, splitting text into fixed-size segments with optional overlap. It ensures chunks never split multi-byte UTF-8 characters.

Examples

use rlm_rs::chunking::{Chunker, FixedChunker};

let chunker = FixedChunker::with_size(100);
let text = "Hello, world! ".repeat(20);
let chunks = chunker.chunk(1, &text, None).unwrap();
for chunk in &chunks {
    assert!(chunk.size() <= 100);
}

Implementations

impl FixedChunker

pub const fn new() -> Self

Creates a new fixed chunker with default settings.

pub const fn with_size(chunk_size: usize) -> Self

Creates a fixed chunker with custom chunk size and no overlap.

Arguments

• chunk_size - Target size for each chunk in characters.

pub const fn with_size_and_overlap(chunk_size: usize, overlap: usize) -> Self

Creates a fixed chunker with custom size and overlap.

Arguments

• chunk_size - Target size for each chunk in characters.
• overlap - Number of characters to overlap between chunks.

pub const fn line_aware(self, enabled: bool) -> Self

Sets whether to align chunks to line boundaries.

When enabled, chunks will end at the nearest newline before the target size (if one exists within a reasonable range).

Trait Implementations

impl Chunker for FixedChunker

fn chunk( &self, buffer_id: i64, text: &str, metadata: Option<&ChunkMetadata>, ) -> Result<Vec<Chunk>>

Chunks the input text into segments.

fn name(&self) -> &'static str

Returns the name of the chunking strategy.

fn description(&self) -> &'static str

Returns a description of the chunking strategy.

fn supports_parallel(&self) -> bool

Returns whether this chunker supports parallel processing.

fn validate(&self, metadata: Option<&ChunkMetadata>) -> Result<()>

Validates configuration before chunking.

impl Clone for FixedChunker

fn clone(&self) -> FixedChunker

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for FixedChunker

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for FixedChunker

fn default() -> Self

Returns the “default value” for a type.