Struct CodeChunker 

pub struct CodeChunker { /* private fields */ }

Code-aware chunker that splits at function/class boundaries.

Supports multiple programming languages and falls back to line-based chunking for unknown languages.

Supported Languages

• Rust (.rs)
• Python (.py)
• JavaScript (.js, .jsx)
• TypeScript (.ts, .tsx)
• Go (.go)
• Java (.java)
• C/C++ (.c, .cpp, .h, .hpp)
• Ruby (.rb)
• PHP (.php)

Examples

use rlm_rs::chunking::{Chunker, CodeChunker, ChunkerMetadata};

let chunker = CodeChunker::new();
let code = r#"
fn main() {
    println!("Hello");
}

fn helper() {
    println!("Helper");
}
"#;

let meta = ChunkerMetadata::new().content_type("rs");
let chunks = chunker.chunk(1, code, Some(&meta)).unwrap();
assert!(!chunks.is_empty());

Implementations

impl CodeChunker

pub const fn new() -> Self

Creates a new code chunker with default settings.

pub const fn with_size(chunk_size: usize) -> Self

Creates a code chunker with custom chunk size.

pub const fn with_size_and_overlap(chunk_size: usize, overlap: usize) -> Self

Creates a code chunker with custom size and overlap.

Trait Implementations

impl Chunker for CodeChunker

fn chunk( &self, buffer_id: i64, text: &str, metadata: Option<&ChunkMetadata>, ) -> Result<Vec<Chunk>>

Chunks the input text into segments.

fn name(&self) -> &'static str

Returns the name of the chunking strategy.

fn description(&self) -> &'static str

Returns a description of the chunking strategy.

fn supports_parallel(&self) -> bool

Returns whether this chunker supports parallel processing.

fn validate(&self, metadata: Option<&ChunkMetadata>) -> Result<()>

Validates configuration before chunking.

impl Clone for CodeChunker

fn clone(&self) -> CodeChunker

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for CodeChunker

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for CodeChunker

fn default() -> Self

Returns the “default value” for a type.