pub struct RecursiveChunker { /* private fields */ }Available on crate feature
rag only.Expand description
Splits text hierarchically: paragraphs → sentences → words.
First splits by paragraph separators (\n\n). If a paragraph exceeds
chunk_size, splits by sentence boundaries (. , ! , ? ). If a
sentence still exceeds chunk_size, splits by word boundaries. Overlap
is applied between chunks at each level.
§Example
ⓘ
use adk_rag::RecursiveChunker;
let chunker = RecursiveChunker::new(512, 100);
let chunks = chunker.chunk(&document);Implementations§
Trait Implementations§
Source§impl Chunker for RecursiveChunker
impl Chunker for RecursiveChunker
Source§impl Clone for RecursiveChunker
impl Clone for RecursiveChunker
Source§fn clone(&self) -> RecursiveChunker
fn clone(&self) -> RecursiveChunker
Returns a duplicate of the value. Read more
1.0.0 · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
Performs copy-assignment from
source. Read moreAuto Trait Implementations§
impl Freeze for RecursiveChunker
impl RefUnwindSafe for RecursiveChunker
impl Send for RecursiveChunker
impl Sync for RecursiveChunker
impl Unpin for RecursiveChunker
impl UnsafeUnpin for RecursiveChunker
impl UnwindSafe for RecursiveChunker
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<T> FutureExt for T
impl<T> FutureExt for T
Source§fn with_context(self, otel_cx: Context) -> WithContext<Self>
fn with_context(self, otel_cx: Context) -> WithContext<Self>
Source§fn with_current_context(self) -> WithContext<Self>
fn with_current_context(self) -> WithContext<Self>
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§fn in_current_span(self) -> Instrumented<Self> ⓘ
fn in_current_span(self) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§impl<T> IntoRequest<T> for T
impl<T> IntoRequest<T> for T
Source§fn into_request(self) -> Request<T>
fn into_request(self) -> Request<T>
Wrap the input message
T in a tonic::RequestCreates a shared type from an unshared type.