pub struct Processor { /* private fields */ }Expand description
Advanced processor for text and HTML content.
Implementations§
Source§impl Processor
impl Processor
Sourcepub fn new(config: ProcessorConfig) -> Self
pub fn new(config: ProcessorConfig) -> Self
Creates a new Processor with the given configuration.
Sourcepub fn process_html(&self, html: &str) -> Result<Vec<String>>
pub fn process_html(&self, html: &str) -> Result<Vec<String>>
Processes HTML content: sanitizes it to text and then chunks it.
Sourcepub fn chunk_text(&self, text: &str) -> Vec<String>
pub fn chunk_text(&self, text: &str) -> Vec<String>
Splits text into overlapping chunks based on the configuration. Tries to split on whitespace to preserve word boundaries.
Auto Trait Implementations§
impl Freeze for Processor
impl RefUnwindSafe for Processor
impl Send for Processor
impl Sync for Processor
impl Unpin for Processor
impl UnwindSafe for Processor
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
Source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§impl<T> IntoRequest<T> for T
impl<T> IntoRequest<T> for T
Source§fn into_request(self) -> Request<T>
fn into_request(self) -> Request<T>
Wrap the input message
T in a tonic::Request