Struct pragmatic_segmenter::Segmenter
source · pub struct Segmenter { /* private fields */ }
Expand description
Segmenter type. It stores the compilation results of regular expressions used internally by pragmatic-segmenter in memory.
use pragmatic_segmenter::Segmenter;
let segmenter = Segmenter::new()?;
let result: Vec<_> = segmenter.segment("Hi Mr. Kim. Let's meet at 3 P.M.").collect();
assert_eq!(result, vec!["Hi Mr. Kim. ", "Let's meet at 3 P.M."]);
Implementations§
source§impl Segmenter
impl Segmenter
sourcepub fn new() -> Result<Self, Box<dyn Error>>
pub fn new() -> Result<Self, Box<dyn Error>>
Create a new Segmenter instance. The regular expressions used internally by pragmatic-segmenter are compiled here.
use pragmatic_segmenter::Segmenter;
let segmenter = Segmenter::new()?;
sourcepub fn segment<'a>(
&'a self,
original_input: &'a str
) -> impl Iterator<Item = &'a str>
pub fn segment<'a>( &'a self, original_input: &'a str ) -> impl Iterator<Item = &'a str>
Separate sentences from given input. Although it is a function that returns an Iterator, not all processing is done by streaming. After pre-processing the entire input once, processing is performed for each sentence by streaming.
use pragmatic_segmenter::Segmenter;
let segmenter = Segmenter::new()?;
let mut iter = segmenter.segment("Hi Mr. Kim. Let's meet at 3 P.M.");
assert_eq!(iter.next(), Some("Hi Mr. Kim. "));
assert_eq!(iter.next(), Some("Let's meet at 3 P.M."));
assert_eq!(iter.next(), None);
Auto Trait Implementations§
impl RefUnwindSafe for Segmenter
impl Send for Segmenter
impl Sync for Segmenter
impl Unpin for Segmenter
impl UnwindSafe for Segmenter
Blanket Implementations§
source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere T: ?Sized,
source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more