Skip to main content

StructuredMatcher

Struct StructuredMatcher 

Source
pub struct StructuredMatcher { /* private fields */ }
Expand description

A mutable matcher with independent sequence state.

Implementations§

Source§

impl StructuredMatcher

Source

pub fn new( program: Arc<StructuredProgram>, ) -> Result<Self, StructuredMatcherError>

Constructs a matcher from an already compiled program.

Source

pub fn backend_name(&self) -> &'static str

Runtime backend selected for this matcher.

Source

pub fn is_accepting(&self) -> bool

Whether the current state accepts a complete document.

Source

pub fn is_dead(&self) -> bool

Whether the current state is a permanent dead end.

Whether end-of-stream is legal at the current state.

Source

pub fn retained_session_bytes(&self) -> usize

Bytes currently retained by this sequence’s mutable backend state.

Source

pub fn peak_session_bytes(&self) -> usize

Peak charged session bytes observed by this matcher.

Source

pub fn advance( &mut self, token_bytes: &[u8], ) -> Result<bool, StructuredMatcherError>

Atomically commits a whole tokenizer token.

Source

pub fn allowed_mask_from_records<'a>( &mut self, mask_vocab_size: usize, records: impl Iterator<Item = (&'a [u8], &'a [u32])>, ) -> Result<Bitmask, StructuredMatcherError>

Computes a mask by speculatively feeding each candidate through one reusable state.

Source

pub fn write_record_mask_words_into<'a>( &mut self, mask_vocab_size: usize, records: impl Iterator<Item = (&'a [u8], &'a [u32])>, out: &mut [u32], ) -> Result<(), StructuredMatcherError>

Clears and fills a packed u32-word mask by scanning vocabulary records, without an intermediate byte buffer. O(1) additional allocation beyond out.

Source

pub fn write_record_mask_le_bytes_into<'a>( &mut self, mask_vocab_size: usize, records: impl Iterator<Item = (&'a [u8], &'a [u32])>, out: &mut [u8], ) -> Result<(), StructuredMatcherError>

Clears and fills a little-endian packed mask by scanning vocabulary records.

Source

pub fn write_mask_le_bytes_into( &mut self, vocabulary: &VocabularyHandle, trie: Option<&BoundByteTrie>, out: &mut [u8], ) -> Result<(), StructuredMatcherError>

Clears and fills a little-endian packed mask without allocating a temporary Bitmask.

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, !>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.