Struct AnalyzedText 

pub struct AnalyzedText { /* private fields */ }

A complete analyzed input with explicit token graph and source end state.

Implementations

impl AnalyzedText

pub fn clone_budgeted( &self, budget: &MemoryBudget, poll: impl FnMut() -> AnalysisResult<()>, ) -> AnalysisResult<Budgeted<Self>>

Copy owned token buffers into this allowance while sharing retained source allocations with the original.

Every term, morphology value, token buffer and hidden terminal token receives its own reservation. Existing source projection and mapping leases remain shared with the input. Failure leaves the borrowed result unchanged.

use uqa_analysis::Tokenizer;
use uqa_core::memory::MemoryBudget;

let budget = MemoryBudget::new(64 * 1024);
let input = Tokenizer::Whitespace.tokenize_with_offsets_budgeted(
    "韓🙂 UQA", &budget, || Ok(()),
)?;
let copied = input.clone_budgeted(&budget, || Ok(()))?;
assert_eq!(*copied, *input);
drop(input);
assert!(budget.used() >= copied.reserved_bytes());
drop(copied);
assert_eq!(budget.used(), 0);

impl AnalyzedText

pub fn tokens(&self) -> &[AnalysisToken]

pub fn into_tokens(self) -> Vec<AnalysisToken>

pub fn into_terms(self) -> AnalysisResult<Vec<String>>

pub fn final_offsets(&self) -> &SourceOffsets

pub fn final_position_increment(&self) -> u32

Trait Implementations

impl Clone for AnalyzedText

fn clone(&self) -> AnalyzedText

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for AnalyzedText

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for AnalyzedText

impl PartialEq for AnalyzedText

fn eq(&self, other: &AnalyzedText) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl Serialize for AnalyzedText

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for AnalyzedText