pub struct CompiledAnalyzer { /* private fields */ }Expand description
A frozen pipeline with compiled expressions, fixed stop sets, and resolved synonym maps.
Implementations§
Source§impl CompiledAnalyzer
impl CompiledAnalyzer
pub fn descriptor(&self) -> &Arc<AnalyzerDescriptor> ⓘ
Sourcepub fn analyze_tokens(&self, text: &str) -> AnalysisResult<AnalyzedText>
pub fn analyze_tokens(&self, text: &str) -> AnalysisResult<AnalyzedText>
Analyze with independent output/source state and no expression compilation or file resolution.
Sourcepub fn analyze_tokens_budgeted(
&self,
text: &str,
budget: &MemoryBudget,
poll: impl FnMut() -> AnalysisResult<()>,
) -> AnalysisResult<Budgeted<AnalyzedText>>
pub fn analyze_tokens_budgeted( &self, text: &str, budget: &MemoryBudget, poll: impl FnMut() -> AnalysisResult<()>, ) -> AnalysisResult<Budgeted<AnalyzedText>>
Execute all prepared stages with one allowance for runtime buffers and retained output.
Immutable compiled resources and borrowed input have separate owners. Character maps, tokenization, common/Korean filters and source projections share this allowance. Errors return no partial output. Prepared regex automata retain analysis-owned search workspaces and poll during traversal and capture resolution.
use uqa_analysis::standard_analyzer;
use uqa_core::memory::MemoryBudget;
let compiled = standard_analyzer("english").compile()?;
let budget = MemoryBudget::new(64 * 1024);
let result = compiled.analyze_tokens_budgeted("The cats and", &budget, || Ok(()))?;
assert_eq!(result.tokens()[0].term(), "cat");
assert_eq!(result.final_position_increment(), 1);
drop(result);
assert_eq!(budget.used(), 0);pub fn analyze(&self, text: &str) -> AnalysisResult<Vec<String>>
Source§impl CompiledAnalyzer
impl CompiledAnalyzer
Sourcepub fn analyze_diagnostic_budgeted(
&self,
text: &str,
budget: &MemoryBudget,
poll: impl FnMut() -> AnalysisResult<()>,
) -> AnalysisResult<Budgeted<String>>
pub fn analyze_diagnostic_budgeted( &self, text: &str, budget: &MemoryBudget, poll: impl FnMut() -> AnalysisResult<()>, ) -> AnalysisResult<Budgeted<String>>
Analyze and encode the complete token graph, source end state, and immutable revision.
Analysis and JSON output share the supplied allowance. The token stream remains reserved until encoding finishes, and cancellation is checked while writing. The returned string retains its output reservation.