# uqa-analysis
`uqa-analysis` is the UQA Engine crate for tokenizers, character filters, token filters, and analyzers.
The optional `nori` feature exposes a validated immutable Korean dictionary, user-rule compiler, and native rolling Viterbi tokenizer with lossless UTF-16 morphology and graph attributes. `nori-tools` adds the offline packer and complete neutral-model verifier. The standalone tokenizer, default Korean analyzer, POS/reading/simple-lowercase filters, optional exact-decimal number composition, and separate normalization are verified against pinned Lucene fixtures. The native common-token bridge preserves raw terms, morphology, graph/end state, and corrected source spans through generic filters. Korean filters share one implementation across native and common tokens, including absent morphology, retained source provenance, and exact composed spans. `NoriResources` resolves immutable bundles by exact artifact hash, shares bounded dictionary and user-rule caches, and uses the bundled data through the optional `nori` feature without implicit I/O. `Analyzer::compile()` prepares immutable existing pipelines with fixed expressions, stop sets, and synonym maps while uncompiled APIs keep file reload behavior. `AnalyzerDescriptor` snapshots canonical resolved inputs and runtime profiles under a versioned fingerprint; `AnalyzerResources` restores those snapshots and shares compiled revisions under bounded cache ownership. Korean tokenizer/filter configurations now compile through the common pipeline with exact resource identities, user-rule snapshots, normalization, and bounded shared ownership. Memory, SQLite, and redb retain exact durable analyzer revisions and complete token graphs; public SQL and bindings execute graph phrases and highlight original source spans. Ported code retains the Lucene license, notice, and modification attribution in `THIRD-PARTY/`. See the [analyzer reference](https://github.com/cognica-io/uqa-engine/blob/main/docs/manual/reference/06-text-analyzers.md#standalone-korean-tokenization) and [bundle format](https://github.com/cognica-io/uqa-engine/blob/main/docs/design/nori-bundle-format.md).
Applications should depend on `uqa-engine` or `uqa-client`. See the [repository README](https://github.com/cognica-io/uqa-engine) and the [manual](https://github.com/cognica-io/uqa-engine/blob/main/docs/manual/README.md).