mdictlib
mdictlib is a safe, library-first Rust reader for MDict .mdx text
dictionaries and .mdd resource dictionaries.
Design
- MDX and MDD use one shared, file-backed parsing core.
- Opening reads bounded metadata and block indexes; key and record blocks stay lazy.
- Every physical key has a
KeyOrdinal, so duplicates retain distinct identities and payloads. - Lookup is global and deterministic: all raw-exact matches win before any header-normalized fallback.
- Public iterators are opaque and fused; wire-format modules remain private.
- MDD spans are source-bound and can stream without exposing raw offsets or materializing a whole resource.
- Per-open
Limitsand aggregate working-memory accounting bound untrusted reads, decompression, allocation, locator construction, and materialization. - Unsafe code is forbidden.
Supported Scope
- MDict major version 2 MDX and MDD sections
- UTF-8, UTF-16LE, GBK/GB2312, GB18030, and Big5 text decoding
- uncompressed and zlib blocks
- optional pure-Rust LZO through the
lzofeature - keyword-index encryption
- passcode-protected keyword-header encryption
- lazy iteration, physical-ordinal access, and duplicate-aware lookup
Independent full-file tests cover every listed encoding and compression path, both encryption modes, multi-block boundaries, duplicate keys, corruption, and hostile declarations. Version 1.x/future-major layouts, writing, HTML/style processing, resource extraction policy, multi-volume discovery, prefix/fuzzy search, mmap, and persistent sidecars are out of scope.
Using mdictlib
[]
= "0.1.0"
Enable LZO when required by a dictionary:
[]
= { = "0.1.0", = ["lzo"] }
MDX
use ;
locate() searches every raw key first. Only a global raw miss enables the
header-controlled normalized index. KeyMatches preserves every matching
ordinal in physical order and reports RawExact or HeaderNormalized.
MDD
Materialize a bounded resource:
use MddFile;
Or resolve a source-bound span and stream it:
use MddFile;
Streaming is not subject to the whole-resource materialization ceiling; individual decoded record blocks and aggregate parser work remain bounded.
Physical Keys And Ordinals
keys() returns original keys in physical file order without reading record
payloads:
use ;
Ordinals are stable only for the same unchanged dictionary file snapshot.
Limits And Memory Diagnostics
use ;
MemoryUsage values are conservative parser budget estimates, not allocator or
operating-system RSS. Payloads already returned to the caller are excluded.
Encrypted Keyword Headers
use ;
Passcode inputs are validated before cloning and Debug output is redacted.
Independent tests generate and open header-encrypted, index-encrypted, and
combined encrypted dictionaries end to end.
Development Checks
RUSTDOCFLAGS='-D warnings -D missing_docs -D rustdoc::broken_intra_doc_links' \
CARGO_NET_OFFLINE=true
The committed workflow runs default/all-feature tests on Linux, macOS, and Windows. Linux also runs formatting, Clippy, strict rustdoc, all seven bounded AddressSanitizer/coverage-guided fuzz smoke targets, and offline package verification. It pins cargo-fuzz 0.13.2 and nightly-2026-08-09.
Reproducible Private-Corpus Checks
Private dictionary bytes are not stored here. Set MDICT_CORPUS_DIR to an
authorized corpus directory and put mdictlib-corpus.tsv at its root. Start
from tests/corpus-manifest.example.tsv.
Rows contain:
path kind bytes sha256 entries
Paths are normalized /-separated paths relative to the corpus root. kind is
mdx or mdd. Record byte counts, SHA-256 values, and entry counts
independently of the parser under test.
Explicit runs fail with setup instructions if the environment, manifest, or a declared asset is absent or wrong. The lookup regression checks every physical row, raw lookup, duplicate ordering, ordinal identity, and payload/span route. For the recorded release manifest, it also checks deidentified exact key and record/resource hashes keyed by the manifest digest.
Measurements
The release harness consumes the same verified manifest:
It reports open-to-first-lookup, metadata open, locator construction, warm
lookup distributions, full key scans, sequential and ordinal MDX payloads, MDD
streaming and materialization, concurrent first lookup, deterministic hashes,
and accounted memory. Optional controls are MDICT_BENCH_FILTER,
MDICT_BENCH_WARM_RUNS, and MDICT_BENCH_THREADS.
The checked-in 0.1.0 release evidence covers seven private v2 files,
3.61 GB, and 804,572 physical entries, including exhaustive raw lookup and
ordinal/payload round trips. Exact commands, hashes, timings, memory accounting,
and externally measured peak RSS are recorded in the repository at
.codex/benchmarks/2026-08-10-macos-arm64.md.
License
MIT. See LICENSE.