Expand description
Extracts text from RAW TEXT layer files, constructs a 9-depth block hierarchy (identity → layers → clusters → items → sentences → tokens → syllables → chars → bytes), and writes the binary output files (microscope.bin, data.bin, meta.bin, merkle.bin, embeddings.bin).
Structs§
Functions§
- build
- Rebuild the binary index from the raw layer files.
- compute_
layers_ hash - maybe_
auto_ rebuild - Consolidate the append log automatically when the configured threshold is reached, or immediately when no main index exists yet.
- rebuild_
pending - Rebuild the main index while holding the same process lock used by stores. The append log is removed only after a successful build.