Module tantivy::store

Compressed/slow/row-oriented storage for documents.

A field needs to be marked as stored in the schema in order to be handled in the Store.

Internally, documents (or rather their stored fields) are serialized to a buffer. When the buffer exceeds block_size (defaults to 16K), the buffer is compressed using LZ4 or Zstd and the resulting block is written to disk.

One can then request for a specific DocId. A skip list helps navigating to the right block, decompresses it entirely and returns the document within it.

If the last document requested was in the same block, the reader is smart enough to avoid decompressing the block a second time, but their is no real uncompressed block* cache.

A typical use case for the store is, once the search result page has been computed, returning the actual content of the 10 best document.

Usage

Most users should not access the StoreReader directly and should rely on either

• at the segment level, the SegmentReader’s doc method
• at the index level, the Searcher::doc() method

Structs

• CacheStats
  CacheStats for the StoreReader.
• StoreReader
  Reads document off tantivy’s Store
• StoreWriter
  Write tantivy’s Store
• ZstdCompressor
  The Zstd compressor, with optional compression level.

Enums

• Compressor
  Compressor can be used on IndexSettings to choose the compressor used to compress the doc store.
• Decompressor
  Decompressor is deserialized from the doc store footer, when opening an index.