Expand description


Tantivy is a search engine library. Think Lucene, but in Rust.

// First we need to define a schema ...

// `TEXT` means the field should be tokenized and indexed,
// along with its term frequency and term positions.
// `STORED` means that the field will also be saved
// in a compressed, row-oriented key-value store.
// This store is useful to reconstruct the
// documents that were selected during the search phase.
let mut schema_builder = Schema::builder();
let title = schema_builder.add_text_field("title", TEXT | STORED);
let body = schema_builder.add_text_field("body", TEXT);
let schema = schema_builder.build();

// Indexing documents

let index = Index::create_in_dir(index_path, schema.clone())?;

// Here we use a buffer of 100MB that will be split
// between indexing threads.
let mut index_writer = index.writer(100_000_000)?;

// Let's index one documents!
    title => "The Old Man and the Sea",
    body => "He was an old man who fished alone in a skiff in \
            the Gulf Stream and he had gone eighty-four days \
            now without taking a fish."

// We need to call .commit() explicitly to force the
// index_writer to finish processing the documents in the queue,
// flush the current index to the disk, and advertise
// the existence of new documents.

// # Searching

let reader = index.reader()?;

let searcher = reader.searcher();

let query_parser = QueryParser::for_index(&index, vec![title, body]);

// QueryParser may fail if the query is not in the right
// format. For user facing applications, this can be a problem.
// A ticket has been opened regarding this problem.
let query = query_parser.parse_query("sea whale")?;

// Perform search.
// `topdocs` contains the 10 most relevant doc ids, sorted by decreasing scores...
let top_docs: Vec<(Score, DocAddress)> =
    searcher.search(&query, &TopDocs::with_limit(10))?;

for (_score, doc_address) in top_docs {
    // Retrieve the actual content of documents given its `doc_address`.
    let retrieved_doc = searcher.doc(doc_address)?;
    println!("{}", schema.to_json(&retrieved_doc));

A good place for you to get started is to check out the example code ( literate programming / source code)


pub use time;
pub use crate::error::TantivyError;


WORM (Write Once Read Many) directory abstraction.
Definition of Tantivy’s errors and results.
Column oriented field storage for tantivy.
The fieldnorm represents the length associated to a given Field of a given document.
Defines tantivy’s merging strategy
Tantivy can (if instructed to do so in the schema) store the term positions in a given field. This positions are expressed as token ordinal. For instance, In “The beauty and the beast”, the term “the” appears in position 0 and position 4. This information is useful to run phrase queries.
Postings module (also called inverted index)
Query Module
Schema definition for tantivy’s indices.
Representations for the space usage of various parts of a Tantivy index.
Compressed/slow/row-oriented storage for documents.
The term dictionary main role is to associate the sorted Terms to a TermInfo struct that contains some meta-information about the term.
Tokenizer are in charge of chopping text into a stream of tokens ready for indexing.


doc! is a shortcut that helps building Document objects.


A date/time value with second precision.
DemuxMapping can be used to reorganize data from multiple segments.
DocAddress contains all the necessary information to identify a document given a Searcher object.
DocIdToSegmentOrdinal maps from doc_id within a segment to the new segment ordinal for demuxing.
Tantivy’s Document is the object that can be indexed and then searched for.
FutureResult is a handle that makes it possible to wait for the completion of an ongoing task.
Search Index
IndexBuilder can be used to create an index.
Meta information about the Index.
IndexReader is your entry point to read and search the index.
Search Index Settings.
Settings to presort the documents in an index
IndexWriter is the user entry-point to add document to an index.
The Inventory register and keeps track of all of the objects alive.
The inverted index reader is in charge of accessing the inverted index associated to a specific field.
A LeasedItem holds an object borrowed from a Pool.
A prepared commit
Holds a list of SegmentReaders ready for search.
Identifies the searcher generation accessed by a Searcher.
A segment is a piece of the index.
Uuid identifying a segment.
SegmentMeta contains simple meta information about a segment.
Entry point to access all of the datastructures of the Segment
Snippet Contains a fragment of a document, and some highlighed parts inside it.
Term represents the value that the token can take.
Your tracked object.
Structure version for the index.


Search executor whether search request are single thread or multithread.
The order to sort by
Defines when a new version of the index should be reloaded.
Enum describing each component of a tantivy segment. Each component is stored in its own file, using the pattern segment_uuid.component_extension, except the delete component that takes an segment_uuid.delete_opstamp.component_extension
UserOperation is an enum type that encapsulates other operation types.


Sentinel value returned when a DocSet has been entirely consumed.


Write-once read many (WORM) abstraction for where tantivy’s data should be stored.
Represents an iterable set of sorted doc ids.
Has length trait
Postings (also called inverted list)
Warmer can be used to maintain segment-level state e.g. caches.


Demux the segments according to demux_mapping. See DemuxMapping. The number of output_directories need to match max new segment ordinal from demux_mapping.
Maps a f64 to u64
Maps a i64 to u64
Reverse the mapping given by i64_to_u64.
Reverse the mapping given by i64_to_u64.
Expose the current version of tantivy as found in Cargo.toml during compilation. eg. “0.11.0” as well as the compression scheme used in the docstore.
Exposes the complete version of tantivy as found in Cargo.toml during compilation as a string. eg. “tantivy v0.11.0, index_format v1, store_compression: lz4”.

Type Definitions

A u32 identifying a document within a segment. Documents have their DocId assigned incrementally, as they are added in the segment.
A u64 assigned to every operation incrementally
Tantivy result.
A Score that represents the relevance of the document to the query
A SegmentOrdinal identifies a segment, within a Searcher or Merger.