Skip to main content

Crate searchlight

Crate searchlight 

Source
Expand description

§Searchlight - 高性能中英文全文搜索引擎

searchlight 是一个功能丰富的 Rust 全文搜索引擎库,支持:

  • 中英文分词: 中文 n-gram 分词 + 英文/数字分词
  • 倒排索引: 高效的检索数据结构
  • 布尔查询: AND/OR/NOT 组合查询
  • 模糊匹配: 基于编辑距离的容错搜索
  • 拼音搜索: 用拼音搜索中文内容
  • 精准短语搜索: 双引号短语匹配
  • BM25 排序: 专业的相关性排序算法
  • 搜索高亮: 结果自动生成高亮片段
  • 批量检索: 一次处理多个查询
  • 自动补全: 基于前缀的搜索建议

§快速开始

use searchlight::SearchEngine;

let mut engine = SearchEngine::new();

// 索引文档
engine.index("Rust 是一门现代系统编程语言,安全且高效");
engine.index("Go 语言以其简洁和并发性能著称");
engine.index("Python 是数据科学和 AI 领域的首选语言");

// 搜索
let results = engine.search("编程语言").unwrap();
for r in &results {
    println!("[{}] {}", r.score, r.snippet.as_deref().unwrap_or(&r.document));
}

// 拼音搜索
let results = engine.search_pinyin("biancheng").unwrap();

// 模糊搜索
let results = engine.search_fuzzy("programing", 2).unwrap();

Structs§

Bm25Params
BM25 ranking parameters
ExecutorContext
Context passed to the query executor.
ExecutorLimits
Limits applied during query evaluation.
FuzzyMatch
A fuzzy match result
Highlighter
HighlighterConfig
InvertedIndex
The inverted index: term -> list of (document_id, positions)
MatchReason
Why a document matched the query (retrieval-level explain for agents).
ParsedQuery
Parsed query with associated metadata
PinyinConverter
Convert Chinese text to pinyin tokens
PinyinIndex
Build a pinyin search index: maps pinyin representations to original terms
QueryEvaluation
Result of evaluating a query AST against the index.
QueryExecutor
Evaluates QueryOp AST with correct boolean semantics.
QueryParser
Query parser for the search engine. Supports:
Ranker
Rank documents using BM25 scoring with LRU-cached IDF and composite bonuses.
RelatedSuggestion
A deterministic related term suggestion derived from already indexed documents.
ScoreBreakdown
Per-component score breakdown (useful for debugging and agent explain).
ScoringInput
Input for the combined scoring pipeline.
SearchEngine
The main search engine.
SearchOptions
Search options for fine-grained control
SearchResult
A search result containing the matched document and metadata
Snippet
A highlighted snippet from a document
Token

Enums§

QueryOp
Represents different types of query operations
SearchlightError
Unified error type for searchlight operations.
TokenKind

Functions§

contains_chinese
damerau_levenshtein
Compute Damerau-Levenshtein distance (includes transpositions).
explain_document
Collect retrieval-level match reasons for a document against a query AST.
fuzzy_match
Find fuzzy matches for a query term from a candidate list, within max distance.
fuzzy_match_limited
Find fuzzy matches with a hard result cap. This prevents typo-search from collecting a huge result vector on large indexes.
fuzzy_match_with_prefix
Find fuzzy matches with prefix boost (matches starting with same prefix score higher)
is_fuzzy_match
Check if two strings are a fuzzy match within tolerance
jaccard_similarity
Compute Jaccard similarity between two token sets (for bag-of-words comparison)
lcs_similarity
Longest Common Subsequence ratio between two strings
levenshtein_distance
Compute Levenshtein (edit) distance between two strings. Distance = minimum number of single-character edits (insertions, deletions, substitutions) to change a into b.
pinyin_matches
Check if a pinyin query matches Chinese text
tokenize
Low-memory tokenizer for mixed Chinese / English text.
tokenize_chars
tokenize_ngrams

Type Aliases§

SearchlightResult