cang-jie 0.2.0

A Chinese tokenizer for tantivy failed to build cang-jie-0.2.0
Please check build logs and if you believe this is' fault, report into this issue report.

cang-jie(仓颉) Build Status latest document dependency status

A Chinese tokenizer for tantivy, based on jieba-rs.

As of now, only support UTF-8.


    let mut schema_builder = SchemaBuilder::default();
    let text_indexing = TextFieldIndexing::default()
        .set_tokenizer(CANG_JIE) // Set custom tokenizer
    let text_options = TextOptions::default()
    // ... Some code   
     let index = Index::create(RAMDirectory::create(), schema.clone())?;
     let tokenizer = CangJieTokenizer {
                        worker: Arc::new(Jieba::empty()), // empty dictionary
                        option: TokenizerOption::Unicode,
     index.tokenizers().register(CANG_JIE, tokenizer); 
    // ... Some code

Full example