ib_matcher/

lib.rs

/*!
A multilingual, flexible and fast string, glob and regex matcher. Support 拼音匹配 (Chinese pinyin match) and ローマ字検索 (Japanese romaji match).

## Features
- Unicode support
  - Fully UTF-8 support and limited support for UTF-16 and UTF-32.
  - Unicode case insensitivity ([simple case folding](https://docs.rs/ib-unicode/latest/ib_unicode/case/#case-folding)).
- [Chinese pinyin](https://en.wikipedia.org/wiki/Pinyin) matching (拼音匹配)
  - Support characters with multiple readings (i.e. heteronyms, 多音字).
  - Support multiple pinyin notations, including [Quanpin (全拼)](https://zh.wikipedia.org/wiki/全拼), [Jianpin (简拼)](https://zh.wikipedia.org/wiki/简拼) and many [Shuangpin (双拼)](https://zh.wikipedia.org/wiki/%E5%8F%8C%E6%8B%BC) notations.
  - Support mixing multiple notations during matching.
- [Japanese romaji](https://en.wikipedia.org/wiki/Romanization_of_Japanese) matching (ローマ字検索)
  - Support characters with multiple readings (i.e. heteronyms, 同形異音語).
  - Support [Hepburn romanization system](https://en.wikipedia.org/wiki/Hepburn_romanization) only at the moment.
- [glob()-style](syntax::glob) pattern matching (i.e. `?`, `*`, `[]` and `**`)
  - Support [different anchor modes](syntax::glob#anchor-modes), [treating surrounding wildcards as anchors](syntax::glob#surrounding-wildcards-as-anchors) and [special anchors in file paths](syntax::glob#anchors-in-file-paths).
*/
//!   - Support two seperators (`//`) or a complement separator (`\`) as a glob star (`*/**`).
/*!
- [Regular expression](regex)
  - Support the same syntax as [`regex`](https://docs.rs/regex/), including wildcards, repetitions, alternations, groups, etc.
  - Support [custom matching callbacks](regex::cp::Regex#custom-matching-callbacks), which can be used to implement ad hoc look-around, backreferences, balancing groups/recursion/subroutines, combining domain-specific parsers, etc.
- Relatively high performance
  - Generally on par with the `regex` crate, depending on the case it can be faster or slower.

And all of the above features are optional. You don't need to pay the performance and binary size cost for features you don't use.

You can also use [ib-pinyin](https://docs.rs/ib-pinyin/) if you only need Chinese pinyin match, which is simpler and more stable.

## Usage
```
// cargo add ib-matcher --features pinyin,romaji
use ib_matcher::matcher::{IbMatcher, PinyinMatchConfig, RomajiMatchConfig};

let matcher = IbMatcher::builder("la vie est drôle").build();
assert!(matcher.is_match("LA VIE EST DRÔLE"));

let matcher = IbMatcher::builder("βίος").build();
assert!(matcher.is_match("Βίοσ"));
assert!(matcher.is_match("ΒΊΟΣ"));

let matcher = IbMatcher::builder("pysousuoeve")
    .pinyin(PinyinMatchConfig::default())
    .build();
assert!(matcher.is_match("拼音搜索Everything"));

let matcher = IbMatcher::builder("konosuba")
    .romaji(RomajiMatchConfig::default())
    .is_pattern_partial(true)
    .build();
assert!(matcher.is_match("この素晴らしい世界に祝福を"));
```
See also [choosing a matcher](#choosing-a-matcher).

## glob()-style pattern matching
See [`glob` module](syntax::glob) for more details. Here is a quick example:
```
// cargo add ib-matcher --features syntax-glob,regex,romaji
use ib_matcher::{
    matcher::MatchConfig,
    regex::lita::Regex,
    syntax::glob::{parse_wildcard_path, PathSeparator}
};

let re = Regex::builder()
    .ib(MatchConfig::builder().romaji(Default::default()).build())
    .build_from_hir(
        parse_wildcard_path()
            .separator(PathSeparator::Windows)
            .call("wifi**miku"),
    )
    .unwrap();
assert!(re.is_match(r"C:\Windows\System32\ja-jp\WiFiTask\ミク.exe"));
```

## Regular expression
See [`regex`] module for more details. Here is a quick example:
```
// cargo add ib-matcher --features regex,pinyin,romaji
use ib_matcher::{
    matcher::{MatchConfig, PinyinMatchConfig, RomajiMatchConfig},
    regex::{cp::Regex, Match},
};

let config = MatchConfig::builder()
    .pinyin(PinyinMatchConfig::default())
    .romaji(RomajiMatchConfig::default())
    .build();

let re = Regex::builder()
    .ib(config.shallow_clone())
    .build("raki.suta")
    .unwrap();
assert_eq!(re.find("「らき☆すた」"), Some(Match::must(0, 3..18)));

let re = Regex::builder()
    .ib(config.shallow_clone())
    .build("pysou.*?(any|every)thing")
    .unwrap();
assert_eq!(re.find("拼音搜索Everything"), Some(Match::must(0, 0..22)));

let config = MatchConfig::builder()
    .pinyin(PinyinMatchConfig::default())
    .romaji(RomajiMatchConfig::default())
    .mix_lang(true)
    .build();
let re = Regex::builder()
    .ib(config.shallow_clone())
    .build("(?x)^zangsounofuri-?ren # Mixing pinyin and romaji")
    .unwrap();
assert_eq!(re.find("葬送のフリーレン"), Some(Match::must(0, 0..24)));
```

[Custom matching callbacks](regex::cp::Regex#custom-matching-callbacks):
```
// cargo add ib-matcher --features regex,regex-callback
use ib_matcher::regex::cp::Regex;

let re = Regex::builder()
    .callback("ascii", |input, at, push| {
        let haystack = &input.haystack()[at..];
        if haystack.len() > 0 && haystack[0].is_ascii() {
            push(1);
        }
    })
    .build(r"(ascii)+\d(ascii)+")
    .unwrap();
let hay = "that4Ｕ this4me";
assert_eq!(&hay[re.find(hay).unwrap().span()], " this4me");
```

## Choosing a matcher
Use [`matcher::IbMatcher`] if:
- You only need plain text matching, optionally with Unicode case insensitivity, Chinese pinyin match and Japanese romaji match.

Use [`regex::lita::Regex`] if:
- You need [`regex`] or [`glob`](syntax::glob) syntax.
- You want high performance (and don't mind some binary footprint).

  [`regex::lita::Regex`] can be much faster than [`regex::cp::Regex`], and slightly faster than the `regex` crate (due to enum dispatch) if the following conditions are met:
  - Your pattern is often a literal string (i.e. plain text, optionally with pinyin/romaji match).
  - A fair portion of your haystacks is ASCII-only.

  A typical use case that meets the above conditions is matching file names and paths.

Use [`regex::cp::Regex`] if:
- You need [`regex`] or [`glob`](syntax::glob) syntax.
- You need `find_iter()` or `captures_iter()`.
- You need `build_many()`.
- You need [custom matching callbacks](regex::cp::Regex#custom-matching-callbacks).
- You want smaller binary size and don't very mind about the performance.
*/
//! ## Performance
//! The following `Cargo.toml` settings are recommended if best performance is desired:
//! ```toml
//! [profile.release]
//! lto = "fat"
//! codegen-units = 1
//! ```
//! These can improve the performance by 5~10% at most.
//!
//! ## Crate features
#![cfg_attr(docsrs, feature(doc_auto_cfg))]
#![cfg_attr(feature = "doc", doc = document_features::document_features!())]

extern crate alloc;

pub mod matcher;
#[cfg(feature = "minimal")]
pub mod minimal;
#[cfg(feature = "pinyin")]
pub mod pinyin;
#[cfg(feature = "regex-automata")]
pub mod regex;
#[cfg(any(
    feature = "syntax-glob",
    feature = "syntax-ev",
    feature = "syntax-regex"
))]
pub mod syntax;

#[cfg(feature = "romaji")]
pub use ib_romaji as romaji;
pub use ib_unicode as unicode;

mod private {
    pub trait Sealed {}
}
use private::Sealed;

#[cfg(test)]
mod tests {
    use crate::{
        matcher::{MatchConfig, PinyinMatchConfig, RomajiMatchConfig},
        regex::{cp::Regex, Match},
    };

    #[test]
    fn regex() {
        let config = MatchConfig::builder()
            .pinyin(PinyinMatchConfig::default())
            .romaji(RomajiMatchConfig::default())
            .build();

        let re = Regex::builder()
            .ib(config.shallow_clone())
            .build("raki.suta")
            .unwrap();
        assert_eq!(re.find("「らき☆すた」"), Some(Match::must(0, 3..18)));

        let re = Regex::builder()
            .ib(config.shallow_clone())
            .build("pysou.*?(any|every)thing")
            .unwrap();
        assert_eq!(re.find("拼音搜索Everything"), Some(Match::must(0, 0..22)));

        let config = MatchConfig::builder()
            .pinyin(PinyinMatchConfig::default())
            .romaji(RomajiMatchConfig::default())
            .mix_lang(true)
            .build();
        let re = Regex::builder()
            .ib(config.shallow_clone())
            .build("(?x)^zangsounofuri-?ren # Mixing pinyin and romaji")
            .unwrap();
        assert_eq!(re.find("葬送のフリーレン"), Some(Match::must(0, 0..24)));
    }
}