[−][src]Crate notmecab
Structs
| Dict | |
| LexerToken | |
| ParserToken |
Enums
| TokenType |
Functions
| parse | Tokenizes a string by creating a lattice of possible tokens over it and finding the lowest-cost path over that lattice. Returns a list of ParserToken and the cost of the tokenization. |
| parse_to_lexertokens | Tokenizes a char slice by creating a lattice of possible tokens over it and finding the lowest-cost path over that lattice. Returns a list of LexerTokens and the cost of the tokenization. |