[−][src]Crate notmecab
Structs
Blob | A blob of bytes. |
Dict | |
LexerToken | |
ParserToken |
Enums
TokenType |
Functions
parse | Tokenizes a string by creating a lattice of possible tokens over it and finding the lowest-cost path over that lattice. Returns a list of ParserToken and the cost of the tokenization. |
parse_to_lexertokens | Tokenizes a char slice by creating a lattice of possible tokens over it and finding the lowest-cost path over that lattice. Returns a list of LexerTokens and the cost of the tokenization. |