Crate sentence

Sentence, a library for lexing English language sentences into tokens for TTS purposes.

Usage:

use sentence::{SentenceTokenizer, Token, Punctuation};
let tokenizer = SentenceTokenizer::new();
let tokens = tokenizer.tokenize("Hello, world!");
assert_eq!(tokens, vec![
  Token::Word("Hello".into()),
  Token::Punctuation(Punctuation::Comma),
  Token::Word("world".into()),
  Token::Punctuation(Punctuation::Exclamation),
]);

Structs

SentenceTokenizer

Sentence Tokenizer.

Enums

Punctuation	Punctuation marks
Token	A parsed token