tokenizer_py 0.2.0

crate with a tokenizer that works like a Python tokenizer
Documentation
1
2
3
max_width = 100

single_line_if_else_max_width = 100

chain_width = 100