tokenizations 0.4.1

Tokenizations alignments library
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

# Publishment flow

## Rust

1. Fix version in `Cargo.toml` (e.g. 0.1.0)
1. Git tag version with prefix `rust/` (e.g. `git tag rust/0.1.0)
1. Push tag to master
1. CI automatically publish crates to crates.io after testing

## Python

1. Fix version in `python/pyproject.toml`, `python/Cargo.toml`, `python/src/lib.rs`
    - Easily done with [pyversionup](https://github.com/tamuhey/pyversionup): `versionup 0.1.0`
1. Git tag version with prefix `python/`
1. Push tag to master
1. CI automatically publish package to pypi after testing