# hat-splitter
This is the home of the HAT splitting rule. We expose it as a Rust crate with
Python bindings so that the same splitting rule can be used in both languages.
This project is WIP. More information and documentation to follow.
## The plan
We've found that HAT models are very sensitive to their splitting rule. As a
result, the splitting rule implemented here must exactly match the behaviour of
the splitter we're currently using.
1. Create a simple placeholder text splitting implementation (e.g., just split
on whitespace).
2. Set up Python bindings with PyO3.
3. Add Scaling as a Python dev dep and test the Python bindings against the
existing splitting rule. Tests will fail.
4. Implement the HAT splitting rule in Rust and make tests green.
Once these basics are in place, we can start thinking about packaging and
publishing.
## Development
### Release process
1. Update the version in `Cargo.toml`. Commit and push to `main`.
2. Tag the commit with the new version, e.g., `git tag v0.1.0`.
3. Push the tag to the remote. CI will take care of the rest.