tiktoken 1.0.1

An efficient Rust implementation of tiktoken, enhanced with the capability to calculate OpenAI token costs. Crafted for performance, accuracy, and seamless integration into your projects.