Crate chat_splitter

source ·
Expand description

Never exceed OpenAI’s chat models’ maximum number of tokens when using the async_openai Rust crate.

chat-splitter splits chats into ‘outdated’ and ‘recent’ messages. You can split by both maximum message count and maximum chat completion token count. We use tiktoken_rs for counting tokens.

Usage

Here’s a basic example:

ⓘ
// Get all your previously stored chat messages...
let mut stored_messages = /* get_stored_messages()? */;

// ...and split into 'outdated' and 'recent',
// where 'recent' always fits the context size.
let (outdated_messages, recent_messages) =
    ChatSplitter::default().split(&stored_messages);

For a more detailed example, see examples/chat.rs.

Contributing

Contributions to chat-splitter are welcome! If you find a bug or have a feature request, please submit an issue. If you’d like to contribute code, please feel free to submit a pull request.

Structs

Traits