chatpack

Rust library for converting chat exports into compact, LLM- and RAG-ready data.

API Docs | Export Guide | Benchmarks | Website

Overview

chatpack is the core Rust crate behind the Chatpack ecosystem. It parses chat exports from Telegram, WhatsApp, Instagram, and Discord, normalizes them into one Message type, and writes token-efficient CSV, JSON, or JSONL output for LLM analysis, RAG ingestion, archival, and analytics.

Raw messenger exports often spend most of their tokens on nested JSON, repeated field names, and metadata. In a real Telegram sample with 34,478 messages, CSV output reduced 11.2M raw-export tokens to about 850K tokens: 13.2x smaller.

Install

cargo add chatpack

Minimal builds can opt into only the parsers and writers they need:

[dependencies]
chatpack = { version = "0.6.0", default-features = false, features = ["telegram", "csv-output"] }

Quick Start

use std::path::Path;

use chatpack::prelude::*;

fn main() -> Result<()> {
    let parser = create_parser(Platform::Telegram);
    let messages = parser.parse(Path::new("result.json"))?;

    let filtered = apply_filters(messages, &FilterConfig::new().with_sender("Alice"));
    let merged = merge_consecutive(filtered);

    write_to_format(
        &merged,
        "chat.jsonl",
        OutputFormat::Jsonl,
        &OutputConfig::new().with_timestamps(),
    )?;

    Ok(())
}

Common Workflows

Parse from a string when the export is already in memory:

use chatpack::prelude::*;

fn main() -> Result<()> {
    let parser = create_parser(Platform::WhatsApp);
    let messages = parser.parse_str("[1/15/24, 10:30 AM] Alice: Hello")?;

    println!("Parsed {} message(s)", messages.len());
    Ok(())
}

Stream large files when loading the full export is not practical:

use std::path::Path;

use chatpack::prelude::*;

fn main() -> Result<()> {
    let parser = create_streaming_parser(Platform::Telegram);

    for result in parser.stream(Path::new("huge_result.json"))? {
        let message = result?;
        println!("{}: {}", message.sender, message.content);
    }

    Ok(())
}

Choose output based on the downstream task:

Optional metadata is controlled by OutputConfig:

let compact = OutputConfig::new();
let detailed = OutputConfig::all();
let timestamps_only = OutputConfig::new().with_timestamps();

Feature Flags

The default feature set is full, which enables every parser, CSV/JSON output, and streaming support.

Documentation

Related Tools

This repository is the Rust core library. Other Chatpack tools live separately:

Development

cargo fmt --all -- --check
cargo test --all-features
cargo clippy --all-targets -- -D warnings
cargo bench --bench parsing

The crate uses Rust 2024 edition, so Rust 1.85 or newer is required. CI currently builds and tests on stable Rust across Linux, macOS, and Windows.