Crate chatpack 

Chatpack

A Rust library for parsing and converting chat exports from popular messaging platforms into LLM-friendly formats.

Overview

Chatpack provides a unified API for working with chat exports from:

• Telegram - JSON exports from Telegram Desktop
• WhatsApp - Text exports (both iOS and Android formats)
• Instagram - JSON exports from Instagram data download
• Discord - JSON/TXT/CSV exports from DiscordChatExporter

The library handles the complexity of different export formats and provides tools for filtering, merging, and outputting messages in formats optimized for use with Large Language Models.

Feature Flags

Chatpack uses feature flags to minimize dependencies:

Feature	Description	Dependencies
telegram	Telegram parser	serde_json
whatsapp	WhatsApp parser	regex
instagram	Instagram parser	serde_json
discord	Discord parser	serde_json, regex, csv
csv-output	CSV output writer	csv
json-output	JSON/JSONL output writers	serde_json
streaming	Streaming parsers for large files	(none)
full	Everything (default)	all above

Quick Start

The parser module provides a unified API with streaming support:

use chatpack::parser::{Parser, Platform, create_parser};
use chatpack::prelude::*;

// Parse a Telegram export
let parser = create_parser(Platform::Telegram);
let messages = parser.parse("telegram_export.json".as_ref())?;

// Merge consecutive messages from the same sender
let merged = merge_consecutive(messages);

// Write to JSON
write_json(&merged, "output.json", &OutputConfig::new())?;

Streaming for Large Files

For files larger than 1GB, use the streaming API to avoid memory issues:

use chatpack::parser::{Parser, Platform, create_streaming_parser};

let parser = create_streaming_parser(Platform::Telegram);

// Process messages one at a time
for result in parser.stream("huge_export.json".as_ref())? {
    if let Ok(msg) = result {
        println!("{}: {}", msg.sender, msg.content);
    }
}

Module Structure

• parser - Unified parser API (recommended)
  • Parser - Unified parser trait with streaming
  • Platform - Supported platforms enum
  • create_parser, create_streaming_parser
• config - Parser configuration types
  • TelegramConfig, WhatsAppConfig, etc.
• core - Core types and functionality
  • core::models - Message, [OutputConfig]
  • core::filter - [FilterConfig], [apply_filters]
  • core::processor - [merge_consecutive], [ProcessingStats]
  • core::output - [write_json], [write_jsonl], [write_csv]
• parsers - Platform-specific parser implementations
  • [TelegramParser], [WhatsAppParser], [InstagramParser], [DiscordParser]
• streaming - Streaming parsers for large files (requires streaming feature)
  • [TelegramStreamingParser], [DiscordStreamingParser]
• [format] - Output format types
  • OutputFormat, write_to_format
• progress - Progress reporting for long-running operations
  • Progress, ProgressCallback
• error - Unified error types (ChatpackError, Result)
• prelude - Convenient re-exports

Re-exports

pub use error::ChatpackError;

pub use error::Result;

pub use message::Message;

Modules

config

Configuration types for parsers and output.

core

Core processing logic for chatpack.

error

Unified error types for chatpack.

format

Output format types for the chatpack library.

message

Core message type for chatpack.

parser

Unified parser trait for chat exports.

parsers

Chat export parsers for various platforms.

parsing

Shared parsing utilities for all platforms.

prelude

Convenient re-exports for common usage.

progress

Progress reporting types for long-running operations.

streaming

Streaming parsers for memory-efficient processing of large chat exports.