chatpack 0.2.2

Compress chat exports from Telegram, WhatsApp, and Instagram into token-efficient CSV for LLMs
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144

# 📦 chatpack


> Feed your chat history to LLMs. Compress exports **13x** with CSV format.

[![CI](https://github.com/berektassuly/chatpack/actions/workflows/ci.yml/badge.svg)](https://github.com/berektassuly/chatpack/actions/workflows/ci.yml)
[![codecov](https://codecov.io/gh/berektassuly/chatpack/branch/main/graph/badge.svg)](https://codecov.io/gh/berektassuly/chatpack)
[![Crates.io](https://img.shields.io/crates/v/chatpack.svg)](https://crates.io/crates/chatpack)
[![Downloads](https://img.shields.io/crates/d/chatpack.svg)](https://crates.io/crates/chatpack)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

**Platforms:** Windows • macOS • Linux

## The Problem


You want to ask Claude/ChatGPT about your conversations, but:
- Raw exports are **80% metadata noise**
- JSON structure wastes tokens on brackets and keys
- Context windows are expensive

## The Solution


```
┌─────────────────┐     ┌──────────┐     ┌─────────────────┐
│ Telegram JSON   │     │          │     │ Clean CSV       │
│ WhatsApp TXT    │ ──▶ │ chatpack │ ──▶│ Ready for LLM   │
│ Instagram JSON  │     │          │     │ 13x less tokens │
└─────────────────┘     └──────────┘     └─────────────────┘
```

## Real Numbers


| Format | Input (Telegram JSON) | Output | Savings |
|--------|----------------------|--------|---------|
| **CSV** | 11.2M tokens | 850K tokens | **92% (13x)** 🔥 |
| JSONL | 11.2M tokens | 1.0M tokens | 91% (11x) |
| JSON | 11.2M tokens | 1.3M tokens | 88% (8x) |

> 💡 **Use CSV for maximum token savings.** JSONL is good for RAG pipelines. JSON keeps full structure but wastes tokens.

## Use Cases


### 💬 Chat with your chat history

```bash
chatpack tg telegram_export.json -o context.txt
# Paste into ChatGPT: "Based on this conversation, what did we decide about...?"

```

### 🔍 Build RAG pipeline

```bash
chatpack tg chat.json -f jsonl -t -o dataset.jsonl
# Each line = one document with timestamp for vector DB

```

### 📊 Analyze conversations

```bash
chatpack wa chat.txt --from "Alice" --after 2024-01-01 -f json
# Filter and export specific messages

```

## Features


- 🚀 **Fast** — 20K+ messages/sec
- 📱 **Multi-platform** — Telegram, WhatsApp, Instagram
- 🔀 **Smart merge** — Consecutive messages from same sender → one entry
- 🎯 **Filters** — By date, by sender
- 📄 **Formats** — CSV (13x compression), JSON, JSONL (for RAG)

## Installation


### Pre-built binaries (recommended)


| Platform | Download |
|----------|----------|
| Windows | [chatpack-windows-x64.exe](https://github.com/berektassuly/chatpack/releases/latest/download/chatpack-windows-x64.exe) |
| macOS (Intel) | [chatpack-macos-x64](https://github.com/berektassuly/chatpack/releases/latest/download/chatpack-macos-x64) |
| macOS (Apple Silicon) | [chatpack-macos-arm64](https://github.com/berektassuly/chatpack/releases/latest/download/chatpack-macos-arm64) |
| Linux | [chatpack-linux-x64](https://github.com/berektassuly/chatpack/releases/latest/download/chatpack-linux-x64) |

### Via Cargo


```bash
cargo install chatpack
```

## Quick Start


```bash
# Telegram

chatpack tg result.json

# WhatsApp  

chatpack wa chat.txt

# Instagram

chatpack ig message_1.json
```

**Output:** `optimized_chat.csv` — ready to paste into ChatGPT/Claude.

## Documentation


| Guide | Description |
|-------|-------------|
| 📤 [Export Guide](docs/EXPORT_GUIDE.md) | How to export from Telegram, WhatsApp, Instagram |
| 📖 [Usage Guide](docs/USAGE.md) | All commands, flags, filters, formats |
| 📊 [Benchmarks](docs/BENCHMARKS.md) | Performance stats and compression metrics |
| 🧪 [Stress Testing](docs/STRESS_TEST.md) | Generate toxic data and run stress tests |

## Quick Reference


```bash
# Output formats

chatpack tg chat.json -f csv      # 13x compression (default)
chatpack tg chat.json -f json     # Structured array
chatpack tg chat.json -f jsonl    # One JSON per line

# Filters  

chatpack tg chat.json --after 2024-01-01
chatpack tg chat.json --from "Alice"

# Metadata

chatpack tg chat.json -t          # Add timestamps
chatpack tg chat.json -t -r -e    # All metadata
```

## Technical Details


| Source | Format | Features |
|--------|--------|----------|
| Telegram | JSON | IDs, timestamps, replies, edits |
| WhatsApp | TXT | Auto-detect locale (US/EU/RU), multiline |
| Instagram | JSON | Mojibake fix, empty message filter |

## Performance


| Metric | Value |
|--------|-------|
| Speed | 20-50K messages/sec |
| CSV compression | 13x (92% token reduction) |
| Tested file size | 500MB+ |

## License


[MIT](LICENSE) © [Mukhammedali Berektassuly](https://berektassuly.com)