excelstream
🦀 ExcelStream is a high-performance XLSX writer/reader for Rust, optimized for massive datasets with constant memory usage.
✨ What's New in v0.9.0:
- � 84% Memory Reduction - New streaming architecture: 2.7 MB vs 17 MB (for 1M rows!)
- � Zero Temp Files - Stream XML directly into ZIP compressor (no disk I/O)
- ⚡ Same Speed - ~1400ms for 1M rows (compression level 6)
- 🎯 ZeroTempWorkbook - New API for absolute minimal memory usage
- � Streaming ZIP Writer - On-the-fly compression with data descriptors
- 💾 Production Ready - 2.7 MB RAM = run in 64 MB containers!
v0.8.0 Features:
- � Removed Calamine - Eliminated calamine dependency completely, now 100% custom implementation
- 🎯 Constant Memory Streaming - Read ANY file size with only 10-12 MB RAM (tested with 1GB+ files!)
- ⚡ 104x Memory Reduction - 1.2GB XML → 11.6 MB RAM (vs 1204 MB with calamine)
- � Faster Performance - Write: 106-118K rows/sec (+70%!), Read: 50-60K rows/sec
- 📊 Multi-sheet Support - Full workbook.xml parsing with sheet_names() and rows_by_index()
- 🌍 Unicode Support - Proper handling of non-ASCII sheet names and special characters
- 🔧 Custom XML Parser - Chunked reading (128 KB buffers) with smart tag boundary detection
- 🐳 Production Ready - Process multi-GB Excel files in tiny 256 MB Kubernetes pods
🎯 Why ExcelStream?
The Problem with Traditional Excel Libraries
Most Excel libraries in Rust (and other languages) load entire files into memory:
// ❌ Traditional approach - Loads ENTIRE file into RAM
let workbook = new?;
for row in workbook.worksheet?.rows
Problems:
- 📈 Memory grows with file size (10MB file = 100MB+ RAM)
- 💥 OOM crashes on large files (>100MB)
- 🐌 Slow startup (must load everything first)
- 🔴 Impossible in containers (<512MB RAM)
What About Calamine?
- Even calamine (popular Rust library) loads full files into memory
- v0.7.x used calamine: 86 MB file → 86 MB RAM (better than most, but not streaming)
The ExcelStream Solution: Streaming Architecture
// ✅✅ ExcelStream ZeroTempWorkbook (v0.9.0+) - Stream XML directly into compressor!
use ZeroTempWorkbook;
let mut writer = new?; // compression level 6
writer.add_worksheet?;
for i in 0..10_000_000
writer.close?;
// ✅ ExcelStream Reader (v0.8.0+) - Custom chunked XML parser!
let mut reader = open?;
for row in reader.rows?
v0.9.0 Architecture:
- Zero temp files - Stream XML directly into ZIP compressor
- On-the-fly DEFLATE compression with data descriptors
- 4KB XML buffer (reused per row)
- Direct write to final .xlsx file
- Result: 84% less memory than v0.8 (2.7 MB vs 17 MB)!
Why This Matters:
| Scenario | Traditional Library | ExcelStream v0.8 | ExcelStream v0.9 |
|---|---|---|---|
| Write 1M rows | 100+ MB RAM | 17 MB RAM | 2.7 MB RAM ✅ |
| Write 10M rows | ❌ Crash | 17 MB RAM | 2.7 MB RAM ✅ |
| Read 86 MB file (1.2 GB XML) | ❌ Crash | 11.6 MB RAM ✅ | 11.6 MB RAM ✅ |
| Read 100 MB file | 1+ GB RAM | ~12 MB RAM ✅ | ~12 MB RAM ✅ |
| K8s pod (<512MB) | ❌ OOMKilled | ✅ Works | ✅ Always works ✅ |
Note: v0.9.0 streaming write architecture (zero temp files) + v0.8.0 custom XML parser (no calamine).
🚀 Real-World Use Cases
1. Processing Large Enterprise Files (>100 MB)
Problem: Sales team sends 500 MB Excel with 2M+ customer records. Traditional libraries crash.
use ExcelReader;
// ✅ Processes 2M rows with only 25 MB RAM
let mut reader = open?;
let mut total_revenue = 0.0;
for row in reader.rows?
println!;
Why ExcelStream wins:
- ✅ Constant 25 MB memory (traditional = 5+ GB)
- ✅ Processes row-by-row (no buffering)
- ✅ Works in K8s pods with 512 MB limit
- ⚡ Starts processing immediately (no load delay)
2. Daily Database Exports (Production ETL)
Problem: Export 430K+ invoice records to Excel every night. Must fit in 512 MB pod.
use ExcelWriter;
use ;
// ✅ Real production code - 430,099 rows in 94 seconds
let mut writer = with_compression?;
writer.set_flush_interval; // Flush every 500 rows
let mut client = connect?;
let mut tx = client.transaction?;
tx.execute?;
loop
writer.save?; // 62 MB file, used only 25 MB RAM
Production Results:
- ✅ 430K rows exported successfully
- ✅ Peak memory: 25 MB (traditional = 500+ MB)
- ✅ Duration: 94 seconds (4,567 rows/sec)
- ✅ Runs nightly in K8s pod (512 MB limit)
- 🐳 Zero OOMKilled errors
3. Real-Time Streaming Exports (No Wait Time)
Problem: User clicks "Export" button. Traditional libraries must load ALL data first = 30+ second wait.
use ExcelWriter;
use StreamExt;
// ✅ Stream directly from async query - starts writing immediately!
let mut writer = new?;
writer.write_header_bold?;
let mut stream = db.query_stream.await?;
// User sees progress immediately! No 30-second wait!
while let Some = stream.next.await
writer.save?;
User Experience:
- ✅ Instant feedback (not 30-second blank screen)
- ✅ Progress bar possible (count rows written)
- ✅ Cancellable (user can abort early)
- 🚀 Feels 10x faster (starts immediately)
4. Kubernetes CronJobs (Memory-Limited)
Problem: K8s pods have 256-512 MB limits. Traditional libraries need 2+ GB for large exports.
use ExcelWriter;
// ✅ Optimized for K8s - uses only 15 MB!
let mut writer = with_compression?;
writer.set_flush_interval; // Aggressive flushing
writer.set_max_buffer_size; // 256 KB buffer
// Export 1M rows in 256 MB pod - impossible with traditional libraries!
for i in 0..1_000_000
writer.save?;
K8s Benefits:
- ✅ Works in 256 MB pods (traditional needs 2+ GB)
- ✅ Predictable memory (no spikes or OOM)
- ✅ Fast compression (level 1 = 2x faster)
- 🐳 Perfect for cost-optimized clusters
5. Processing Large Excel Imports (v0.8.0+)
Problem: Users upload 100 MB+ Excel files. Traditional readers load entire file = OOM crash.
use ExcelReader;
// ✅ Process 1 GB Excel file with only 12 MB RAM!
// v0.8.0: Custom XML parser, no calamine!
let mut reader = open?;
let mut total = 0.0;
let mut count = 0;
for row_result in reader.rows?
println!;
Import Benefits (v0.8.0):
- ✅ 1 GB file (1.2 GB uncompressed XML) = only 11.6 MB RAM
- ✅ 50K-60K rows/sec processing speed
- ✅ 104x less memory than loading full file (1204 MB → 11.6 MB)
- ✅ Works in 256 MB Kubernetes pods
- ✅ 100% accurate - captures all rows without data loss
- ✅ No calamine dependency - custom chunked XML parser
- ⚡ Starts processing immediately (no 30-second load wait)
6. Multi-Tenant SaaS Exports
Problem: 100 concurrent users export reports. Traditional = 100 × 500 MB = 50 GB RAM!
use ExcelWriter;
// ✅ Each export uses only 20 MB
async
// 100 concurrent exports = 100 × 20 MB = 2 GB (not 50 GB!)
SaaS Benefits:
- ✅ 100 concurrent users = 2 GB (traditional = 50+ GB)
- ✅ Scales horizontally (predictable memory)
- ✅ No "export queue" needed
- 💰 Lower infrastructure costs
📦 Installation
Add to your Cargo.toml:
[]
= "0.7"
Latest version: 0.7.0 - Worksheet protection, cell merging, functional column widths
🚀 Quick Start
Reading Excel Files (Streaming)
use ExcelReader;
v0.8.0 Note: ExcelReader now uses a custom chunked XML parser (no calamine). Memory usage is constant (~10-12 MB) regardless of file size!
Reading Large Files (Streaming - v0.8.0)
ExcelReader provides constant memory usage (~10-12 MB) for ANY file size:
use ExcelReader;
Performance (v0.8.0):
- ✅ Memory: Constant 10-12 MB (file can be 1 GB+!)
- ✅ Speed: 50K-60K rows/sec
- ✅ K8s Ready: Works in 256 MB pods
- ⚡ No Dependencies: Custom XML parser, no calamine
- 🎯 104x Reduction: 1.2 GB XML → 11.6 MB RAM
Architecture:
- Custom chunked XML parser (128 KB chunks)
- Smart buffering with split-tag handling
- SST loaded once, rows streamed incrementally
- No formula/formatting support (raw values only)
Writing Excel Files (Zero Temp Streaming - v0.9.0)
New in v0.9.0: ZeroTempWorkbook - Stream XML directly into ZIP compressor! 84% less memory than v0.8.
use ZeroTempWorkbook;
Performance (v0.9.0):
- ✅ Memory: Only 2.7 MB for ANY SIZE (84% less than v0.8!)
- ✅ Speed: 50K-60K rows/sec (same as v0.8)
- ✅ Zero Temp Files: Streams directly to final .xlsx
- ✅ K8s Ready: Works in 128 MB pods
- 🎯 Architecture: On-the-fly DEFLATE with data descriptors
Legacy Writer (v0.2.0 - still available):
use ExcelWriter;
Key Benefits:
- ✅ Constant ~80MB memory usage regardless of dataset size
- ✅ High throughput: 60K-70K rows/sec (UltraLowMemoryWorkbook fastest at 69.5K)
- ✅ Direct ZIP streaming - data written to disk immediately
- ✅ Full formatting support: bold, styles, column widths, row heights
Writing with Typed Values (Recommended)
For better Excel compatibility and performance, use typed values:
use ExcelWriter;
use CellValue;
Benefits of write_row_typed():
- ✅ Numbers are stored as numbers (not text)
- ✅ Booleans display as TRUE/FALSE
- ✅ Excel formulas work correctly
- ✅ Better type safety
- ✅ Excellent performance: 62.7K rows/sec (+2% faster than string-based)
Preserving Leading Zeros (Phone Numbers, IDs)
New in v0.6.1: Proper handling of string numbers with leading zeros!
Problem: Leading Zeros Lost
// ❌ WRONG: Auto number detection loses leading zeros
writer.write_row; // Displays as 90899 in Excel
Solution 1: Use write_row() (All Strings)
// ✅ CORRECT: write_row() treats ALL values as strings
writer.write_row;
// Displays: "090899", "00123", "ID-00456" ✅ Leading zeros preserved!
As of v0.6.1, write_row() no longer auto-detects numbers. All values are treated as strings.
Solution 2: Use write_row_typed() with Explicit Types
use CellValue;
// ✅ String type: preserves leading zeros
writer.write_row_typed?;
// ✅ Int type: actual number (no leading zero)
writer.write_row_typed?;
Solution 3: Use write_row_styled() for Full Control
use ;
writer.write_row_styled?;
Type Handling Summary:
| Method | String "090899" | Int 90899 | Use When |
|---|---|---|---|
write_row(&[&str]) |
"090899" ✅ | N/A | All data is text (IDs, codes) |
write_row_typed(CellValue) |
"090899" ✅ | 90899 (number) | Mixed types, explicit control |
write_row_styled() |
"090899" ✅ | 90899 (number) | Need formatting + type control |
Best Practice:
- Phone numbers, IDs, ZIP codes → Use
CellValue::Stringorwrite_row() - Amounts, quantities, calculations → Use
CellValue::IntorCellValue::Float
Writing Excel Formulas
use ExcelWriter;
use CellValue;
Supported Formulas:
- ✅ Basic arithmetic:
=A1+B1,=A1*B1,=A1/B1 - ✅ SUM, AVERAGE, COUNT, MIN, MAX
- ✅ Cell ranges:
=SUM(A1:A10) - ✅ Complex formulas:
=IF(A1>100, "High", "Low") - ✅ All standard Excel functions
Cell Formatting and Styling
New in v0.3.0: Apply 14 predefined cell styles including bold headers, number formats, highlights, and borders!
Bold Headers
use ExcelWriter;
let mut writer = new?;
// Write bold header
writer.write_header_bold?;
// Regular data rows
writer.write_row?;
writer.write_row?;
writer.save?;
Styled Cells
Apply different styles to individual cells:
use ;
use ExcelWriter;
let mut writer = new?;
writer.write_header_bold?;
// Mix different styles in one row
writer.write_row_styled?;
writer.write_row_styled?;
writer.save?;
Uniform Row Styling
Apply the same style to all cells in a row:
// All cells bold
writer.write_row_with_style?;
// All cells highlighted yellow
writer.write_row_with_style?;
Available Cell Styles
| Style | Format Code | Example | Use Case |
|---|---|---|---|
CellStyle::Default |
None | Plain text | Regular data |
CellStyle::HeaderBold |
Bold | Header | Column headers |
CellStyle::NumberInteger |
#,##0 | 1,234 | Whole numbers |
CellStyle::NumberDecimal |
#,##0.00 | 1,234.56 | Decimals |
CellStyle::NumberCurrency |
$#,##0.00 | $1,234.56 | Money amounts |
CellStyle::NumberPercentage |
0.00% | 95.00% | Percentages |
CellStyle::DateDefault |
MM/DD/YYYY | 01/15/2024 | Dates |
CellStyle::DateTimestamp |
MM/DD/YYYY HH:MM:SS | 01/15/2024 14:30:00 | Timestamps |
CellStyle::TextBold |
Bold | Bold text | Emphasis |
CellStyle::TextItalic |
Italic | Italic text | Notes |
CellStyle::HighlightYellow |
Yellow bg | 🟨 Text | Warnings |
CellStyle::HighlightGreen |
Green bg | 🟩 Text | Success |
CellStyle::HighlightRed |
Red bg | 🟥 Text | Errors |
CellStyle::BorderThin |
Thin borders | ▭ Text | Boundaries |
Complete Example
use ExcelWriter;
use ;
See also: Run cargo run --example cell_formatting to see all 14 styles in action!
Column Width, Row Height, and Cell Merging
New in v0.7.0: Full layout control with column widths, row heights, and cell merging!
Column Width
Set column widths before writing any rows:
use ExcelWriter;
let mut writer = new?;
// Set column widths BEFORE writing rows
writer.set_column_width?; // Column A = 25 units wide
writer.set_column_width?; // Column B = 12 units wide
writer.set_column_width?; // Column C = 15 units wide
// Now write rows
writer.write_header_bold?;
writer.write_row?;
writer.save?;
Important:
- ⚠️ Column widths must be set before writing any rows
- Default column width is 8.43 units
- One unit ≈ width of one character in default font
- Typical range: 8-50 units
Row Height
Set row height for the next row to be written:
use ExcelWriter;
let mut writer = new?;
// Set height for header row (taller)
writer.set_next_row_height?;
writer.write_header_bold?;
// Regular row (default height)
writer.write_row?;
// Set height for next row
writer.set_next_row_height?;
writer.write_row?;
writer.save?;
Important:
- Height is in points (1 point = 1/72 inch)
- Default row height is 15 points
- Typical range: 10-50 points
- Setting is consumed by next
write_row()call
Complete Example with Sizing
use ExcelWriter;
use ;
Cell Merging
Merge cells horizontally (for titles) or vertically (for grouped data):
use ExcelWriter;
use ;
let mut writer = new?;
// Set column widths
writer.set_column_width?;
writer.set_column_width?;
// Title row spanning 3 columns
writer.write_row_styled?;
writer.merge_cells?; // Horizontal merge
writer.write_row?; // Empty row
// Headers
writer.write_header_bold?;
// Data with vertical merge for region
writer.write_row?;
writer.write_row?;
writer.write_row?;
writer.merge_cells?; // Vertical merge - "North" spans 3 rows
writer.save?;
Common Patterns:
- Title rows:
merge_cells("A1:F1")- Header spanning all columns - Grouped data:
merge_cells("A2:A5")- Category name for multiple items - Subtotals:
merge_cells("A10:C10")- "Total" label spanning columns
See also:
cargo run --example column_width_row_height- Layout control democargo run --example column_merge_demo- Complete merging examples
Direct FastWorkbook Usage (Maximum Performance)
For maximum performance, use FastWorkbook directly:
use FastWorkbook;
🧠 Hybrid SST Optimization (v0.5.0)
New in v0.5.0: Intelligent selective deduplication for optimal memory usage!
How It Works
The Hybrid Shared String Table (SST) strategy intelligently decides which strings to deduplicate:
// Automatic optimization - no code changes needed!
let mut workbook = new?;
workbook.add_worksheet?;
// Numbers → Inline as number type (no SST)
workbook.write_row?;
// Long strings (>50 chars) → Inline (usually unique)
workbook.write_row?;
// Short repeated strings → SST (efficient deduplication)
workbook.write_row?;
Memory Improvements
| Workbook Type | Before v0.5.0 | After v0.5.0 | Reduction |
|---|---|---|---|
| Simple (5 cols, 1M rows) | 49 MB | 18.8 MB | 62% |
| Medium (19 cols, 1M rows) | 125 MB | 15.4 MB | 88% |
| Complex (50 cols, 100K rows) | ~200 MB | 22.7 MB | 89% |
| Multi-workbook (4 × 100K rows) | 251 MB | 25.3 MB | 90% |
Strategy Details
The hybrid approach uses these rules:
- Numbers (
123,456.78) → Inline as<c t="n">(no SST) - Long strings (>50 chars) → Inline as
<c t="inlineStr">(usually unique) - SST Full (>100k unique) → New strings inline (graceful degradation)
- Short strings (≤50 chars) → SST for deduplication (efficient)
Performance Impact
ExcelWriter.write_row(): 16,250 rows/sec (baseline)
ExcelWriter.write_row_typed(): 19,642 rows/sec (+21%)
ExcelWriter.write_row_styled(): 18,581 rows/sec (+14%)
FastWorkbook (hybrid SST): 25,682 rows/sec (+58%) ⚡
Key Benefits:
- ✅ 89% less memory for complex workbooks
- ✅ 58% faster due to fewer SST lookups
- ✅ Handles 50+ columns with mixed data types
- ✅ Automatic - no API changes required
- ✅ Graceful degradation - caps at 100k unique strings
See also: HYBRID_SST_OPTIMIZATION.md for technical details
🔒 Worksheet Protection (v0.7.0)
New in v0.7.0: Protect worksheets with passwords and granular permissions!
Basic Protection
use ;
let mut writer = new?;
// Protect with password - users can view but not edit
let protection = new
.with_password;
writer.protect_sheet?;
writer.write_header_bold?;
writer.write_row?;
writer.save?;
Granular Permissions
Control exactly what users can do:
use ;
let mut writer = new?;
// Allow formatting but prevent data changes
let protection = new
.with_password
.allow_select_locked_cells
.allow_select_unlocked_cells
.allow_format_cells // ✅ Can format
.allow_format_columns // ✅ Can resize columns
.allow_format_rows; // ✅ Can resize rows
// Everything else is protected (insert, delete, edit)
writer.protect_sheet?;
writer.save?;
Data Entry Forms
Allow users to insert/delete rows but protect headers:
let protection = new
.with_password
.allow_insert_rows // ✅ Can add rows
.allow_delete_rows // ✅ Can delete rows
.allow_sort; // ✅ Can sort data
writer.protect_sheet?;
// Headers are protected, but users can add data rows
writer.write_header_bold?;
writer.write_row?;
writer.save?;
Available Permissions
| Permission | Description | Use Case |
|---|---|---|
allow_select_locked_cells |
Can select protected cells | View-only (default: true) |
allow_select_unlocked_cells |
Can select editable cells | Data entry (default: true) |
allow_format_cells |
Can change cell formats | Template customization |
allow_format_columns |
Can resize columns | Layout adjustments |
allow_format_rows |
Can resize rows | Layout adjustments |
allow_insert_rows |
Can insert new rows | Data entry forms |
allow_delete_rows |
Can delete rows | Data cleanup |
allow_insert_columns |
Can insert new columns | Schema changes |
allow_delete_columns |
Can delete columns | Schema changes |
allow_sort |
Can sort data | Data analysis |
allow_auto_filter |
Can use filters | Data analysis |
Common Use Cases:
- Templates: Protect formulas, allow data entry
- Reports: Lock everything (read-only)
- Data Collection: Allow insert/delete rows, protect headers
- Shared Sheets: Allow formatting, prevent structure changes
See also: cargo run --example worksheet_protection - Complete protection demo
🗜️ Compression Level Configuration (v0.5.1)
New in v0.5.1: Control ZIP compression levels to balance speed vs file size!
Understanding Compression Levels
Excel files (.xlsx) are ZIP archives. ExcelStream lets you control the compression level:
| Level | Speed | File Size | Use Case | Recommended For |
|---|---|---|---|---|
| 0 | Fastest | Largest (no compression) | Debugging only | Testing |
| 1 | Very Fast ⚡ | ~2x larger | Fast exports | Development, testing |
| 3 | Fast | Balanced | Good compromise | CI/CD pipelines |
| 6 | Moderate | Smallest 📦 | Best compression | Production exports |
| 9 | Slowest | Smallest | Maximum compression | Archives, long-term storage |
Default: Level 6 (balanced performance and file size)
Setting Compression Level
Method 1: At Workbook Creation
use ExcelWriter;
// Create writer with fast compression (level 1)
let mut writer = with_compression?;
writer.write_header?;
writer.write_row?;
writer.save?;
Method 2: After Creation
use ExcelWriter;
let mut writer = new?;
// Change compression level
writer.set_compression_level; // Fast balanced compression
writer.write_header?;
writer.write_row?;
writer.save?;
Method 3: With UltraLowMemoryWorkbook
use UltraLowMemoryWorkbook;
let mut workbook = with_compression?;
workbook.add_worksheet?;
workbook.write_row?;
workbook.write_row?;
workbook.close?;
Method 4: Environment-Based Configuration
use ExcelWriter;
// Use fast compression for debug builds, production compression for release
let compression_level = if cfg! else ;
let mut writer = with_compression?;
writer.write_header?;
writer.write_row?;
writer.save?;
Real-World Performance (1M rows)
Tested with 1 million rows × 4 columns on production hardware:
| Configuration | Flush Interval | Buffer Size | Compression | Time | File Size | Memory |
|---|---|---|---|---|---|---|
| Aggressive | 100 | 256 KB | Level 1 | 3.93s ⚡ | 172 MB | <30 MB |
| Balanced | 500 | 512 KB | Level 3 | 5.03s | 110 MB | <30 MB |
| Default | 1000 | 1 MB | Level 6 | 7.37s | 88 MB 📦 | <30 MB |
| Conservative | 5000 | 2 MB | Level 6 | 8.00s | 88 MB | <30 MB |
Key Findings:
- Level 1 is ~2x faster but files are ~2x larger than level 6
- Level 3 provides a good balance between speed and size
- Memory usage is constant <30 MB regardless of compression level
- Production exports typically use level 6 for optimal file size
Complete Example: Configurable Compression
use ExcelWriter;
Recommendations
For Development & Testing:
let mut writer = with_compression?;
writer.set_flush_interval;
- ✅ Fast exports (2x speed improvement)
- ✅ Quick iteration cycles
- ⚠️ Larger files (not for production)
For CI/CD Pipelines:
let mut writer = with_compression?;
writer.set_flush_interval;
- ✅ Good balance of speed and size
- ✅ Reasonable export times
- ✅ Acceptable file sizes
For Production Exports:
let mut writer = with_compression?; // Default
writer.set_flush_interval;
- ✅ Smallest file size
- ✅ Best for network transfers
- ✅ Optimal for storage
For Archives:
let mut writer = with_compression?;
- ✅ Maximum compression
- ⚠️ Slower export times
- 📦 Best for long-term storage
See also: Run cargo run --example compression_level_config for a complete demonstration!
Memory-Constrained Writing (For Kubernetes Pods)
With v0.5.0+ and compression configuration (v0.5.1), memory usage is ultra-low:
use ExcelWriter;
Memory usage in v0.5.1:
- 10-30 MB peak with optimized settings (was 80-100 MB)
- 80-90% memory reduction vs v0.4.x
- Handles 50+ columns with mixed data types
- Suitable for Kubernetes pods with limited resources
- Automatic hybrid SST optimization
Configuration Presets:
| Preset | Flush | Buffer | Compression | Use Case | Memory Peak |
|---|---|---|---|---|---|
| Aggressive | 100 | 256 KB | Level 1 | <256 MB RAM pods | 10-15 MB |
| Balanced | 500 | 512 KB | Level 3 | <512 MB RAM pods | 15-20 MB |
| Default | 1000 | 1 MB | Level 6 | Standard pods | 20-30 MB |
| Conservative | 5000 | 2 MB | Level 6 | High-memory pods | 25-35 MB |
PostgreSQL Streaming Export (Production-Tested)
ExcelStream has been tested with real production databases. Example: 430,099 e-invoice records exported successfully.
use ExcelWriter;
use ;
Production Results (430K rows):
- Duration: 1m 34s (94.17 seconds)
- Throughput: 4,567 rows/sec
- File Size: 62.22 MB
- Memory Peak: <30 MB
- Columns: 25 mixed data types (int, float, text, dates)
Key Optimizations:
- ✅ Cursor-based streaming (no full table load)
- ✅ Small batch size (500 rows) for memory efficiency
- ✅ Proper NULL handling with
try_get().ok().flatten() - ✅ Fast compression (level 3) for balanced performance
- ✅ Frequent flushing (500 rows) to disk
See also: examples/postgres_streaming.rs for complete implementation
Multi-sheet workbook
use ExcelWriterBuilder;
📚 Examples
The examples/ directory contains detailed examples:
Basic Usage:
basic_read.rs- Basic Excel file readingbasic_write.rs- Basic Excel file writingstreaming_read.rs- Reading large files with streamingstreaming_write.rs- Writing large files with streaming
Performance Comparisons:
three_writers_comparison.rs- Compare all 3 writer types (recommended!)write_row_comparison.rs- String vs typed value writingwriter_comparison.rs- Standard vs fast writer comparisonfast_writer_test.rs- Fast writer performance benchmarks
Advanced Features:
memory_constrained_write.rs- Memory-limited writing with compression configauto_memory_config.rs- Auto memory configuration democompression_level_config.rs- Compression level configuration examplescsv_to_excel.rs- CSV to Excel conversionmulti_sheet.rs- Creating multi-sheet workbooks
PostgreSQL Integration:
postgres_to_excel.rs- Basic PostgreSQL exportpostgres_streaming.rs- Production-tested streaming export (430K rows)postgres_to_excel_advanced.rs- Advanced async with connection pooling
Running examples:
# Create sample data first
# Read Excel file
# Streaming with large files
# Performance comparisons (RECOMMENDED)
# Memory-constrained writing with compression
MEMORY_LIMIT_MB=512
# Compression level examples
# Multi-sheet workbooks
# PostgreSQL examples (requires database setup)
🔧 API Documentation
ExcelReader
open(path)- Open Excel file for readingsheet_names()- Get list of sheet namesrows(sheet_name)- Iterator for streaming row readingread_cell(sheet, row, col)- Read specific celldimensions(sheet_name)- Get sheet dimensions (rows, cols)
ExcelWriter (Streaming)
new(path)- Create new writer with default compression (level 6)with_compression(path, level)- Create with custom compression level (0-9)write_row(data)- Write row with stringswrite_row_typed(cells)- Write row with typed values (recommended)write_header(headers)- Write header rowwrite_header_bold(headers)- Write bold header rowwrite_row_styled(cells)- Write row with individual cell styleswrite_row_with_style(cells, style)- Write row with uniform styleadd_sheet(name)- Add new sheetset_flush_interval(rows)- Configure flush frequency (default: 1000)set_max_buffer_size(bytes)- Configure buffer size (default: 1MB)set_compression_level(level)- Set compression level (0-9, default: 6)compression_level()- Get current compression levelset_column_width(col, width)- Set column width (before writing rows)set_next_row_height(height)- Set height for next rowsave()- Save and finalize workbook
FastWorkbook (Direct Access)
new(path)- Create fast writer with default compression (level 6)with_compression(path, level)- Create with custom compression level (0-9)add_worksheet(name)- Add worksheetwrite_row(data)- Write row (optimized)set_flush_interval(rows)- Set flush frequencyset_max_buffer_size(bytes)- Set buffer limitset_compression_level(level)- Set compression level (0-9)compression_level()- Get current compression levelclose()- Finish and save file
Types
CellValue- Enum: Empty, String, Int, Float, Bool, DateTime, Error, FormulaRow- Row with index and cells vectorCell- Cell with position (row, col) and value
🎯 Use Cases
Processing Large Excel Files (100MB+)
// Streaming ensures only small portions are loaded into memory
let mut reader = open?;
let mut total = 0.0;
for row_result in reader.rows?
Exporting Database to Excel
let mut writer = new?;
writer.write_header?;
// Fetch from database and write directly
for record in database.query?
writer.save?;
Converting CSV to Excel
use File;
use ;
let csv = new;
let mut writer = new?;
for in csv.lines.enumerate
writer.save?;
⚡ Performance
Benchmarked with 1 million rows × 30 columns (mixed data types):
| Writer Method | Throughput | Memory Usage | Features |
|---|---|---|---|
| UltraLowMemoryWorkbook (direct) | 69,500 rows/sec ⚡ | ~80MB constant ✅ | FASTEST - Low-level API, max control |
| ExcelWriter.write_row_typed() | 62,700 rows/sec | ~80MB constant ✅ | Type-safe, best balance (+2% vs baseline) |
| ExcelWriter.write_row() | 61,400 rows/sec | ~80MB constant ✅ | Simple API, string-based (baseline) |
| ExcelWriter.write_row_styled() | 43,500 rows/sec | ~80MB constant ✅ | Cell formatting - 29% slower due to styling overhead |
Key Characteristics:
- ✅ High throughput - 43K-70K rows/sec depending on method
- ✅ Constant memory - stays at ~80MB regardless of dataset size
- ✅ Streaming write - data written directly to disk via ZIP
- ✅ Predictable performance - no memory spikes or slowdowns
- ⚡ UltraLowMemoryWorkbook is FASTEST - Direct low-level access (+13% vs baseline)
- ⚠️ Styling has overhead - write_row_styled() is 29% slower but adds formatting
Recommendations
| Use Case | Recommended Method | Why |
|---|---|---|
| General use | write_row_typed() |
Best balance - Type-safe, fast (62.7K rows/sec, +2%) |
| Simple exports | write_row() |
Easy API, good performance (61.4K rows/sec) |
| Formatted reports | write_row_styled() |
Cell formatting - slower but worth it (43.5K rows/sec, -29%) |
| Maximum speed | UltraLowMemoryWorkbook |
FASTEST - Direct low-level access (69.5K rows/sec, +13%) |
📖 Documentation
- README.md (this file) - Complete guide with examples
- CONTRIBUTING.md - How to contribute
- CHANGELOG.md - Version history
- Examples in
/examplesdirectory - Doc tests in source code
🛠️ Development
Build
Test
Run examples
Benchmark
📋 Requirements
- Rust 1.70 or higher
- Dependencies:
zip- ZIP compression for reading/writingthiserror- Error handling- No calamine - Custom XML streaming parser (v0.8.0+)
🚀 Production Ready
- ✅ Battle-tested - Handles 1M+ row datasets with ease
- ✅ High performance - 43K-70K rows/sec depending on method (UltraLowMemoryWorkbook fastest!)
- ✅ Memory efficient - Constant ~80MB usage, perfect for K8s pods
- ✅ Reliable - 50+ comprehensive tests covering edge cases
- ✅ Safe - Zero unsafe code, full Rust memory safety
- ✅ Compatible - Excel, LibreOffice, Google Sheets
- ✅ Unicode support - Special characters, emojis, CJK
- ✅ CI/CD - Automated testing on every commit
🤝 Contributing
Contributions welcome! Please see CONTRIBUTING.md for guidelines.
Areas for Contribution:
- Cell formatting and styling (Phase 3)
- Conditional formatting
- Charts and images support
- More examples and documentation
- Performance optimizations
All contributions must:
- Pass
cargo fmt --check - Pass
cargo clippy -- -D warnings - Pass all tests
cargo test --all-features - Include tests for new features
📄 License
MIT License - see LICENSE file for details.
🙏 Credits
This library uses:
- Custom XML streaming parser - Chunked reading for constant memory (v0.8.0+)
- Custom FastWorkbook - High-performance streaming writer
- No external Excel dependencies (calamine removed in v0.8.0)
📧 Contact
For questions or suggestions, please create an issue on GitHub.
Made with ❤️ and 🦀 by the Rust community