Module duckdb 

DuckDB query engine for SQL dump analytics.

This module provides the ability to load SQL dumps into an embedded DuckDB database and execute analytical queries on them.

Features

• Zero dependencies: DuckDB is bundled and compiled into sql-splitter
• Multi-dialect support: MySQL, PostgreSQL, and SQLite dumps
• Memory management: Auto-switches to disk mode for large dumps
• Caching: Optional persistent cache for repeated queries

Example

use sql_splitter::duckdb::{QueryEngine, QueryConfig, QueryResultFormatter, OutputFormat};
use std::path::Path;

let config = QueryConfig::default();
let mut engine = QueryEngine::new(&config).unwrap();
engine.import_dump(Path::new("dump.sql")).unwrap();

let result = engine.query("SELECT COUNT(*) FROM users").unwrap();
println!("{}", QueryResultFormatter::format(&result, OutputFormat::Table));

Structs

BatchManager

Manages batched INSERT operations for multiple tables

CacheManager

Manager for cached DuckDB databases

DumpLoader

Loads SQL dumps into a DuckDB database

ImportStats

Statistics from dump import

InsertBatch

A batch of rows for a single table

QueryConfig

Configuration for the query engine

QueryEngine

The main query engine that wraps DuckDB

QueryResult

Result of a query execution

QueryResultFormatter

Formatter for query results

TypeConverter

Converter for SQL types to DuckDB-compatible types

Enums

OutputFormat

Output format for query results

Constants

DISK_MODE_THRESHOLD

Threshold for automatic disk mode (2 GB)

MAX_ROWS_PER_BATCH

Maximum rows to accumulate per batch before flushing

Functions

flush_batch

Flush a batch using DuckDB’s Appender API with transactional fallback

should_use_disk_mode

Determine if disk mode should be used based on file size