TORM - Tokio ORM
TORM is a Rust ORM (Object-Relational Mapping) library built on the Tokio async runtime, providing GORM-like functionality with a layered module design (Database / ORM / Utils / Monitoring).
🎯 Key Features
- ✅ Standard SQLite Support - Built on rusqlite, generates standard SQLite file format (readable by sqlite3 and other SQLite tools)
- ✅ Pure Rust Storage Engine - Built-in zero-dependency in-memory storage engine (StorageEngine)
- ✅ PostgreSQL Support - Native wire protocol implementation (cleartext / MD5 / SCRAM-SHA-256 auth, parameterized queries)
- ✅ MySQL Support - Native wire protocol implementation (mysql_native_password / caching_sha2_password / sha256_password auth, text/binary protocol parameterized queries)
- ✅ Async/await Support - Fully based on the Tokio async runtime
- ✅ Multi-Database Support - MySQL, PostgreSQL, SQLite
- ✅ Fluent Query Builder - Clean and intuitive query API
- ✅ Query Direct Execution -
insert/update/deleteexecute SQL directly, inspect withreturn_sql() - ✅ Advanced Queries - JOIN, GROUP BY, HAVING, aggregate functions
- ✅ Zero
SqlValue-where_*/insert/updateaccept plain Rust values (i32/f64/&str/bool...) viaInto<SqlValue> - ✅ Dapper-style Typed Mapping -
QueryExecutor::models::<M>()maps rows back into typedVec<M> - ✅ Model Trait - Automatic management of created_at, updated_at timestamps
- ✅
#[derive(Model)]Macro - Generate theModelimpl from a plain struct, eliminating boilerplate - ✅ GORM-style Model CRUD -
create/first/last/all/update/deleteonDatabase - ✅ Auto-increment Primary Key -
#[derive(Model)]marks integer PK auto-increment;id: 0auto-assigns & refills - ✅ GORM-style Indexes -
primaryKey/index/uniqueIndexfield tags withauto_migratetable/index creation - ✅ Transaction Support - Create, commit, and rollback transactions
- ✅ Connection Pooling - Pools for SQLite/MySQL/PostgreSQL
- ✅ SQL Injection Protection - Identifier validation/quotation, string escaping, and dangerous-pattern detection (
utils::sql_safety) - ✅ Logging & Performance Monitoring - Built-in logging system and performance stats
📦 Dependencies
[]
= "1.53" # Async runtime
= { = "0.30", = ["bundled"] } # SQLite (standard file format)
= "1.0" # UUID generation
= "1.0" # Serialization
= "1.0" # JSON support
= "0.4" # Time handling
= "0.1" # Async traits
= "1.0" # Error derivation
# PostgreSQL / MySQL wire protocol authentication
= "0.10" # PostgreSQL SCRAM-SHA-256 / MySQL caching_sha2_password
= "0.10" # MySQL mysql_native_password authentication
= "0.10" # PostgreSQL MD5 authentication
= "0.4" # Byte/hex encoding
= "0.22" # SCRAM base64 encoding
# RSA encryption for MySQL caching_sha2_password full auth (MySQL 8.0+)
= "0.9"
= "0.4"
= "0.8"
Database Layer Implementation
| Feature | Implementation | Status |
|---|---|---|
| SQLite | rusqlite (standard file format) | ✅ Complete |
| In-memory engine | Pure Rust StorageEngine | ✅ Complete |
| MySQL | Custom wire protocol (native) | ✅ Complete |
| PostgreSQL | Custom wire protocol (native) | ✅ Complete |
| Type safety | SqlValue + auto-conversion (Into<SqlValue>) |
✅ Complete |
| Typed mapping | Dapper-style: model CRUD maps rows back to typed structs | ✅ Complete |
| Auto-increment PK | #[derive(Model)] marks integer PK auto-increment |
✅ Complete |
| Transactions | Custom implementation | ✅ Complete |
🏗 Module Structure
src/
├── lib.rs # Module declarations and exports
├── db/ # Database layer
│ ├── db_types.rs # SQL type system (SqlValue, Row, QueryResult)
│ ├── database.rs # Connection abstraction, transactions, factory, Database
│ ├── driver.rs # DBDriver, Dsn
│ ├── error.rs # TormError
│ ├── storage.rs # Pure Rust in-memory storage engine
│ ├── sqlite.rs # SQLite implementation (rusqlite backend)
│ ├── mysql.rs # MySQL wire protocol implementation
│ ├── postgresql.rs # PostgreSQL wire protocol implementation
│ └── pool.rs # Connection pools
├── orm/ # ORM layer
│ ├── model.rs # Model trait
│ ├── query.rs # Query/QueryBuilder
│ ├── advanced_query.rs # Advanced queries (JOIN/GROUP BY/aggregates)
│ ├── relations.rs # Relationships
│ └── migration.rs # Migrations
├── utils/ # Utils layer (zero-dependency implementations)
│ ├── simple_pool.rs # Simple connection pool
│ ├── simple_lru.rs # LRU cache
│ ├── simple_error.rs # Simplified errors
│ ├── simple_uuid.rs # UUID/ID generation
│ └── sql_safety.rs # SQL injection protection (identifiers, escaping, detection)
└── monitoring/ # Monitoring layer
├── logger.rs # Logging system
└── performance.rs # Performance monitoring
🚀 Quick Start
Basic Usage
The recommended way is to define a #[derive(Model)] struct first, then use the high-level ORM API. All insert / query / update / delete values are plain Rust types — no SqlValue needed.
use ;
// Define a model; the macro generates the schema, column mapping and from_row.
async
The generated mydb.db is a standard SQLite file, directly inspectable with sqlite3 mydb.db:
||
If you prefer low-level raw SQL (e.g. for arbitrary queries), use
db.execute(sql, &[SqlValue...])— see the "Type-Safe SQL Values" section below.
Type-Safe SQL Values
let value: SqlValue = 42.into; // I32(42)
let value: SqlValue = "hello".into; // String("hello")
let value: SqlValue = true.into; // Bool(true)
let value = DateTime; // DateTime(...)
// SQL string conversion
let sql = value.to_sql_string; // "42", "'hello'", "TRUE"
SQL Injection Protection
The utils::sql_safety module (re-exported at the crate root) provides defense-in-depth against SQL injection. While parameterized queries (? / $1 placeholders) are the first line of defense for values, identifiers (table/column names) are still interpolated directly into SQL. The library automatically validates identifiers in Query / AdvancedQuery / model CRUD; for custom SQL you can use these utilities directly:
use ;
// 1. Validate / quote identifiers before splicing them into SQL
assert_eq!;
assert!;
assert_eq!;
// SqlSanitizer::identifier returns a safe, splicable string
// (falls back to "" and warns when the identifier is unsafe)
let col = identifier;
let query = format!; // safe
// 2. Escape string literals if you must inline values
let value = escape_string; // "O''Reilly"
// 3. Heuristically audit raw SQL for dangerous patterns
// (skips string literals & comments to reduce false positives)
assert!;
assert!;
Note:
contains_injection_patternis a heuristic audit tool for assisting review — it does not replace parameterized queries.
Query Builder
Query provides a fluent builder that can execute directly against a &Database, or inspect the generated SQL with return_sql().
use ;
async
All where_* / update / insert values accept plain Rust values (i32 / i64 / f32 / f64 / &str / String / bool / Vec<u8> / chrono::DateTime<Utc> and the unsigned variants) via Into<SqlValue> — you never need to write SqlValue::Type(...).
Query::query(db) returns a QueryExecutor for chaining read operations:
QueryExecutor::count()- executesSELECT COUNT(*), returning a result set with aCOUNT(*)columnQueryExecutor::select()- executesSELECT *QueryExecutor::models::<M>()- executesSELECT *and maps every row back into a typedVec<M>via theModeltrait (requires#[derive(Model)])
Query also returns a SqlStatement from build() / count() / build_update() / etc., which offers both execution and inspection:
SqlStatement::execute(&db)/SqlStatement::query(&db)- run the statement directlySqlStatement::return_sql()- get the(sql, params)pairQuery::return_sql()- get the(sql, params)of the most recently built / executed operation
Note: SQLite and MySQL use
?placeholders; PostgreSQL uses$1/$2/.... The conversion happens automatically during execution.
Deriving a Model
Instead of hand-writing the Model impl, annotate your struct with #[derive(Model)] and a #[model(table_name = "...")] attribute. The macro generates columns(), from_row(), primary-key accessors, timestamp accessors, and the schema() for auto_migrate — all zero SqlValue. An integer primary key is marked auto-increment automatically (AUTOINCREMENT on SQLite, AUTO_INCREMENT on MySQL, SERIAL on PostgreSQL), so inserting a model with id: 0 assigns the id and writes it back.
use ;
use ;
Supported field types: String, bool, i8/i16/i32/i64, f32/f64, chrono::DateTime<Utc>, Uuid, Vec<u8> and their Option<...> wrappers. Other types are skipped automatically; use #[model(skip)] to exclude a field explicitly, and #[model(column = "...")] to rename a DB column.
GORM-style Indexes (primaryKey / index / uniqueIndex)
Like GORM, you can declare the primary key and indexes directly on the struct fields. The macro records them in Model::schema() so Database::auto_migrate() can create the table and its indexes automatically.
use ;
Supported field tags (inside #[model(...)]):
primaryKey— marks the field as the primary key.index— creates a plain index. Without a name it defaults toidx_<table>_<column>. Fields sharing the same explicit index name form a composite index.uniqueIndex— creates a unique index. On a single column it also implies aUNIQUEcolumn constraint. Without a name it defaults toidx_<table>_<column>.
Then create the table and all indexes on startup (idempotent, uses IF NOT EXISTS):
let db = sqlite.await?;
db..await?;
Dapper-style Typed CRUD
Once a model is derived, insert / query / update / delete never touch SqlValue — values are plain Rust types and results come back as typed structs (Dapper-style Query<T>).
use ;
async
Database::update(model, &[(column, value), ...]) executes the UPDATE immediately and returns the number of affected rows. Values are plain Rust types (i32/&str/... ) when all columns share a type; for mixed-type columns, pass them as SqlValue (e.g. &[("age", SqlValue::I32(30)), ("email", SqlValue::String("x".into()))]).
Connection Pool
use Pool;
let config = sqlite
.with_max_connections;
let pool = sqlite.await?;
let conn = pool.get_connection.await?;
MySQL Connection
use ;
async
📊 Database Support Status
✅ SQLite (Production-ready, standard file format)
- Built on rusqlite, generates standard SQLite files (sqlite3 compatible)
- Full CRUD operations
- Parameterized queries
- Transaction support
- Foreign key constraints
- Status: Ready for production
✅ Pure Rust In-Memory Engine (StorageEngine)
- Zero-dependency in-memory database
- Custom binary persistence format (TORMDB01)
- Full CRUD + WHERE conditions (AND/OR/comparison/LIKE)
- Status: Usable as a lightweight in-memory database
✅ MySQL (Native wire protocol, production-ready)
- Real TCP connection via
tokio::net::TcpStream - Full initial handshake (Protocol 10) and handshake response
- Authentication:
mysql_native_password,caching_sha2_password(fast/full auth with RSA encryption),sha256_password - AuthSwitchRequest / AuthMoreData auth exchange flow
- Text protocol (
COM_QUERY) for parameterless queries - Binary protocol (
COM_STMT_PREPARE/COM_STMT_EXECUTE) for parameterized queries - Column definition, text-row / binary-row decoding, OK/EOF/Error packets
- Supports
CLIENT_DEPRECATE_EOF(MySQL 5.7+) and classic EOF protocol - Transactions (BEGIN / COMMIT / ROLLBACK)
- Status: Ready for production use with MySQL 5.7+
✅ PostgreSQL (Native wire protocol, production-ready)
- Real TCP connection via
tokio::net::TcpStream - Full startup handshake (StartupMessage, protocol 3.0)
- Authentication: cleartext, MD5, SCRAM-SHA-256 (with server signature verification)
- Simple query protocol (
Q) for multi-statement SQL - Extended query protocol (Parse/Bind/Describe/Execute/Sync) for parameterized statements
- Row decoding: bool, int2/4/8, float4/8, text/varchar, bytea, json/jsonb, date/timestamp/timestamptz, numeric
- Transactions (BEGIN / COMMIT / ROLLBACK)
- Status: Ready for production use with PostgreSQL 10+
🏃 Run Examples
All examples follow the same pattern: define a #[derive(Model)] struct first, then use the high-level ORM API — insert / query / update / delete never touch SqlValue.
# Dapper-style typed CRUD (define a struct, then create / first / last / all / update / delete)
# Async concurrency: struct + auto_migrate, parallel queries mapped back to Vec<Product>
# Ergonomic Query builder: plain Rust values, no SqlValue::Type(...)
# Full integration: connection, auto-migrate, typed CRUD, count
# Basic usage (uuid / error handling / cache / connection pool) + typed model
# Complete feature demo + file persistence via a derived model
# Advanced features (JOIN / GROUP BY / HAVING / aggregates)
# Run tests
The PostgreSQL example (postgresql_example.rs) additionally shows the low-level raw SQL + SqlValue binding for parameterized queries, which is only needed when you bypass the ORM and write SQL by hand.
🔄 Database Migration Tools
TORM ships with six standalone CLI tools (under src/bin/) that migrate schema and data between databases using the TORM native protocol drivers. Each tool discovers the source tables, translates the schema to the target dialect, and streams data in batches inside per-batch transactions.
| Tool | Direction |
|---|---|
mysql2postgresql |
MySQL → PostgreSQL |
postgresql2mysql |
PostgreSQL → MySQL |
sqlite2postgres |
SQLite → PostgreSQL |
postgres2sqlite |
PostgreSQL → SQLite |
mysql2sqlite |
MySQL → SQLite |
sqlite2mysql |
SQLite → MySQL |
Build
Usage
# MySQL → PostgreSQL
# PostgreSQL → MySQL
# SQLite → PostgreSQL (SQLite file is a positional argument)
# PostgreSQL → SQLite
Running any tool with no arguments prints its help.
Common Options
| Option | Description |
|---|---|
--tables t1,t2 |
Migrate only the specified tables (default: all) |
--batch N |
Rows per batch (default 1000) |
--create-only |
Create schema only, skip data |
--data-only |
Migrate data only, skip schema |
Behavioral Notes
- Schema translation: MySQL/PostgreSQL types are mapped to the target dialect; auto-increment columns map to
SERIAL/BIGSERIAL(PostgreSQL) orAUTO_INCREMENT(MySQL) /INTEGER PRIMARY KEY AUTOINCREMENT(SQLite). CompositeUNIQUEconstraints are preserved as table-level constraints. - Stable batching: reads are
ORDER BYprimary key soLIMIT/OFFSETpagination never duplicates or drops rows. - JSON & large text:
json/jsonb/text/varcharmap toLONGTEXT(MySQL) /TEXT(PostgreSQL / SQLite) to avoid truncation; columns used as keys downgrade toVARCHAR(255)where required. - Case sensitivity: MySQL target tables use the
utf8mb4_bincollation soUNIQUE/primary-key semantics match PostgreSQL (case-sensitive), preventing false duplicates. - Default values: PostgreSQL function defaults such as
timezone('utc', now())are normalized toCURRENT_TIMESTAMP.
🛠 Tech Stack
External Dependencies
- Async Runtime: Tokio 1.53+
- SQLite Implementation: rusqlite 0.30 (bundled)
- UUID Generation: uuid 1.0
- Serialization: Serde 1.0
- Time Handling: Chrono 0.4
Custom Implementations
- Pure Rust Storage Engine: StorageEngine (zero-dependency in-memory database)
- MySQL Protocol: MySqlConnection (native wire protocol)
- PostgreSQL Protocol: PostgresConnection (native wire protocol)
- Type System: SqlValue, Row, QueryResult
- Connection Abstraction: DatabaseConnection trait
- Transaction System: Transaction
- Connection Pools: Pool / SimplePool
- Utilities: SimpleUuid, SimpleLruCache, SimpleError, SqlSanitizer (SQL injection protection)
📚 Documentation
- README.md - English README
- README.zh.md - Chinese README
- DATABASE_REPLACEMENT.md - Database layer replacement details
- DEPENDENCY_OPTIMIZATION.md - Dependency optimization details
- PROJECT_SUMMARY.md - Project summary
🎓 Learning Value
TORM demonstrates:
- How to implement database protocols in Rust
- Type-safe database abstraction design
- Async I/O and network programming
- MySQL and PostgreSQL protocol fundamentals
- Production-grade SQLite implementation
- Zero-dependency utility libraries (UUID, LRU cache, connection pool)
🎯 Use Cases
Production
- ✅ SQLite applications (mobile, desktop, lightweight web)
- ✅ Projects requiring standard SQLite file format (interoperable with other SQLite tools)
- ✅ MySQL applications (web services, enterprise apps, supports MySQL 5.7+)
- ✅ PostgreSQL applications (web services, enterprise apps, supports PostgreSQL 10+)
- ✅ Projects with strict dependency control
Learning & Development
- ✅ Database protocol learning
- ✅ Rust async programming
- ✅ ORM design patterns
📝 License
MIT
🤝 Contributing
Issues and Pull Requests are welcome!