Crate ferrule_sql 

ferrule-sql — the embeddable, synchronous, bounded-memory SQL core.

This crate owns the unified neutral Value/Row types, the DatabaseUrl parser, the Connection trait and its per-backend drivers, the connect dispatcher (direct, HTTP-proxy, and SSH-tunnel transports), the transaction helpers, and the cross-backend copy / bulk-load write path. It carries no rendering (tabled) or credential-resolution (ferrule-config) dependency, so it can be embedded by callers that supply already-resolved connection details.

Backends are feature-gated (postgres, mysql, mssql, sqlite, oracle); the SSH tunnel transport is behind ssh. The default feature set is empty — enable the backends you need.

Why these properties

• Synchronous public API. No async fn / Future crosses the crate boundary, so a host with no runtime of its own can call straight through. The async network drivers are still used — each Connection owns a private current-thread tokio runtime and blocks on it — but that runtime is an implementation detail. (SQLite and Oracle are natively synchronous and call through directly.)
• Bounded memory. Connection::query_cursor streams from a native database cursor one back-pressured batch at a time, and write_rows flushes an arbitrarily large row iterator in fixed-size batches — both stay O(batch) regardless of total row count. The eager Connection::query still materializes a full Vec<Row>, but it is capped by per-cell / per-row / per-result SizeGuards so a pathological result fails fast instead of OOMing.
• Caller-resolved credentials. connect takes the password as a secrecy::SecretString on ConnectOptions; the crate performs no credential resolution and depends on no keyring / prompt library.

Embedding flow

The three steps a host follows — connect → streaming read → batched write — line up with the runnable examples/embed.rs (cargo run -p ferrule-sql --example embed --features sqlite):

1. Connect with a resolved secret

Parse a DatabaseUrl, hand the host-resolved credential to ConnectOptions::password, and call connect. The returned Connection owns its private runtime and blocks on every call.

2. Streaming read (the bounded cursor)

Connection::query_cursor returns a RowCursor. Pull rows with RowCursor::next_batch(n) (a bounded chunk) or by iterating; either way the driver only fetches more from the server as you consume, so peak memory is O(batch), never the whole result. The cursor borrows the connection for its lifetime, so scope it before issuing the next statement.

3. Batched write (back-pressured, structured report)

write_rows consumes any IntoIterator<Item = Row> and flushes it in fixed-size batches (WriteOptions::batch_size), buffering one batch at a time. It reuses the cross-DB copy path’s SQL generation and transaction control and returns a WriteReport naming exactly which batches / rows landed and which were rejected. Pair it with the cursor from step 2 for an end-to-end bounded-memory pipe.

use ferrule_sql::{
    connect, write_rows, Backend, ColumnInfo, ConnectOptions, DatabaseUrl,
    Row, TypeHint, Value, WriteOptions,
};
use secrecy::SecretString;

// 1. Connect. The password is a caller-resolved `SecretString`;
//    SQLite ignores it, a networked backend would consume it.
let url = DatabaseUrl::parse("sqlite:///tmp/embed-demo.db")?;
let opts = ConnectOptions {
    insecure: false,
    password: Some(SecretString::from("resolved-by-the-host")),
};
let mut conn = connect(&url, &opts, None)?;
conn.execute("CREATE TABLE t (id INTEGER PRIMARY KEY, name TEXT)")?;

// 2. Streaming read at bounded memory: two rows per pull.
{
    let mut cursor = conn.query_cursor("SELECT id, name FROM t ORDER BY id")?;
    while !cursor.next_batch(2)?.is_empty() {
        // process this bounded batch, then pull the next one
    }
}

// 3. Batched write: one batch buffered at a time.
let columns = [
    ColumnInfo { name: "id".into(), type_hint: TypeHint::Int64, nullable: false },
    ColumnInfo { name: "name".into(), type_hint: TypeHint::String, nullable: true },
];
let rows: Vec<Row> = (1..=1000)
    .map(|i| vec![Value::Int64(i), Value::String(format!("n{i}"))])
    .collect();
let report = write_rows(
    &mut *conn,
    Backend::Sqlite,
    "t",
    &columns,
    rows,
    &WriteOptions { batch_size: 200, ..Default::default() },
)?;
assert!(report.is_complete());

Reentrancy

The private runtime is current-thread, so a Connection (and its RowCursor) must not be driven from inside another block_on on the same thread. An async host hops to a blocking thread (tokio::task::spawn_blocking or a dedicated OS thread) first.

Re-exports

pub use backend::connect_with_tunnel;

pub use backend::Backend;

pub use backend::connect;

pub use connection::BulkInsert;

pub use connection::ConnectOptions;

pub use connection::Connection;

pub use connection::ExecutionSummary;

pub use connection::ForeignKey;

pub use connection::QueryResult;

pub use connection::SchemaInfo;

pub use connection::StatementResult;

pub use copy::AllTablesOptions;

pub use copy::BulkMode;

pub use copy::CopyFormat;

pub use copy::CopyOptions;

pub use copy::CopySource;

pub use copy::CycleError;

pub use copy::IfExists;

pub use copy::copy_all_tables;

pub use copy::copy_rows;

pub use copy::discover_tables;

pub use copy::quote_identifier;

pub use copy::topo_sort;

pub use copy::translate_ddl;

pub use copy::translate_type;

pub use dialect::Dialect;

pub use error::SqlError;

pub use guard::SizeGuards;

pub use proxy::ProxiedConnection;

pub use proxy::ProxyConfig;

pub use proxy::is_no_proxy;

pub use proxy::resolve_proxy_from_env;

pub use query_builder::apply_paging;

pub use render::quote_string;

pub use render::render_value;

pub use stream::BoxRowStream;

pub use stream::DEFAULT_CURSOR_CAPACITY;

pub use stream::RowCursor;

pub use sync::SyncConnection;

pub use transaction::begin_transaction;

pub use transaction::commit_transaction;

pub use transaction::rollback_transaction;

pub use tunnel::SshConfig;

pub use tunnel::KeySource;

pub use tunnel::SshSession;

pub use tunnel::TunnelError;

pub use tunnel::TunnelHandle;

pub use tunnel::TunnelStream;

pub use tunnel::TunnelTransport;

pub use tunnel::TunnelTransportResult;

pub use tunnel::TunneledConnection;

pub use url::DatabaseUrl;

pub use value::ColumnInfo;

pub use value::Row;

pub use value::TypeHint;

pub use value::Value;

pub use write::BatchOutcome;

pub use write::DEFAULT_WRITE_BATCH;

pub use write::RejectedBatch;

pub use write::RejectedRow;

pub use write::WriteMode;

pub use write::WriteOptions;

pub use write::WriteReport;

pub use write::write_rows;

Modules

backend

backends

Per-backend driver modules, one feature-gated submodule per backend.

connection

copy

Cross-DB row copy: stream rows from one backend’s table or query into another backend’s table, translating types via the unified Value enum.

dialect

SQL dialect classification shared across the driver and write-path modules.

error

guard

Constant-memory size guards for the read paths.

proxy

HTTP CONNECT proxy support.

query_builder

render

SQL-literal rendering helpers for inline value substitution.

stream

Bounded-memory streaming reads — the native-cursor counterpart to the eager Connection::query.

sync

Synchronous wrapper that turns an async backend driver into the public blocking Connection API.

transaction

Backend-aware transaction-control helpers.

tunnel

SSH tunnel support — types and lifecycle.

url

value

write

Embeddable batched write path with structured partial-failure reporting.