Crate rudb 

The embedding API: connections, prepared statements, configuration and results.

Rank 13 in the layer rule. See xtask/layers.toml and spec/18-package-layout.md.

This is the crate somebody who wants a database depends on. Everything below it is an implementation detail that happens to be published, and everything above it is a different way of reaching this same API: rudb-c-api is this over the C ABI and rudb-cli is this behind a prompt.

use rudb::{Database, Field, LogicalType, Value};

let db = Database::new();
db.create_table("t", vec![Field::new("x", LogicalType::Integer)])?;
db.append("t", &[vec![Value::Integer(1)], vec![Value::Integer(7)]])?;

let result = db.query("SELECT x FROM t WHERE x > 5")?;
assert_eq!(result.len(), 1);
assert_eq!(result.value_at(0, 0), Value::Integer(7));

What a query is today

Parse, bind, optimize, execute. The optimizer is one pass, which is column pruning, so a scan reads the columns something above it asks for and the plan is otherwise the shape the binder built it. The rest of spec/09-optimizer.md’s sequence is M1 work. There are no transactions, and rudb-txn is in the dependency list for the same reason: the seam is where it will be and nothing goes through it yet.

Database::create_table and Database::append are how rows get in without SQL, and they are a real part of the API rather than a test helper, since an embedded analytical database gets most of its data from a program rather than from a string of SQL. The DDL statements bind to the same catalog calls these make.

Prepared statements

Database::prepare and Connection::prepare parse a statement once and hand back a Prepared that runs with values for its parameters, written ?, ?1, $1 or $name. The statement is bound again for each set of values rather than planned once and filled in, because an analytical plan depends on what the values are: a scan that keeps one row in a million and a scan that keeps half the table want different plans, and the binder is cheap next to either.

Threading

A Database is a handle. Cloning one, or calling Database::connect, gives another handle on the same database, and every method takes &self, so the program embedding this is the one that decides how many threads there are. The catalog is behind a reader writer lock: any number of queries read at once and a statement that writes has the database to itself while it runs.

That lock is the whole of the concurrency story until rudb-txn has one. A statement that writes is serialized against every reader rather than isolated from them, which is correct and is coarse, and the thing that makes it finer is a transaction rather than a different lock.

Stopping a query

Two ways, and they answer two different questions. Config::with_query_timeout is a limit the statement enforces on itself, which is what a harness running somebody else’s SQL wants, because the thing it is guarding against is a query that never ends rather than a person who changed their mind. Connection::interrupt is the other end of a token somebody else holds, which is what a signal handler wants, and it is DuckDB’s model as well: duckdb_interrupt takes a connection.

use std::time::Duration;
use rudb::{Config, Database};

let db = Database::with_config(Config::new().with_query_timeout(Duration::from_millis(50)));
let error = db.query("SELECT count(*) FROM range(100000000000)").expect_err("too slow");
assert_eq!(error.code().duckdb_name(), "Interrupt Error");

A query stops at its next chunk boundary rather than immediately, which is a thousand rows of work later, and the reason is in Cancel. Nothing is rolled back, because there are no transactions yet: a stopped INSERT has written nothing, since the source runs to completion before anything is appended, and a stopped CREATE TABLE AS SELECT leaves no table behind for the same reason. That stops being true the day the writes stream, and the thing that makes it true again is a transaction.

Running out of memory

The third way a query stops. Config::with_memory_limit is a budget for the whole database, and the operators that buffer without bound charge what they hold against it. A query that asks for more than is left stops with an Out of Memory Error rather than being killed from outside, which is the difference between a harness that reports a result for a file and a harness that reports nothing because the process died.

There is a budget without anybody setting one. It is eighty percent of what the machine has, the way DuckDB’s is, and Config::memory_limit says what it is on this machine and what to do to turn it off. The default is the whole point of the error: a limit that has to be typed is a limit that is not there on the machine where the query went wrong.

use rudb::{Config, Database};

let db = Database::with_config(Config::new().with_memory_limit(1 << 20));
let error = db.query("SELECT * FROM range(10000000) ORDER BY range").expect_err("too large");
assert_eq!(error.code().duckdb_name(), "Out of Memory Error");
// And the budget is given back, so the connection is still usable.
assert_eq!(db.memory().used(), 0);
assert_eq!(db.value("SELECT 1").expect("a small query still runs"), rudb::Value::Integer(1));

What is counted is what the operators said they were holding, which is not the resident size of the process. rudb_common::Memory says exactly what that covers and which direction it errs in.

Modules

arrow

Arrow interchange, which is what QueryResult::to_arrow hands back.

Structs

Cancel

A reason for a running query to stop, which is either somebody asking or a clock running out.

Chunk

A batch of columns of equal length.

Config

The settings a database is opened with.

Connection

A connection to a database.

Database

An in process database.

Error

An error, carrying a code, a message and optionally where in the query it happened.

Field

A named field of a STRUCT or a UNION, and a named column of a table.

Prepared

A prepared statement.

QueryResult

The rows a query produced, with the names and types of its columns.

Span

A byte range into the query text.

Statement

One statement out of a script, and where it started.

Enums

ErrorCode

What kind of thing went wrong.

LogicalType

What SQL thinks a value is.

RowOrder

Whether the rows a statement produces come back in an order it asked for.

Value

A single SQL value.

Functions

accepts

Whether the grammar accepts this statement.

is_complete

Whether a script ends on a statement boundary.

line_and_column

Where in the text an error is about, as a line and a column, both counting from one.

parse_size

A size written the way a person writes one, in bytes.

parses

Whether the grammar accepts this statement, as a plain yes or no.

row_order

How the rows of a statement are ordered, as far as the text says.

split

The text split into statements, with text that does not tokenize handed back whole.

statements

Every statement in a script, in order.

where_it_happened

The span of an error, as a line and a column into the statement it came from.

Type Aliases

Result

The result type used everywhere in the workspace.