Crate rustdb[−][src]

Expand description

This crate (rustdb) implements a high-performance database written entirely in Rust.

The SQL-like language is relatively minimal, and does not (currently) include features such as joins or views. Instead it has high performance SET .. FROM … and FOR .. FROM statements to access database tables, generally using an INDEX.

The complete language manual is available at run-time via the pre-configured (but optional) init::INITSQL database initialisation string, which also includes many functions which illustrate how the language works, including generic table browsing/editing, date and other functions.

Read-only transactions run immediately and concurrently on a virtual read-only copy of the database, and cannot be blocked. Write transactions run sequentially (and should typically execute in around 100 micro-seconds). The Storage trait allows a variety of underlying storage, including SimpleFileStorage, MemFile and AtomicFile.

Interface

The method Database::run (or alternatively Database::run_timed) is called to execute an SQL query. This takes a Transaction parameter which accumulates SELECT results and which also has methods for accessing input parameters and controlling output. Custom builtin functions implement CExp and have access to the transaction via an EvalEnv parameter, which can be downcast if necessary.

Examples

(a) Simple single-threaded web server using SimpleFileStorage as underlying storage.s

use rustdb::{standard_builtins, AccessPagedData, Database, SharedPagedData, SimpleFileStorage, WebTransaction, INITSQL, BuiltinMap};
use std::net::TcpListener;
use std::sync::Arc;

let sfs = Box::new(SimpleFileStorage::new( "..\\test.rustdb" ));
let spd = Arc::new(SharedPagedData::new(sfs));
let apd = AccessPagedData::new_writer(spd);
let mut bmap = BuiltinMap::new();
standard_builtins( &mut bmap );
let bmap = Arc::new(bmap);
let db = Database::new(apd, INITSQL, bmap);

let listener = TcpListener::bind("127.0.0.1:3000").unwrap();
for tcps in listener.incoming() {
    if let Ok(mut tcps) = tcps {
        if let Ok(mut tr) = WebTransaction::new(&tcps) {
            // tr.trace();
            let sql = "EXEC web.Main()";
            // Execute SQL. http response, SQL output, (status,headers,content) is accumulated in wq.
            db.run_timed(&sql, &mut tr);
            // Write the http response to the TCP stream.
            let _err = tr.write(&mut tcps);
            // Save database changes to disk.
            db.save();
        }
    }
}

(b) See here for more advanced example - an Axum-based webserver using AtomicFile, the ARGON hash function, read-only queries and query logging.

Features

This crate supports two cargo features.

builtin : Allows extra SQL builtin functions to be defined.
max : Exposes maximal interface, including all internal modules (default).

General Design of Database

SortedFile stores fixed size Records in a tree of Pages. SortedFile is used to implement:

Variable length values which are split into fragments - see bytes module - although up to 254 bytes can be stored inline.
Database Table storage. Each record has a 64-bit Id.
Index storage ( an index record refers back to the main table ).

When a page becomes too big, it is split into two pages.

Each page is implemented as a binary tree ( so there is a tree of trees ).

SharedPagedData allows logical database pages to be shared to allow concurrent readers.

AtomicFile ensures that database updates are all or nothing.

ToDo List

Decide whether to have several crates : “Core”, “Lang”, “Server” say. Core being data storage, Lang being SQL-subset, Server being interfacing to http/serde etc.

Call web.Main directly ( using function ptr ).

Unify GenTransaction and WebTransaction. Check multiple file upload works ok with local http parsing.

Implement DROP INDEX, ALTER TABLE, fully implement CREATE INDEX.

Implement email in example program. Replication of log files. Server status.

Sort out error handling for PARSEINT etc.

Work on improving/testing SQL code, browse schema, float I/O. Login.