Expand description
This crate (rustdb) implements a high-performance database written entirely in Rust.
The SQL-like language is relatively minimal, and does not (currently) include features such as joins or views. Instead it has high performance SET .. FROM … and FOR .. FROM statements to access database tables, generally using an INDEX.
Read-only transactions run immediately and concurrently on a virtual read-only copy of the database, and cannot be blocked. Write transactions run sequentially (and should typically execute in around 100 micro-seconds). The Storage trait allows a variety of underlying storage, including SimpleFileStorage, MemFile and AtomicFile.
Transactions that modify the database can be logged, which allows for database replication.
Interface
The method Database::run is called to execute an SQL query. This takes a Transaction parameter which accumulates SELECT results and which also has methods for accessing input parameters and controlling output. Custom builtin functions implement CExp and have access to the transaction via an EvalEnv parameter, which can be downcast if necessary.
It is also possible to access the table data directly, see email_loop in example program.
Example
See here for an example program - a webserver, with timed jobs, password hashing, data compression, email transmission and database replication. Also has a Manual for the SQL-like language, user interface for database browsing/editing etc.
Features
This crate supports the following cargo features:
gentrans
: enables gentrans module ( sample implementation of Transaction ).builtin
: Allows extra SQL builtin functions to be defined.max
: maximal interface, including internal modules.verify
: Allows database structure to be verified using builtin function VERIFYDB.pack
: Allows database pages to be packed using builtin function REPACKFILE.
By default, all features are enabled.
General Design of Database
SortedFile stores fixed size Records in a tree of Pages. SortedFile is used to implement:
-
Database Table storage. Each fixed size record has a 64-bit Id.
-
Variable length values which are split into fragments, although up to 249 bytes can be stored in the fixed size record.
-
Index storage - an index record refers back to the main table using the 64-bit Id.
When a page becomes too big, it is split into two pages.
Each page is implemented as a binary tree ( so there is a tree of trees ).
SharedPagedData allows logical database pages to be shared to allow concurrent readers.
AtomicFile ensures that database updates are all or nothing.
The hierarchy overall: Table -> SortedFile -> PagedData -> CompactFile -> AtomicFile -> Storage.
Test example
use rustdb::*;
use std::sync::Arc;
let file = Box::new(MemFile::default());
let upd = Box::new(MemFile::default());
let stg = Box::new(AtomicFile::new(file, upd));
let spd = Arc::new(SharedPagedData::new(stg));
let wapd = AccessPagedData::new_writer(spd);
let mut bmap = BuiltinMap::default();
standard_builtins(&mut bmap);
let bmap = Arc::new(bmap);
let db = Database::new(wapd, "", bmap);
let mut tr = GenTransaction::default();
let sql = "
CREATE SCHEMA test GO
CREATE TABLE test.Cust(Name string) GO
INSERT INTO test.Cust(Name) VALUES ('freddy')
SELECT Name FROM test.Cust
";
db.run(&sql, &mut tr);
assert!( db.changed() );
assert!( db.save() > 0 );
assert!( tr.rp.output == b"freddy" );
Re-exports
pub use crate::atomfile::AtomicFile;
pub use crate::builtin::standard_builtins;
pub use crate::pstore::AccessPagedData;
pub use crate::stg::MemFile;
pub use crate::stg::SimpleFileStorage;
pub use crate::stg::Storage;
pub use crate::gentrans::GenTransaction;
pub use crate::gentrans::Part;
pub use crate::builtin::check_types;
pub use crate::compile::c_bool;
pub use crate::compile::c_float;
pub use crate::compile::c_int;
pub use crate::compile::c_value;
pub use crate::exec::EvalEnv;
pub use crate::expr::ObjRef;
pub use crate::expr::Block;
pub use crate::expr::DataKind;
pub use crate::expr::Expr;
pub use crate::run::CExp;
pub use crate::run::CExpPtr;
pub use crate::run::CompileFunc;
pub use crate::value::Value;
Modules
- Compilation of builtin functions, standard_builtins.
- Storage of variable length values : ByteStorage.
- Structs that implement CExp trait.
- CompactFile : storage of logical pages in smaller regions of backing storage.
- Functions to compile parsed expressions, checking types.
- EvalEnv : Instruction execution.
- GenTransaction ( implementation of Transaction ).
- Page for SortedFile.
- Page storage and sharing, SharedPagedData and AccessPagedData. Note: several structs and their fields are pub to allow diagnostics but are subject to change.
- Instruction and other run time types.
- SortedFile : Record storage.
- Backing Storage for database. See also AtomicFile.
- System table functions.
- Utility functions and macros, SmallSet.
- Run-time Value.
Structs
- Database with SQL-like interface.
Traits
- Input/Output message. Query and Response.
Type Aliases
- Map that defines SQL pre-defined functions.
Rc<Database>
Arc<Vec<u8>>