Crate ledb

Expand description

Lightweight embedded database

Features

Processing documents which implements Serialize and Deserialize traits from serde.
Identifying documents using auto-incrementing integer primary keys.
Indexing any fields of documents using unique or duplicated keys.
Searching and ordering documents using indexed fields or primary key.
Selecting documents using complex filters with fields comparing and logical operations.
Updating documents using rich set of modifiers.
Storing documents into independent storages so called collections.
Flexible query! macro which helps write clear and readable queries.
Using LMDB as backend for document storage and indexing engine.

Usage example

extern crate serde;
#[macro_use]
extern crate serde_derive;
// This allows inserting JSON documents
#[macro_use]
extern crate serde_json;
#[macro_use]
extern crate ledb;
// This allows define typed documents easy
#[macro_use]
extern crate ledb_derive;
extern crate ledb_types;

use ledb::{Storage, Options, IndexKind, KeyType, Filter, Comp, Order, OrderKind, Identifier, Primary, Document};

#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, Document)]
struct MyDoc {
    // primary key field
    #[document(primary)]
    id: Option<Primary>,
    // unique key field
    #[document(unique)]
    title: String,
    // key field
    #[document(index)]
    tag: Vec<String>,
    timestamp: u32,
}

fn main() {
    let db_path = ".test_dbs/my_temp_db";
    let _ = std::fs::remove_dir_all(&db_path);

    // Open storage
    let storage = Storage::new(&db_path, Options::default()).unwrap();
    
    // Get collection
    let collection = storage.collection("my-docs").unwrap();
    
    // Ensure indexes
    query!(index for collection
        title str unique,
        tag str,
        timestamp int unique,
    ).unwrap();

    // Ensure indexes using document type
    query!(index MyDoc for collection).unwrap();
    
    // Insert JSON document
    let first_id = query!(insert into collection {
        "title": "First title",
        "tag": ["some tag", "other tag"],
        "timestamp": 1234567890,
    }).unwrap();
    
    // Insert typed document
    let second_id = collection.insert(&MyDoc {
        id: None,
        title: "Second title".into(),
        tag: vec![],
        timestamp: 1234567657,
    }).unwrap();

    // Find documents
    let found_docs = query!(
        find MyDoc in collection
        where title == "First title"
    ).unwrap().collect::<Result<Vec<_>, _>>().unwrap();
    
    // Update documents
    let n_affected = query!(
        update in collection modify title = "Other title"
        where title == "First title"
    ).unwrap();

    // Find documents with descending ordering
    let found_docs = query!(
        find MyDoc in collection order desc
    ).unwrap().collect::<Result<Vec<_>, _>>().unwrap();

    // Remove documents
    let n_affected = query!(
        remove from collection where title == "Other title"
    ).unwrap();
}

Field names

Field name is a sequence of dot-separated identifiers which represents nesting of value in document. Also fully qualified field names supports wildcard pattern (*).

For example, in document below:

{
    "a": "abc",
    "b": {
        "c": 11
    },
    "d": [
      "a"
    ],
    "s": [
      { n: 1 },
      { n: 2 }
    ],
    "h": {
      "a": { s: "a" },
      "b": { s: "b" }
    }
}

You can access fields by the next ways:

a == "abc"
a += "def"
b.c > 10
b.c += 3
d == "a"
d[-1..] += ["b", "c"]
s.n > 1
h.*.s == "a"

Indexing

Index query example:

query!(
    index for some_collection
        some_field Int unique, // unique index
        other_field.with.sub_field String,
        wildcarded.*.sub_field Int,
        // ...next fields
)

Index kinds

Internal Type	JSON Type	Description
Index	“index”	The values can be duplicated
Unique	“unique”	Each value is unique

Unique index guarantee that each value can be stored once, any duplicates disalowed.

The operation will fail in two cases:

When you try to insert new document which duplicate unique field
When you try to ensure unique index for field which have duplicates

Unique fields is pretty fit for sorting.

TODO: Full-text index kind for searching

Key types

Internal Type	JSON Type	Description
Int	“int”	64-bit signed integers
Float	“float”	64-bit floating point numbers
Bool	“bool”	boolean values
String	“string”	UTF-8 strings
Binary	“binary”	raw binary data

Filters

Comparison operations

Internal Repr	JSON Repr	Query (where)	Description
Eq(value)	{“$eq”: value}	field == val	General Equality
In(Vec)	{“$in”: […values]}	field of […val]	One of
Lt(value)	{“$lt”: value}	field < val	Less than
Le(value)	{“$le”: value}	field <= val	Less than or equal
Gt(value)	{“$gt”: value}	field > val	Greater than
Ge(value)	{“$ge”: value}	field >= val	Greater than or equal
Bw(a, true, b, true)	{“$bw”: [a, true, b, true]}	field in a..b	Between including a b
Bw(a, false, b, false)	{“$bw”: [a, false, b, false]}	field <in> a..b	Between excluding a b
Bw(a, true, b, false)	{“$bw”: [a, true, b, false]}	field in> a..b	Between incl a excl b
Bw(a, false, b, true)	{“$bw”: [a, false, b, true]}	field <in a..b	Between excl a incl b
Has	“$has”	field ?	Has value (not null)

NOTE: To be able to use particular field of document in filters you need create index for it first.

Some examples:

query!(@filter field == 123)
query!(@filter field.subfield != "abc")
query!(@filter field > 123)
query!(@filter field <= 456)
query!(@filter field of [1, 2, 3])
query!(@filter field in 123..456)   // [123 ... 456]
query!(@filter field <in> 123..456) // (123 ... 456)
query!(@filter field <in 123..456)  // (123 ... 456]
query!(@filter field in> 123..456)  // [123 ... 456)

Logical operations

Internal Repr	JSON Repr	Query (where)	Description
Not(Box)	{“$not”: filter}	! filter	Filter is false
And(Vec)	{“$and”: […filters]}	filter && …filters	All filters is true
Or(Vec)	{“$or”: […filters]}	filter \|\| …filters	Any filter is true

NOTE: Be careful with using complex ORs and global NOTs since it may slow down your queries.

Some examples:

// negate filter condition
query!(@filter ! field == "abc")

// and filter conditions
query!(@filter field > 123 && field <= 456)

// or filter conditions
query!(@filter field <= 123 || field > 456)

Results ordering

Internal Repr	JSON Repr	Query (where)	Description
Primary(Asc)	“$asc”	>, asc (default)	Ascending ordering by primary key
Primary(Desc)	“$desc”	<, desc	Descending ordering by primary key
Field(field, Asc)	{“field”: “$asc”}	field >, field asc	Ascending ordering by field
Field(field, Desc)	{“field”: “$desc”}	field <, field desc	Descending ordering by field

Examples:

// ascending ordering by primary key
query!(@order >)
query!(@order asc)

// descending ordering by primary key
query!(@order <)
query!(@order desc)

// ascending ordering by field
query!(@order by field >)
query!(@order by field asc)

// descending ordering by other.field
query!(@order by other.field <)
query!(@order by other.field desc)

Modifiers

Internal Repr	JSON Repr	Query (where)	Description
Set(value)	{“$set”: value}	field = value	Set field value
Delete	“$delete”	field ~	Delete field
Add(value)	{“$add”: value}	field += value	Add value to field
Sub(value)	{“$sub”: value}	field -= value	Substract value from field
Mul(value)	{“$mul”: value}	field *= value	Multiply field to value
Div(value)	{“$div”: value}	field /= value	Divide field to value
Toggle	“$toggle”	field !	Toggle boolean field
Replace(pat, sub)	{“$replace”: [“pat”, “sub”]}	field ~= “pat” “sub”	Replace using regexp
Splice(from, to, Vec)	{“$splice”: [from, to]}	field[from..to] ~	Remove from an array
Splice(from, to, Vec)	{“$splice”: [from, to, …ins]}	field[from..to] = ins	Splice an array
Merge(object)	{“$merge”: object}	field ~= object	Merge an object

The negative range value means position from end of an array:

-1 the end of an array
-2 the last element
-3 the element before the last
…and so on

Extended behavior of modifiers

Internal Repr	JSON Repr	Query (where)	Description
Add(values)	{“$add”: […values]}	field += [..values]	Add unique values to an array as a set
Sub(values)	{“$sub”: […values]}	field -= [..values]	Remove unique values to an array as a set
Add(text)	{“$add”: “text”}	field += “text”	Append text to a string

Examples:

// set single fields
query!(@modify field = 123)
query!(@modify other.field = "abc")

// set multiple fields
query!(@modify 
    field = 1;
    other.field = "abc";
)

// numeric operations
query!(@modify field += 1) // add value to field
query!(@modify field -= 1) // substract value from field
query!(@modify field *= 1) // multiply field to value
query!(@modify field /= 1) // divide field to value

query!(@modify - field) // remove field
query!(@modify ! field) // toggle boolean field

query!(@modify str += "addon") // append piece to string
query!(@modify str ~= "abc" "def") // regexp replace

// modify array as list
query!(@modify list[0..0] = [1, 2, 3]) // prepend to array
query!(@modify list[-1..0] = [1, 2, 3]) // append to array
query!(@modify - list[1..2]) // remove from array
query!(@modify list[1..2] = [1, 2, 3]) // splice array

// modify array as set
query!(@modify set += [1, 2, 3]) // add elements
query!(@modify set -= [4, 5, 6]) // remove elements

// merge an object
query!(@modify obj ~= { a: true, b: "abc", c: 123 })
query!(@modify obj ~= extra)