Crate ledb

Expand description

§Lightweight embedded database

§Features

Processing documents which implements Serialize and Deserialize traits from serde.
Identifying documents using auto-incrementing integer primary keys.
Indexing any fields of documents using unique or duplicated keys.
Searching and ordering documents using indexed fields or primary key.
Selecting documents using complex filters with fields comparing and logical operations.
Updating documents using rich set of modifiers.
Storing documents into independent storages so called collections.
Flexible query! macro which helps write clear and readable queries.
Using LMDB as backend for document storage and indexing engine.

§Usage example

use serde::{Serialize, Deserialize};
use serde_json::json;
use ledb::{Storage, Options, IndexKind, KeyType, Filter, Comp, Order, OrderKind, Identifier, Primary, Document, query, query_extr};

#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, Document)]
struct MyDoc {
    // primary key field
    #[document(primary)]
    id: Option<Primary>,
    // unique key field
    #[document(unique)]
    title: String,
    // key field
    #[document(index)]
    tag: Vec<String>,
    timestamp: u32,
}

fn main() {
    let db_path = ".test_dbs/my_temp_db";
    let _ = std::fs::remove_dir_all(&db_path);

    // Open storage
    let storage = Storage::new(&db_path, Options::default()).unwrap();

    // Get collection
    let collection = storage.collection("my-docs").unwrap();

    // Ensure indexes
    query!(index for collection
        title str unique,
        tag str,
        timestamp int unique,
    ).unwrap();

    // Ensure indexes using document type
    query!(index MyDoc for collection).unwrap();

    // Insert JSON document
    let first_id = query!(insert into collection {
        "title": "First title",
        "tag": ["some tag", "other tag"],
        "timestamp": 1234567890,
    }).unwrap();

    // Insert typed document
    let second_id = collection.insert(&MyDoc {
        id: None,
        title: "Second title".into(),
        tag: vec![],
        timestamp: 1234567657,
    }).unwrap();

    // Find documents
    let found_docs = query!(
        find MyDoc in collection
        where title == "First title"
    ).unwrap().collect::<Result<Vec<_>, _>>().unwrap();

    // Update documents
    let n_affected = query!(
        update in collection modify title = "Other title"
        where title == "First title"
    ).unwrap();

    // Find documents with descending ordering
    let found_docs = query!(
        find MyDoc in collection order desc
    ).unwrap().collect::<Result<Vec<_>, _>>().unwrap();

    // Remove documents
    let n_affected = query!(
        remove from collection where title == "Other title"
    ).unwrap();
}

§Field names

Field name is a sequence of dot-separated identifiers which represents nesting of value in document. Also fully qualified field names supports wildcard pattern (*).

For example, in document below:

{
    "a": "abc",
    "b": {
        "c": 11
    },
    "d": [
      "a"
    ],
    "s": [
      { n: 1 },
      { n: 2 }
    ],
    "h": {
      "a": { s: "a" },
      "b": { s: "b" }
    }
}

You can access fields by the next ways:

a == "abc"
a += "def"
b.c > 10
b.c += 3
d == "a"
d[-1..] += ["b", "c"]
s.n > 1
h.*.s == "a"

§Indexing

Index query example:

query!(
    index for some_collection
        some_field Int unique, // unique index
        other_field.with.sub_field String,
        wildcarded.*.sub_field Int,
        // ...next fields
)

§Index kinds

Internal Type	JSON Type	Description
Index	“index”	The values can be duplicated
Unique	“unique”	Each value is unique

Unique index guarantee that each value can be stored once, any duplicates disalowed.

The operation will fail in two cases:

When you try to insert new document which duplicate unique field
When you try to ensure unique index for field which have duplicates

Unique fields is pretty fit for sorting.

TODO: Full-text index kind for searching

§Key types

Internal Type	JSON Type	Description
Int	“int”	64-bit signed integers
Float	“float”	64-bit floating point numbers
Bool	“bool”	boolean values
String	“string”	UTF-8 strings
Binary	“binary”	raw binary data

§Filters

§Comparison operations

Internal Repr	JSON Repr	Query (where)	Description
Eq(value)	{“$eq”: value}	field == val	General Equality
In(Vec)	{“$in”: […values]}	field of […val]	One of
Lt(value)	{“$lt”: value}	field < val	Less than
Le(value)	{“$le”: value}	field <= val	Less than or equal
Gt(value)	{“$gt”: value}	field > val	Greater than
Ge(value)	{“$ge”: value}	field >= val	Greater than or equal
Bw(a, true, b, true)	{“$bw”: [a, true, b, true]}	field in a..b	Between including a b
Bw(a, false, b, false)	{“$bw”: [a, false, b, false]}	field <in> a..b	Between excluding a b
Bw(a, true, b, false)	{“$bw”: [a, true, b, false]}	field in> a..b	Between incl a excl b
Bw(a, false, b, true)	{“$bw”: [a, false, b, true]}	field <in a..b	Between excl a incl b
Has	“$has”	field ?	Has value (not null)

NOTE: To be able to use particular field of document in filters you need create index for it first.

Some examples:

query!(@filter field == 123)
query!(@filter field.subfield != "abc")
query!(@filter field > 123)
query!(@filter field <= 456)
query!(@filter field of [1, 2, 3])
query!(@filter field in 123..456)   // [123 ... 456]
query!(@filter field <in> 123..456) // (123 ... 456)
query!(@filter field <in 123..456)  // (123 ... 456]
query!(@filter field in> 123..456)  // [123 ... 456)

§Logical operations

Internal Repr	JSON Repr	Query (where)	Description
Not(Box)	{“$not”: filter}	! filter	Filter is false
And(Vec)	{“$and”: […filters]}	filter && …filters	All filters is true
Or(Vec)	{“$or”: […filters]}	filter \|\| …filters	Any filter is true

NOTE: Be careful with using complex ORs and global NOTs since it may slow down your queries.

Some examples:

// negate filter condition
query!(@filter ! field == "abc")

// and filter conditions
query!(@filter field > 123 && field <= 456)

// or filter conditions
query!(@filter field <= 123 || field > 456)

§Results ordering

Internal Repr	JSON Repr	Query (where)	Description
Primary(Asc)	“$asc”	>, asc (default)	Ascending ordering by primary key
Primary(Desc)	“$desc”	<, desc	Descending ordering by primary key
Field(field, Asc)	{“field”: “$asc”}	field >, field asc	Ascending ordering by field
Field(field, Desc)	{“field”: “$desc”}	field <, field desc	Descending ordering by field

Examples:

// ascending ordering by primary key
query!(@order >)
query!(@order asc)

// descending ordering by primary key
query!(@order <)
query!(@order desc)

// ascending ordering by field
query!(@order by field >)
query!(@order by field asc)

// descending ordering by other.field
query!(@order by other.field <)
query!(@order by other.field desc)

§Modifiers

Internal Repr	JSON Repr	Query (where)	Description
Set(value)	{“$set”: value}	field = value	Set field value
Delete	“$delete”	field ~	Delete field
Add(value)	{“$add”: value}	field += value	Add value to field
Sub(value)	{“$sub”: value}	field -= value	Substract value from field
Mul(value)	{“$mul”: value}	field *= value	Multiply field to value
Div(value)	{“$div”: value}	field /= value	Divide field to value
Toggle	“$toggle”	field !	Toggle boolean field
Replace(pat, sub)	{“$replace”: [“pat”, “sub”]}	field ~= “pat” “sub”	Replace using regexp
Splice(from, to, Vec)	{“$splice”: [from, to]}	field[from..to] ~	Remove from an array
Splice(from, to, Vec)	{“$splice”: [from, to, …ins]}	field[from..to] = ins	Splice an array
Merge(object)	{“$merge”: object}	field ~= object	Merge an object

The negative range value means position from end of an array:

-1 the end of an array
-2 the last element
-3 the element before the last
…and so on

§Extended behavior of modifiers

Internal Repr	JSON Repr	Query (where)	Description
Add(values)	{“$add”: […values]}	field += [..values]	Add unique values to an array as a set
Sub(values)	{“$sub”: […values]}	field -= [..values]	Remove unique values to an array as a set
Add(text)	{“$add”: “text”}	field += “text”	Append text to a string