Crate lbasedb

lbasedb is a powerful low level DBMS that is focused on dataset structure. The algorithms are optimized for the compact data storage and for high performance on get and append operations. Particularly, due to this, deleting or indexing are not supported. The allowed data types are also limited (integers, floats and bytes) for making easy integration with C-like or similar common interfaces (like Python, CUDA, JSON and so on). The database has asynchronous access to the entities powered by tokio. It is supposed to be used for the data that have billions and more records and thousands columns of simple data types that must be appended without extra overhead.

Usage example:

use lbasedb::prelude::*;

let conn = Conn::new("./tmp/db").await?;
 
if !conn.feed_exists("xyz").await {
    conn.feed_add("xyz").await?;
}
 
println!(
    "Feed list: {:?}", 
     conn.feed_list().await.iter()
         .map(|i| i.get_name())
         .collect::<Vec<String>>()
);
 
if !conn.col_exists("xyz", "x").await {
    conn.col_add("xyz", "x", "Int64").await?;
}
 
if !conn.col_exists("xyz", "y").await {
    conn.col_add("xyz", "y", "Float64").await?;
}
 
println!(
    "Col list: {:?}", 
    conn.col_list("xyz")?.iter()
        .map(|i| i.get_name())
        .collect::<Vec<String>>()
);
 
if conn.size_get("xyz")? == 0 {
    let ds: Dataset = std::collections::HashMap::from([
        ("x".to_string(), vec![Dataunit::I(2), Dataunit::I(5)]),
        ("y".to_string(), vec![Dataunit::F(2.15), Dataunit::F(5.55)]),
    ]);
    conn.data_push("xyz", &ds).await?;
}
 
println!("Size: {:?}", conn.size_get("xyz")?);
 
let ds = conn.data_get("xyz", 0, 2, 
                       &["x".to_string(), "y".to_string()]).await?;
println!("ds = {:?}", ds);

Re-exports

pub use crate::prelude::*;

Modules

col

Col is a wrapper over Seq for an arbitrary sized datatype so it can be represented as its bytes and stored in a file using the Seq interface.

conn

Conn is a basic structure for the connection that provides the full interface to the DBMS.

dataset

Dataset is an alias for HashMap that keeps vectors of basic data (provided as Dataunit) by keys, so it represents a common dataset having columns (the keys) and rows.

datatype

Converting between datatypes for different purposes: into bytes and back, serializations, from and into strings and so on.

items

FeedItem and ColItem are the basic entities of the DBMS that are responsible for the options of feeds (like tables or collections) and cols (like columns of fields).

list

List is a way to store structures in a file mainly for the management purposes. Unlike Seq or Col philosophy it supports fetching by unique key and removing, although it is a wrapper over Col. The performance of the operations is relatively low.

prelude

Common used imports such that Conn, Dataset and others.

seq

Seq is the basic structure to manage the data storing in a file. It works exactly with byte blocks, supports asynchronous interface, allows to fetch, push and update data.

utils

Common functions of the library that mainly relate to byte converting.

Macros

path_concat

Concatenate given paths.