Crate bitrush_index

Bitrush-Index

Bitrush-Index is a Rust library that provides a serializable bitmap index able to index millions values/sec on a single thread. On default this library build bitmap-index using ozbcbitmap but if you want you can also use another compressed/uncrompressed bitmap. Only equality-query (A = X) are supported.

Example

 use bitrush_index::{
     BitmapIndex,
     OZBCBitmap,
 };
 
 use rand::Rng;
 use std::time::Instant;
 
 fn main() {
     // number of values to insert (defalut 10M)
     const N: usize = (1 << 20) * 10;
     let mut rng = rand::thread_rng();
 
     let build_options = bitrush_index::new_default_index_options::<u32>();
     let mut b_index = match BitmapIndex::<OZBCBitmap, u32>::new(build_options) {
         Ok(b_index) => b_index,
         Err(err) => panic!("Error occured creating bitmap index: {:?}", err)
     };
     let mut values: Vec<u32> = Vec::new();
 
     for _i in 0..N {
         let val: u32 = rng.gen::<u32>();
         values.push(val);
     }
     println!("--------------------------------------------------");
     println!("Inserting {} values in bitmap index...", N);
     let timer = Instant::now();
     //if index is opened in memory mode you can ignore the push_values result.
     let _result_index = b_index.push_values(&values); 
     let time_b_index_insert = timer.elapsed();
     println!("Bitmap index created in {:?}.", time_b_index_insert);
     println!("Insert per second = {}.", N / (time_b_index_insert.as_millis() as usize) * 1000);
     println!("--------------------------------------------------");
 
     let random_index: usize = rng.gen::<usize>() % values.len();
     let val_to_find = values[random_index];
     let timer = Instant::now();
     let values_indexes: Vec<u64> = match b_index.run_query(val_to_find, None, None) {
         Ok(indexes) => indexes,
         Err(err) => panic!("Error occured running looking for value = {}, error: {:?}", val_to_find, err)
     };
 
     let time_linear_search = timer.elapsed();
     println!("Bitmap index search runned in {:?}, match values founded: {}.", time_linear_search, values_indexes.len());
     println!("--------------------------------------------------");
 }

Structs

BitmapIndex

BitmapIndex struct that requires a bitmap that implement Bitmap trait and a type that implement BitValue trait.

BuildOptions

BuildOptions defines how many bitmap compose a BitmapIndex and how many values must represent every chunk in BitmapIndex. On a BitmapIndex of a BitValue ‘U’ with size_in_bit(U) = L, bit_block_size represent the size in bit of each sub-index. For example a BitmapIndex of a BitValue ‘U=u16’ with ‘size_in_bit(u16) = 16’ and bit_block_size = 8 is composed from ‘16 / 8 = 2’ blocks of ‘2^8 = 256’ bitmaps each, so is composed from ‘2 * 256 = 512’ bitmaps.

MetaData

MetaData defines the meta data of a BitmapIndex.

OZBCBitmap

StorageIdx

StorageIdx defines a BitmapIndex opened in read-only storage mode.

Enums

ChunkSize

BitmapIndex works in chunks, each chunk represent ChunkSize values, possibily values for ChunkSize are: 1 Mega, 2 Mega, 4 Mega, 8 Mega, 16 Mega, 32 Mega. If BitmapIndex is created in storage mode, bitmaps are serialized every time a chunk is full.

Traits

BitValue

BitValue trait. On default BitValue is implemented for: u8, u16, u32, u64, u128, i8, i16, i32, i64, i128.

Bitmap

Functions

new_default_index_options

Return default options to create a BitmapIndex.