Crate mtbl_sys 

Rust FFI interface to the mtbl C library for dealing with SSTables (write-once sorted map files).

SSTables are basically constant on-disk maps, like those used by CDB (which also has Rust bindings, except using sorted maps instead of hashmaps.

Version 0.2 of mtbl-sys covers the 0.8 version of the MTBL C library.

Function documentation

For documentation about each function, see MTBL’s extensive man pages, e.g. man mtbl_reader.

More details about MTBL

Quoting from the MTBL documentation:

mtbl is not a database library. It does not provide an updateable key-value data store, but rather exposes primitives for creating, searching and merging SSTable files. Unlike databases which use the SSTable data structure internally as part of their data store, management of SSTable files – creation, merging, deletion, combining of search results from multiple SSTables – is left to the discretion of the mtbl library user.

mtbl SSTable files consist of a sequence of data blocks containing sorted key-value pairs, where keys and values are arbitrary byte arrays. Data blocks are optionally compressed using zlib or the Snappy library. The data blocks are followed by an index block, allowing for fast searches over the keyspace.

The basic mtbl interface is the writer, which receives a sequence of key-value pairs in sorted order with no duplicate keys, and writes them to data blocks in the SSTable output file. An index containing offsets to data blocks and the last key in each data block is buffered in memory until the writer object is closed, at which point the index is written to the end of the SSTable file. This allows SSTable files to be written in a single pass with sequential I/O operations only.

Once written, SSTable files can be searched using the mtbl reader interface. Searches can retrieve key-value pairs based on an exact key match, a key prefix match, or a key range. Results are retrieved using a simple iterator interface.

The mtbl library also provides two utility interfaces which facilitate a sort-and-merge workflow for bulk data loading. The sorter interface receives arbitrarily ordered key-value pairs and provides them in sorted order, buffering to disk as needed. The merger interface reads from multiple SSTables simultaneously and provides the key-value pairs from the combined inputs in sorted order. Since mtbl does not allow duplicate keys in an SSTable file, both the sorter and merger interfaces require a caller-provided merge function which will be called to merge multiple values for the same key. These interfaces also make use of sequential I/O operations only.

Structs

mtbl_fileset

mtbl_fileset_options

mtbl_iter

mtbl_merger

mtbl_merger_options

mtbl_metadata

mtbl_reader

mtbl_reader_options

mtbl_sorter

mtbl_sorter_options

mtbl_source

mtbl_writer

mtbl_writer_options

Enums

CompressionType

Compression method used when writing an MTBL file.

MtblRes

Functions

mtbl_crc32c^⚠

mtbl_fileset_destroy^⚠

mtbl_fileset_init^⚠

mtbl_fileset_options_destroy^⚠

mtbl_fileset_options_init^⚠

mtbl_fileset_options_set_merge_func^⚠

mtbl_fileset_options_set_reload_interval^⚠

mtbl_fileset_reload^⚠

mtbl_fileset_reload_now^⚠

mtbl_fileset_source^⚠

mtbl_fixed_decode32^⚠

mtbl_fixed_decode64^⚠

mtbl_fixed_encode32^⚠

mtbl_fixed_encode64^⚠

mtbl_iter_destroy^⚠

mtbl_iter_next^⚠

mtbl_merger_add_source^⚠

mtbl_merger_destroy^⚠

mtbl_merger_init^⚠

mtbl_merger_options_destroy^⚠

mtbl_merger_options_init^⚠

mtbl_merger_options_set_merge_func^⚠

mtbl_merger_source^⚠

mtbl_metadata_bytes_data_blocks^⚠

Total number of bytes consumed by data blocks.

mtbl_metadata_bytes_index_block^⚠

Total number of bytes consumed by the index.

mtbl_metadata_bytes_keys^⚠

Total number of bytes that all keys would occupy if stored end-to-end in a byte array with no delimiters.

mtbl_metadata_bytes_values^⚠

Total number of bytes that all values in the file would occupy if stored end-to-end in a byte array with no delimiters.

mtbl_metadata_compression_algorithm^⚠

One of the compression values allowed by mtbl_writer(3).

mtbl_metadata_count_data_blocks^⚠

Total number of data blocks.

mtbl_metadata_count_entries^⚠

Total number of key-value entries.

mtbl_metadata_data_block_size^⚠

Maximum size of an uncompressed data block, see mtbl_writer(3).

mtbl_metadata_index_block_offset^⚠

Byte offset in the MTBL file where the index begins.

mtbl_reader_destroy^⚠

mtbl_reader_init^⚠

mtbl_reader_init_fd^⚠

mtbl_reader_metadata^⚠

mtbl_reader_options_destroy^⚠

mtbl_reader_options_init^⚠

mtbl_reader_options_set_madvise_random^⚠

mtbl_reader_options_set_verify_checksums^⚠

mtbl_reader_source^⚠

mtbl_sorter_add^⚠

mtbl_sorter_destroy^⚠

mtbl_sorter_init^⚠

mtbl_sorter_iter^⚠

mtbl_sorter_options_destroy^⚠

mtbl_sorter_options_init^⚠

mtbl_sorter_options_set_max_memory^⚠

mtbl_sorter_options_set_merge_func^⚠

mtbl_sorter_options_set_temp_dir^⚠

mtbl_sorter_write^⚠

mtbl_source_get^⚠

mtbl_source_get_prefix^⚠

mtbl_source_get_range^⚠

mtbl_source_iter^⚠

mtbl_varint_decode32^⚠

mtbl_varint_decode64^⚠

mtbl_varint_encode32^⚠

mtbl_varint_encode64^⚠

mtbl_varint_length^⚠

mtbl_varint_length_packed^⚠

mtbl_writer_add^⚠

mtbl_writer_destroy^⚠

mtbl_writer_init^⚠

mtbl_writer_init_fd^⚠

mtbl_writer_options_destroy^⚠

mtbl_writer_options_init^⚠

mtbl_writer_options_set_block_restart_interval^⚠

mtbl_writer_options_set_block_size^⚠

mtbl_writer_options_set_compression^⚠