Struct PayloadSlice

Source

pub struct PayloadSlice { /* private fields */ }

Expand description

Payload slice is a slice of variable-sized elements (payloads) encoded in a single block of memory. This way, sequences of, say, strings, can be indexed into without loading all the elements in memory, but rather using a memory mapped buffer.

§Use case

The primary use case of this struct is not necessarily to limit the bytes that are loaded in memory, but rather to limit the time to initialize it. For example, one can use it in a command line program to quickly look up one or several values from a file-encoded vector, without the overhead of parsing the entire file and loading it in memory.

§Examples

use memmap::Mmap;

// We will store out vector to a temporary directory
let temp = TempDir::new()?;
let file_path = temp.path().join("words.bin");

// We can use any elements that implement `AsRef<[u8]>`.
let words = vec!["dog", "cat", "gnu"];

// One way of encoding is to collect elements to a PayloadVector in memory
let payloads: PayloadVector = words.into_iter().collect();

// Write to file
let mut output = File::create(&file_path)?;
payloads.write(&mut output);
drop(output);

// Load as payload slice
let input = File::open(&file_path)?;
let bytes = unsafe { Mmap::map(&input)? };
let payloads = PayloadSlice::new(&bytes);

// Note that it returns byte slices.
assert_eq!(&payloads[0], b"dog");
assert_eq!(&payloads[1], b"cat");
assert_eq!(&payloads[2], b"gnu");

// Non-panicing access.
assert_eq!(payloads.get(3), None);

// Collect to a vector of strings
let items: Vec<_> = payloads
    .iter()
    .map(|b| String::from_utf8(b.to_vec()).unwrap())
    .collect();
assert_eq!(items, vec![
    "dog".to_string(),
    "cat".to_string(),
    "gnu".to_string()
]);