serde_pipe 0.1.1

Turn serde+bincode into a pipe: push `T`s and pull `u8`s, or vice versa. This library gives you a `Serializer` pipe, into which you can push `T`s and pull `u8`s; and a `Deserializer` pipe, into which you can push `u8`s and pull `T`s.
Documentation

serde_pipe

Crates.io MIT / Apache 2.0 licensed Build Status

Docs

Turn serde+bincode into a pipe: push Ts and pull u8s, or vice versa.

This library gives you a Serializer pipe, into which you can push Ts and pull u8s; and a Deserializer pipe, into which you can push u8s and pull Ts.

Both are bounded in their memory usage, i.e. they do not simply allocate a vector for serde+bincode to serialize into. Instead, libfringe is leveraged to turn serde+bincode into a Generator from which u8s can be pulled from/pushed to on demand.

This is useful for example if you have 10GiB memory available, and want to serialize+send or receive+deserialize an 8GiB vector. Note, this is perfectly possible with serde+bincode normally, but you'd need to dedicate a thread that blocks until completion to the task.

Example

extern crate serde_pipe;
use serde_pipe::Serializer;

let large_vector = (0..1u64<<30).collect::<Vec<_>>();
let mut serializer = Serializer::new();
serializer.push().unwrap()(large_vector);

while let Some(pull) = serializer.pull() {
	let byte = pull();
	println!("byte! {}", byte);
}

Note

This crate currently depends on libfringe, and as such inherits these limitations:

  • Rust nightly is required for the asm and naked_functions features;
  • The architectures currently supported are: x86, x86_64, aarch64, or1k;
  • The platforms currently supported are: bare metal, Linux (any libc), FreeBSD, DragonFly BSD, macOS. Windows is not supported.

License

Licensed under either of

at your option.

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.