Skip to main content

Module stream

Module stream 

Source
Expand description

Streaming encode/verify/reconstruct for large data.

Provides encode_stream, verify_stream, and reconstruct_stream methods that process data in configurable blocks, avoiding the need to load entire files into memory.

§Example

use rustfs_erasure_codec::galois_8::ReedSolomon;
use rustfs_erasure_codec::stream::StreamOptions;
use std::fs::File;
use std::io::BufReader;

let rs = ReedSolomon::new(10, 4).unwrap();

let mut data_readers: Vec<BufReader<File>> = (0..10)
    .map(|i| BufReader::new(File::open(format!("data_{i}.bin")).unwrap()))
    .collect();
let mut parity_writers: Vec<File> = (0..4)
    .map(|i| File::create(format!("parity_{i}.bin")).unwrap())
    .collect();

let opts = StreamOptions::new().with_block_size(4 * 1024 * 1024);
rs.encode_stream(&mut data_readers, &mut parity_writers, &opts).unwrap();

Structs§

StreamError
Error returned by streaming operations.
StreamOptions
Configuration for streaming encode/verify/reconstruct operations.

Enums§

StreamErrorKind
Classification of StreamError.