Module seq_io::policy

This module defines the BufPolicy trait, which configures how the internal buffer of the parsers should grow upon encountering large sequences that don't fit into the buffer.

The standard policy (DoubleUntil) causes the initial buffer to double its size until it reaches 32 MiB, then further grow linearly until it reaches a size of 1 GiB. Larger sequences will cause an error of ErrorKind::BufferLimit in readers.

These numbers can be changed by using DoubleUntilLimited, or the limit can be completely removed by using DoubleUntil.

Example

The following code applies a policy that lets the buffer double until it reaches 64 MiB, and then further grow linearly without any limit:

use seq_io::fasta::Reader;
use seq_io::policy::DoubleUntil;

let policy = DoubleUntil(64 * 1024 * 1024);
let mut reader = Reader::from_path("input.fasta").unwrap()
    .set_policy(policy);
// (...)

Custom policy

This example shows how to implement a custom buffer policy:

use seq_io::prelude::*;
use seq_io::policy::BufPolicy;
use seq_io::fasta::Reader;
use std::io::stdin;

struct Max1G;

// This policy lets the buffer double each time, but
// limits the buffer size to 1 GiB. Note that this is similar to how
// `DoubleUntilLimited` works.
impl BufPolicy for Max1G {
    fn grow(&mut self, current_size: usize) -> usize {
        current_size * 2
    }

    fn limit(&self) -> Option<usize> {
        Some(1 << 30)
    }
}

let mut reader = Reader::new(stdin()).set_policy(Max1G);

while let Some(record) = reader.next() {
    println!("{}", record.unwrap().id().unwrap());
}

Structs

DoubleUntil	Buffer size doubles until it reaches a given limit (in bytes). Above, it will increase linearly in steps of 'limit'. Buffer size is not limited, it could theoretically grow indefinitely.
DoubleUntilLimited	Buffer size doubles until it reaches a given limit (in bytes). Above, it will increase linearly in steps of 'double_until'. Buffer size is additionally limited to limit bytes. Readers will return an error if this limit is .
StdPolicy	Standard buffer policy: This policy corresponds to DoubleUntilLimited(1 << 25, 1 << 30), meaning that buffer size doubles until it reaches 32 MiB, then increases in steps of 32 MiB until it reaches 1 GiB, which is the limit.

Traits

BufPolicy

Policy that configures how the internal buffer grows upon encountering large sequences that don't fit into the current buffer.