dino_seq

Dino Seq is a small Rust parser crate for streaming FASTQ and FASTA inputs. It focuses on low-allocation record access for downstream tools rather than preprocessing workflows.

The default crate is the raw parser core. Compression is a transport layer: enable gzip, bgzf, or transport only when file-magic convenience matters.

Which API Should I Use?

Scope

Dino Seq provides:

• borrowed FASTQ batches over any Read
• one-pass FASTQ visitors for callers that do not need batch side tables
• FASTQ count/stat fast paths that do not build record views
• bounded FASTQ chunk sinks for callers that fill their own output buffers
• streaming multiline FASTA batches and visitors
• ordered paired-end validation for separate R1/R2 or adjacent interleaved reads
• optional BGZF, gzip, mmap, libdeflate, SIMD, and pack-path features

Dino Seq does not trim adapters, filter reads, align reads, produce QC reports, or synchronize reordered mates.

Hard contracts:

• FASTQ input is ordinary four-line FASTQ; multiline FASTQ is unsupported.
• Paired FASTQ validation is ordered only. Dino Seq validates separate R1/R2 streams or adjacent interleaved records, but it does not synchronize reordered mates.
• Ordinary multiline FASTA is supported, but APIs that return a contiguous seq() slice fold sequence lines into reusable parser storage. Use strict two-line APIs when the file shape allows avoiding that work.

FASTQ Entry Points

• FastqReader::next_batch: use when the caller needs RecordRef side tables.
• FastqReader::new(std::fs::File::open(path)?): raw-file hot path with a concrete reader type.
• FastqReader::count_records, count_fastq_read, and count_fastq_bytes: use for count/stat workloads that do not need record views.
• FastqReader::visit_records: use for whole-stream one-pass consumers.
• FastqReader::next_chunk_with_sink: use for aligners or pipelines that need bounded chunks in caller-owned structs.
• visit_fastq_bytes: use when the complete FASTQ input is already resident.
• open_fastq and open_fastq_with_config: convenience boxed file openers. They open raw files by default and detect gzip/BGZF when those features are enabled.

use dino_seq::FastqReader;

let data = b"@r1\nACGT\n+\nIIII\n";
let mut reader = FastqReader::new(&data[..]);

while let Some(batch) = reader.next_batch()? {
    for record in batch.records() {
        assert_eq!(record.seq(), b"ACGT");
    }
}
# Ok::<(), dino_seq::FastqError>(())

Use FastqReader::next_chunk_with_sink when downstream code wants to fill owned buffers without building a batch side table.

FASTA Entry Points

• FastaReader::next_batch: stream multiline FASTA batches.
• FastaReader::new(std::fs::File::open(path)?): raw-file hot path with a concrete reader type.
• FastaReader::visit_records: visit records without retaining a batch.
• visit_fasta_bytes: parse resident multiline FASTA bytes.
• visit_two_line_fasta_bytes and visit_two_line_fasta_read: strict fast paths for canonical >header / sequence FASTA.
• open_fasta and open_fasta_with_config: convenience boxed file openers. They open raw files by default and detect gzip/BGZF when those features are enabled.

use dino_seq::FastaReader;

let data = b">chr1\nAC\nGT\n";
let mut reader = FastaReader::new(&data[..]);
let batch = reader.next_batch()?.unwrap();
let record = batch.records().next().unwrap();

assert_eq!(record.name_without_gt(), b"chr1");
assert_eq!(record.seq(), b"ACGT");
# Ok::<(), dino_seq::FastqError>(())

Features

• default: raw FASTQ/FASTA parser core only
• gzip: ordinary gzip input by gzip magic
• bgzf: BGZF reader, writer, detection, indexing, and parallel block helpers
• transport: gzip + BGZF convenience transports
• libdeflate: optional libdeflate BGZF backends
• gzip-libdeflate: gzip + libdeflate for explicit buffered gzip openers
• mmap: resident file visitors and counters backed by read-only memory maps
• simd: stable std::arch acceleration where supported

Unsafe And Optional Features

The default parser core is dependency-light: stable Rust plus memchr, with no compression or memory-map dependency. Optional features add explicit systems surfaces:

• mmap maps files through the operating system for resident-file visitors and counters. It is useful for local files that should be scanned as byte slices, not for stdin, pipes, or compressed streams.
• simd enables stable std::arch kernels for pack paths where the target supports them.
• gzip, bgzf, transport, libdeflate, and gzip-libdeflate are transport or backend choices, not required parser semantics.

Unsafe code is allowed only for these low-level integration points, such as memory mapping and architecture intrinsics. The raw FASTQ/FASTA reader APIs do not require callers to use unsafe code.

Peer Fit

This is a workload map, not a speed ranking.

Adoption Docs

• API surface: intended public surface before 1.0.
• Benchmarking: in-tree smoke benchmark policy and external artifact boundary.
• Security policy: supported input-security scope and hard unsupported cases.
• Changelog: release-facing behavior and feature changes.

Publication benchmark artifacts live outside this crate so generated datasets, peer harnesses, and large result tables do not become package weight. Keep small explanatory figures in this package when they document the public API.

CLI

cargo run --release --bin dino-seq -- stats reads.fastq
cargo run --release --bin dino-seq -- stats --format fasta reference.fa
cargo run --release --bin dino-seq -- checksum --format fastq reads.fastq
cargo run --release --bin dino-seq -- checksum --format fasta reference.fa
cargo run --release --bin dino-seq -- fasta-index reference.fa
cargo run --release --bin dino-seq -- fasta-fetch reference.fa --fai reference.fa.fai --name chr1 --start 0 --end 100

Checks

cargo fmt --all --check
cargo test -p dino-seq --no-default-features
cargo test -p dino-seq --all-features
cargo clippy -p dino-seq --all-targets --all-features -- -D warnings
cargo bench --all-features
cargo package --allow-dirty --list