qbix

qbix indexes and retrieves BAM records by QNAME (read name) using a .qbi index.

Use it when you need records for one or more QNAMEs from a BAM file without scanning the file.

The index stores:

• XXH3-64 hashes of QNAMEs
• BGZF virtual offsets
• BAM size, mtime, and header hash for stale-index detection

The .qbi file-format versions are documented in docs/qbi-format.md: v1 uses magic QBI1, while the default v2 uses magic QBI2.

Draft paper (PDF): qbix: Read-name lookup for coordinate-sorted BAM files

qbix was inspired by jts/bri. .qbi is not compatible with .bri. .qbi stores QNAME hashes and BGZF virtual offsets instead of names. Lookup candidates are checked against BAM QNAME before output.

Install

Download a prebuilt binary from GitHub Releases.

Build From Source

Requirements:

• Rust and Cargo
• htslib
• pkg-config recommended

Install htslib and pkg-config with your system package manager:

# macOS
brew install htslib pkg-config

# Ubuntu/Debian
sudo apt-get install libhts-dev pkg-config

cargo build --release

The binary is:

target/release/qbix

If htslib is installed under a custom prefix:

HTSDIR=/path/to/htslib cargo build --release

For static htslib linking:

HTSLIB_STATIC=1 cargo build --release

Optional colored SAM output uses libbiosyntax, which is GPL-3.0-only. It is disabled by default. Enable it explicitly if that license is acceptable:

cargo build --release --features biosyntax

Prebuilt binaries from GitHub Releases are built without this feature and do not include libbiosyntax.

With this feature enabled, the build downloads libbiosyntax v0.1.0 into Cargo's build directory. To use an existing checkout instead:

LIBBIOSYNTAX_DIR=/path/to/libbiosyntax cargo build --release --features biosyntax

You can also build and install from crates.io:

cargo install qbix

The crates.io install builds from source and expects htslib headers and libraries to be available on the system.

Basic Use

Create an index:

qbix index reads.bam

QBI file-format v2 (QBI2) is the default. The original v1 format (QBI1) remains available explicitly; get, show, check, and stats detect either version:

qbix index --index-format qbi1 reads.bam

Use --qbi2-radix-bits 8, 12, or 16 to select a QBI2 layout explicitly. P=12 is the speed/space intermediate choice. When omitted, the conservative automatic policy selects P=8 for at most 522,240 records and P=16 otherwise. An explicit value always takes precedence.

This writes:

reads.bam.qbi

If the QBI output path is an existing symlink, qbix index preserves the symlink and replaces its target. A dangling output symlink is rejected.

Fetch records by QNAME. Output is SAM:

qbix get reads.bam read_a read_b

SAM output contains matching alignment records without the SAM header by default. Use --with-header to include a header based on the source BAM. BAM output always includes it. When a header is written, qbix get adds an @PG record with its version and command line. Query-order output sets @HD SO:unknown and removes SS and GO, since query order may differ from the input sort order. --bam-order preserves the input @HD sort metadata.

qbix get --with-header reads.bam read_a read_b

Fetch records from a newline-delimited read-name file:

qbix get reads.bam -f names.txt

Use -f - to read names from stdin:

cat names.txt | qbix get reads.bam -f -

Process each input read name only once while preserving the order of its first occurrence:

qbix get --unique reads.bam read_a read_a read_b
qbix get --unique reads.bam -f names.txt

This only removes duplicate query names. If the BAM contains multiple records for a read name, all of those records are still written.

Write query names that have no matching BAM record to a newline-delimited file:

qbix get --missing missing.txt reads.bam -f names.txt

Missing names are written in query order. Duplicate queries are reported once per occurrence unless --unique is also used. The report file is empty when all query names are found.

Write matching records as BAM:

qbix get reads.bam -f names.txt -b -o hits.bam
qbix get reads.bam -f names.txt -Ob -o hits.bam

BAM output defaults to BAM file order. When the input header declares SO:coordinate and the BAM is written to a file, qbix builds a .bai index alongside it (or .csi when a reference is longer than 512 Mbp). Use --no-index to skip the index, or --query-order to request query-order BAM output; query-order output is not indexed. BAM output to stdout has no sidecar index.

When replacing a BAM, qbix moves any old .bai/.csi aside before publishing the new BAM. If interrupted before publishing the new index, the BAM may be left without a coordinate index, but not with the old one beside the requested output path.

If the output BAM is a symlink, qbix keeps the link, writes the new index next to the requested link, and removes stale indexes beside every link in that symlink chain and the resolved BAM target. Independent aliases that were not used as the output path cannot be discovered automatically.

When -o names a file, qbix writes to a temporary file beside it and replaces the destination only after successful output. A read or write error leaves an existing output file untouched. The same applies to --missing reports; when both paths are used, they are published separately. Standard output (-o -) remains streaming.

When built with --features biosyntax, SAM output to a terminal is colored by default. Pipes and files are left plain. Use --color always or --color never to override:

qbix get --color always reads.bam read_a
qbix get --color never reads.bam read_a

Use more BGZF/htslib I/O threads:

qbix index --bgzf-threads 4 reads.bam
qbix get -@ 4 reads.bam read_a

Use an explicit index path:

qbix index -i reads.qbi reads.bam
qbix get -i reads.qbi reads.bam read_a

Output Order

SAM output defaults to query order:

qbix get reads.bam read_a read_b
qbix get --query-order reads.bam read_a read_b

Query-order output streams lookups without collecting all hits in memory. When read names come from -f -, each name is looked up as it is read from stdin; qbix does not wait for EOF or collect all input names first. --unique retains only the set of names already seen.

For multiple read names, --bam-order reads records in BAM file-offset order. This can reduce random seeking:

qbix get --bam-order reads.bam read_a read_b

This is also the default for BAM output. --bam-order must collect the input names and sort matching hits before output, including when names come from stdin.

If name-sorted output is needed, sort downstream:

qbix get --bam-order reads.bam read_a read_b | samtools sort -N -O SAM -

Other Commands

Quickly check that an index matches its BAM size, mtime, and header hash:

qbix check reads.bam
qbix check --quick reads.bam

check defaults to --quick. Use --full to validate complete QBI2 directory and suffix ordering, seek to every indexed record, and verify its read-name hash:

qbix check --full reads.bam

Print records-per-hash statistics and QBI1/QBI2 size estimates from the index. stat is accepted as an alias for stats:

qbix stats reads.bam
qbix stat reads.bam
qbix stats -i reads.qbi reads.bam

Use JSON output for scripts:

qbix stats --json reads.bam

Show raw index rows:

qbix show reads.bam.qbi

show prints:

qhash<TAB>voff

Print the version:

qbix --version

Notes

• .qbi files are tied to the BAM size, mtime, and header hash.
• Rebuild the index after replacing or rewriting the BAM.
• Read names are not stored in the index. Hash hits are verified against the BAM record QNAME before output.

Rust Library

qbix also exposes a small Rust API:

let index_path = qbix::build_index("reads.bam", qbix::BuildOptions::default())?;
qbix::check_index("reads.bam", qbix::CheckOptions::default())?;
qbix::check_index(
    "reads.bam",
    qbix::CheckOptions {
        mode: qbix::CheckMode::Full,
        ..qbix::CheckOptions::default()
    },
)?;
let mut bam = qbix::IndexedBam::open("reads.bam", qbix::LookupOptions::default())?;
let hits = bam.lookup("read_a")?;

C Library

qbix can be built as a C library from source:

cargo build --release

Libraries are written under target/release, for example libqbix.so, libqbix.a, or libqbix.dylib. The header is include/qbix.h.

For HTSlib-based applications, build qbix against the same HTSlib installation as the host application. C API errors are stored per calling thread. qbix_index_t handles are not thread-safe.

Example:

#include "qbix.h"

int main(void) {
    qbix_hit_t *hits = 0;
    size_t n_hits = 0;

    if (qbix_build_index("reads.bam", 0, 1) != 0) return 1;
    if (qbix_check_index("reads.bam", 0, 1, QBIX_CHECK_QUICK) != 0) return 1;

    qbix_index_t *idx = qbix_index_open("reads.bam", 0, 1);
    if (!idx) return 1;

    if (qbix_index_lookup(idx, "read_a", &hits, &n_hits) == 0) {
        qbix_hits_free(hits, n_hits);
    }

    qbix_index_close(idx);
    return 0;
}