Crate libradicl

libradicl is a crate for reading (and writing) RAD (Reduced Alignment Data) format files. The RAD format is a binary format designed to encode alignment information about sequencing reads and how they map to a set of targets (a genome, metagenome, transcriptome, etc.). The format is “reduced” because it is allowed to contain sparser information than e.g. a SAM format file.

While the eventual goal of this crate is to provide a generic API to read and write RAD files that may be designed for any purpose, it is driven mostly by our (the COMBINE-lab’s) needs within the tools we produce that use the RAD format (e.g. alevin-fry and piscem-infer). Thus, features are generally developed and added in the order that is most urgent to the development of these tools. However, we welcome external contributions via pull requests, and are happy to discuss your potential use cases for the RAD format, and how they might be supported.

This crate is broken into several components that cover the various parts of RAD files including the type tag system, the header, and the main data chunks. The names of each module are fairly self-explanatory.

Modules

chunk

Types and functions that primarily deal with the reading and writing of data Chunks in the RAD file.

constants

Constants relevant for the RAD format

exit_codes

This module contains custom exit codes that designate specific error conditions.

header

This module contains types, functions and traits to deal with RAD file headers, and also top-level functionality to encapsulate a RAD prelude (which consists of the header, and the initial TagSections; basically everything up to the first chunk).

io

Free functions to help with reading and writing specific libradicl types into and out of primitive types.

rad_types

This module contains the relevant structures and traits for most of the core RAD types. This includes the integer, numeric and composite types, as well as other relevant types built from them (e.g. TagSections). It also contains the types and traits related to parsing and writing values of specific types.

readers

This module contains types and traits that provide a high-level iterface to reading and parsing RAD files. Additionally, it provides types that give an interface for parsing RAD chunks in parallel for improved processing performance.

record

This module contains types and traits related to RAD records, including the traits for MappedRecords and RecordContexts. It also defines concrete types implementing these traits for alevin-fry and piscem-infer.

schema

This module contains basic type-related information that doesn’t fit cleanly into the other, more focused modules.

utils

This module contains some utility constants and functions that are helpful in processing RAD information.

Macros

as_u64

Convert from an underlying newtype (e.g. a crate::libradicl::io::NewU8, crate::libradicl::io::NewU16, crate::libradicl::io::NewU32, crate::libradicl::io::NewU64, crate::libradicl::io::NewU128) into a native u64. Note that conversion from a crate::libradicl::io::NewU128 will panic! as the underlying native type is too narrow to hold the contents of the integer.

as_u128

Convert from an underlying newtype (e.g. a crate::libradicl::io::NewU8, crate::libradicl::io::NewU16, crate::libradicl::io::NewU32, crate::libradicl::io::NewU64, crate::libradicl::io::NewU128) into a native u128.

tag_value_try_into_int

try_as_u64

Try to convert from an underlying newtype (e.g. a crate::libradicl::io::NewU8, crate::libradicl::io::NewU16, crate::libradicl::io::NewU32, crate::libradicl::io::NewU64, crate::libradicl::io::NewU128) into a native u64. If the conversion is successful, we produce an Ok(u64), otherwise we produce an std::result::Result::Err.

try_as_u128

Try to convert from an underlying newtype (e.g. a crate::libradicl::io::NewU8, crate::libradicl::io::NewU16, crate::libradicl::io::NewU32, crate::libradicl::io::NewU64, crate::libradicl::io::NewU128) into a native u128. If the conversion is successful, we produce an Ok(u128), otherwise we produce an std::result::Result::Err.

u8_to_vec_of

u8_to_vec_of_bool

write_tag_value_array

Structs

BarcodeLookupMap

GlobalEqCellList

TempBucket

Represents a temporary bucket of barcodes whose records will be written together and then collated later in memory.

Functions

as_u8_slice

as_u8_slice_u8

as_u8_slice_u16

collate_temporary_bucket_twopass

Given a BufReader<T> from which to read a set of records that should reside in the same collated bucket, this function will collate the records by cell barcode, filling them into a chunk of memory exactly as they will reside on disk. If compress is true the collated chunk will be compressed, and then the result will be written to the output guarded by owriter.

collate_temporary_bucket_twopass_atac

dump_corrected_cb_chunk_to_temp_file

Read an input chunk from reader and write the resulting records to the corresponding in-memory buffers local_buffers. As soon as any buffer reaches flush_limit, flush the buffer by writing it to the output_cache.

dump_corrected_cb_chunk_to_temp_file_atac

Read an input chunk from reader and write the resulting records to the corresponding in-memory buffers local_buffers. As soon as any buffer reaches flush_limit, flush the buffer by writing it to the output_cache.

lib_name