Struct CsrCellChunk 

pub struct CsrCellChunk {
    pub data_raw: RawCounts,
    pub data_norm: Vec<F16>,
    pub library_size: usize,
    pub indices: Vec<u32>,
    pub original_index: usize,
    pub to_keep: bool,
}

CsrCellChunk

This structure is designed to store the data of a single cell in a CSR-like format optimised for rapid access on disk.

Raw counts are stored as a RawCounts enum which transparently handles both u16 and u32 storage. Gene indices are stored as u32 to support data sets with more than 65,535 features.

Fields

data_raw: RawCounts

Raw counts for this cell, stored as either u16 or u32 depending on the data range

data_norm: Vec<F16>

Vector of the norm counts of this cell. A lossy compression for f16 is applied.

library_size: usize

Total library size/UMI counts of the cell.

indices: Vec<u32>

Index positions of the genes (u32 to support >65k features)

original_index: usize

Original index in the data

to_keep: bool

Flag if the cell should be kept. (Not used at the moment.)

Implementations

impl CsrCellChunk

pub fn from_data<T, U>( data: &[T], col_idx: &[U], original_index: usize, size_factor: f32, to_keep: bool, ) -> Self

where T: FloatAndUInt, U: FloatAndUInt,

Function to generate the chunk from R data

Assumes columns = genes, rows = cells.

Params

• data - The raw counts present in this cell
• col_idx - The column indices where the gene is expressed.
• original_index - Original row index in the matrix
• size_factor - To which size to normalise to. 1e6 -> CPM normalisation.
• to_keep - Whether the cell passes QC

Returns

The CsrCellChunk for this cell.

pub fn write_to_bytes(&self, writer: &mut impl Write) -> Result<()>

Write directly to bytes on disk

Binary layout (header = 32 bytes):

[0..4)   data_raw_len    u32
[4..8)   data_norm_len   u32
[8..12)  indices_len     u32
[12..20) library_size    u64
[20..28) original_index  u64
[28]     to_keep         u8
[29]     raw_elem_size   u8  (2 = u16, 4 = u32)
[30..32) padding         2 bytes

Followed by:

• data_raw: data_raw_len * raw_elem_size bytes
• data_norm: data_norm_len * 2 bytes
• indices: indices_len * 4 bytes (u32)

pub fn to_compressed_bytes(&self) -> Result<Vec<u8>>

Serialise and lz4-compress the chunk into the on-disk payload.

Pure with respect to the writer, so callers can build many payloads in parallel and hand them to CellGeneSparseWriter::write_compressed_cell_chunks in order.

Returns

The compressed payload, size-prepended as the reader expects.

pub fn read_from_buffer(buffer: &[u8]) -> Result<Self, ScDataErrors>

Read data from buffer

Params

• buffer - A slice of u8’s representing the buffer

Return

The CsrCellChunk

pub fn generate_chunks_sparse_data<T, U>( sparse_data: CompressedSparseData2<T, U>, cell_qc: MinCellQuality, ) -> (Vec<CsrCellChunk>, CellQuality)

where T: Clone + Default + Into<u32> + Sync, U: Clone + Default,

Generate a vector of Chunks from CompressedSparseData2

Params

• sparse_data - The CompressedSparseData2 (in CSR format!)
• cell_qc - Minimum QC thresholds

Returns

A tuple of the Vec<CsrCellChunk> and if the cell should be kept.

pub fn get_qc_info(&self) -> (usize, usize)

Helper function to get QC parameters for this cell

Returns

A tuple of (no_genes, library_size)

pub fn filter_genes(&mut self, genes_to_keep: &[bool])

Filter to keep only specified genes

This will remove the genes that are not set to true in the boolean vector. This assumes that the indices of the genes and the booleans are the same!

Params

• genes_to_keep - Keep only the genes within this boolean vectors.

impl CsrCellChunk

pub fn add_cells_scrublet( cell1: &CsrCellChunk, cell2: &CsrCellChunk, hvg_indices: &FxHashSet<usize>, hvg_lib_size_combined: usize, target_size: f32, log_transform: bool, doublet_index: usize, ) -> Self

Combine two cells into a synthetic doublet for doublet simulation.

Filters both cells to HVG genes, sums their raw counts, normalises to target_size using the combined HVG library size, and optionally log-transforms.

Params

• cell1 - First parent cell.
• cell2 - Second parent cell.
• hvg_indices - Set of HVG gene indices to retain.
• hvg_lib_size_combined - Sum of parent HVG library sizes.
• target_size - Normalisation target (e.g. mean HVG library size).
• log_transform - Whether to apply ln(1 + x) after normalisation.
• doublet_index - Index to assign to the new chunk.

Returns

A new CsrCellChunk representing the synthetic doublet.

impl CsrCellChunk

pub fn from_doublet_simulation( indices: Vec<u32>, raw_counts: Vec<u32>, norm_values: Vec<f32>, effective_lib_size: usize, doublet_index: usize, ) -> Self

Construct a chunk from pre-computed doublet simulation data.

Unlike from_data, this does not recompute normalisation! The caller has already applied the correct normalisation (with noise-injected raw counts and the effective library size).

Params

• indices - Selected genes-remapped gene positions (sparse, sorted).
• raw_counts - Raw counts at each position (post-noise).
• norm_values - Pre-computed normalised values (same length as indices).
• effective_lib_size - Sum of raw counts after all noise treatments.
• doublet_index - Index of this doublet in the simulation batch (used as original_index).

Returns

A CsrCellChunk ready for downstream PCA and feature engineering.

impl CsrCellChunk

pub fn to_v4_bytes( &self, target_size: f32, codec: ChunkCodec, ) -> Result<Vec<u8>, ScDataErrors>

Serialise the cell into a v4 chunk.

The norm layer is dropped when every entry recomputes bit for bit from the raw count, the library size and target_size.

Params

• target_size - Target size of the file the cell goes into, 0.0 if unknown, which stores the norm
• codec - Body codec

Returns

The chunk bytes, or ScDataErrors::V4ChunkCorrupt for layers of unequal length.

Trait Implementations

impl Debug for CsrCellChunk

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.