Skip to main content

streaming_sparse_running_stats_folded

Function streaming_sparse_running_stats_folded 

Source
pub fn streaming_sparse_running_stats_folded(
    data_vec: &SparseIoVec,
    block_size: Option<usize>,
    progress_label: &str,
    row_to_gene: &[u32],
    n_folded: usize,
) -> Result<(SparseRunningStatistics<f32>, SparseRunningStatistics<f32>)>
Expand description

The same pass, accumulating a second set of statistics on a folded axis.

Returns (row_stats, folded_stats). row_to_gene[r] names the folded bucket row r belongs to; buckets are 0..n_folded.

The fold happens inside the block read, so it costs O(nnz) arithmetic and no extra I/O. It exists because two of the four statistics do NOT survive a fold applied afterwards:

  • s2 of a sum is not the sum of s2. The cross term 2ab is gone, so a variance rebuilt from per-row squares understates a bucket whose rows covary, which two splice tracks of one gene certainly do.
  • npos of a sum is not the sum of npos. A cell detected on both rows of a bucket is one detection, not two, and the difference is exactly the bimodality that breaks a detection-based cutoff on a channelized matrix.

s1 and the mean do survive, which is why a caller that needs only those can fold a row-axis result afterwards instead of asking for this.