pub fn stream_column_selection(
data: &dyn SparseIo<IndexIter = Vec<usize>>,
selected_columns: &[usize],
row_filter: Option<&[usize]>,
out_row_names: &[Box<str>],
out_col_names: &[Box<str>],
file_out: &str,
backend_out: &SparseIoBackend,
) -> Result<(usize, usize, usize)>Expand description
Stream a column selection (with an optional ASCENDING row filter) into a fresh backend, without ever materialising the survivors.
The shape split and subsample share: pick columns, optionally keep an
ascending subset of rows, write a new file. Both used to read every
surviving triplet into one Vec and hand it to the sorting triplet writer —
24 B/nnz of residency that OOMs at imaging scale. Here the exact output nnz
comes from the resident indptr (or one counting pass when rows are
filtered), and survivors flow through the validated streaming writer in
byte-budgeted slabs.
row_filter, when given, must be ascending: the renumbering is then
monotone, so within-column row order survives and no per-column sort is
needed. Both callers sample or select ascending; the debug assert keeps the
next caller honest rather than silently writing unsorted columns (which the
streaming writer would refuse anyway).