Struct ExternalSorter 

pub struct ExternalSorter { /* private fields */ }

External sorter for minimizer tuples

Manages RAM-bounded sorting with temp file spillover and k-way merge.

Implementations

impl ExternalSorter

pub fn new( tmp_dir: impl AsRef<Path>, ram_limit_gib: usize, num_threads: usize, verbose: bool, ) -> Result<Self>

Create a new external sorter

pub fn buffer_size_per_thread(&self) -> usize

Calculate buffer size per thread in number of tuples

Formula: (ram_limit * GiB) / (2 * sizeof(tuple) * num_threads) The factor of 2 accounts for temporary memory during parallel sort.

pub fn sort_and_flush( &self, buffer: &mut Vec<MinimizerTupleExternal>, ) -> Result<u64>

Sort a buffer and flush to a temp file

Returns the file ID. Thread-safe via atomic counter.

pub fn num_files(&self) -> u64

Number of temp files created

pub fn merge(&self) -> Result<MergeResult>

Merge all temp files into final sorted output

Returns statistics: (num_minimizers, num_positions, num_super_kmers)

pub fn read_merged_tuples(&self) -> Result<Vec<MinimizerTuple>>

Read merged tuples into memory as internal MinimizerTuples

Call this after merge() to get the final sorted tuples. For large datasets, prefer [open_merged_file] to avoid full materialization.

pub fn open_merged_file(&self) -> Result<FileTuples>

Open the merged file for sequential buffered access.

Returns a FileTuples handle that provides sequential access via buffered readers. Each scan pass opens a fresh BufReader, keeping RSS proportional to the buffer size (~4 MB) instead of the file size.

Important: The returned FileTuples takes ownership of cleanup responsibility for the merged file.

pub fn remove_merged_file(&self) -> Result<()>

Remove the merged file (cleanup)

Trait Implementations

impl Drop for ExternalSorter

fn drop(&mut self)

Executes the destructor for this type.