Struct DcPoissonStats 

pub struct DcPoissonStats {
    pub k: usize,
    pub num_features: usize,
    pub membership: Vec<usize>,
    pub gene_sum: Vec<f64>,
    pub size_sum: Vec<f64>,
    pub log_gene: Vec<f32>,
    pub log_size_offset: Vec<f32>,
    pub count: Vec<usize>,
    pub min_gain_z: f64,
    /* private fields */
}

Sufficient statistics with cached log quantities for O(K · nnz(row)) scoring.

gene_sum[g·K + k] = Σ_{e : z_e = k} y_eg; size_sum[k] = Σ_{e : z_e = k} s_e. The per-feature tables are laid out feature-major (g·K + k): scoring one entity walks its nonzero features once and, for each, reads the K group values side by side, instead of striding across a K × M table once per candidate group. Use DcPoissonStats::idx to address them. Log caches score every non-current block with no log; the current block’s leave-one-out score takes one fast_ln per nonzero feature, and delta_move refreshes the logs of the columns a move touches.

The score

An entity is scored against block k by the log-likelihood ratio of the block’s profile to the pooled profile π, Σ_g y_eg · ln(θ_kg / π_g), with each log-ratio estimated by

u_kg = ln(G_kg + c) − E₀ ln(G + c), G ~ Poisson(S_k · π_g).

c = prior_count is a pseudo-count in every block total. The second term is the mean of the first for a block that has the pooled profile (NullLog), so u_kg is zero on average for such a block whatever its mass, and blocks of the pooled profile score alike whatever their size. A plain ln((G + a) / (S + α)) does not have that property: its mean falls with the block’s mass (the log of a noisy total is biased low), and on a feature the block has not seen it reads a rate proportional to 1 / S_k. With some 20,000 heavy-tailed features most totals of a small block are zero or one, and either effect outweighs the evidence.

ln((G + c) / (S_k π_g + c)) is the posterior-mean rate ratio under a Gamma(c, c) prior on θ_kg / π_g: a feature expected less than c times in the block stays near the pooled rate, one expected many times reads G / (S_k π_g). Blocks that share a profile other than the pooled one are therefore shrunk toward the pooled profile by different amounts when their masses differ, and an entity of that profile fits the larger one better. A smaller c shrinks less and is noisier.

π_g is taken without the entity scored ((P_g − y_eg) / (T − s_e), P the pooled totals): a rare feature’s pooled rate is otherwise made of the entity’s own counts. A feature nothing else shows carries no evidence for any block. The block scored is not taken out of π: a block that makes up most of the pooled mass is in part its own null, and is favoured over a small block of its profile.

The null mean depends on (k, g) through S_k π_g only, so it is kept per block on a grid of ln π (RateClasses), and an entity carries its counts summed onto the same grid: the score costs one cached log per (nonzero, block) and one short dot product per block.

What remains under the null is zero-mean noise of variance Σ_g y_eg² · Var₀ ln(G_kg + c), read off the same grid. A destination is charged min_gain_z standard deviations of the difference between its score and the current block’s, z · sqrt(Σ_g y_eg² (Var₀[cur, g] + Var₀[k, g])): a move has to beat what the block estimates alone would produce between two blocks of one profile.

The moments are those of Poisson totals. Totals of profiles weighted by FeatureWeighting::FisherInfoNb are not counts, but under the negative binomial model the weights come from (w = 1 / (1 + μ φ), the ratio of a count’s mean to its variance) a weighted total again has a variance equal to its mean.

Log caches are f32 — values live in roughly [-21, +25] and scoring is noise-dominated, so f32’s ~7-digit precision is ample. Halves the largest allocation versus f64.

Fields

k: usize

num_features: usize

membership: Vec<usize>

gene_sum: Vec<f64>

Feature-major M × K: entry g·K + k.

size_sum: Vec<f64>

log_gene: Vec<f32>

Feature-major M × K: entry g·K + k. Holds ln(G_kg + c).

log_size_offset: Vec<f32>

−ln(S_k + M · LOG_EPS). Not part of the score, which compares a block with the pooled profile feature by feature.

count: Vec<usize>

Members per block.

min_gain_z: f64

Standard deviations of estimate noise a move has to clear.

Implementations

impl DcPoissonStats

pub fn idx(&self, k: usize, g: usize) -> usize

Index of (group k, feature g) in gene_sum / log_gene.

pub fn from_profiles( profiles: &Profiles, k: usize, membership: &[usize], ) -> Self

Statistics under the default RefineParams scoring.

pub fn from_profiles_with( profiles: &Profiles, k: usize, membership: &[usize], params: &RefineParams, ) -> Self

Statistics scored with params.prior_count and params.min_gain_z.

pub fn delta_move( &mut self, e: usize, k_from: usize, k_to: usize, profiles: &Profiles, )

Apply an entity’s reassignment and refresh only the affected log entries.

Trait Implementations

impl Clone for DcPoissonStats

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.