Struct CorpusFeatures 

pub struct CorpusFeatures {
    pub n_items: usize,
    pub n_categories: usize,
    pub dim: usize,
    pub mean_members_per_category: f64,
    pub category_size_entropy: f64,
    pub mean_sparsity: f64,
    pub axis_utilization_entropy: f64,
    pub noise_estimate: f64,
    pub mean_intra_category_similarity: f64,
    pub mean_inter_category_similarity: f64,
    pub category_separation_ratio: f64,
}

Low-dimensional profile of a corpus. Computed once per corpus; fed into any MetaModel to predict the pipeline config that’s likely to work best on it.

Fields

n_items: usize

Total item count.

n_categories: usize

Unique category count.

dim: usize

Embedding dimensionality.

mean_members_per_category: f64

n_items / n_categories.

category_size_entropy: f64

Shannon entropy of the category-size distribution, normalized to [0, 1] by dividing by log(n_categories). High = balanced category sizes; low = heavily skewed.

mean_sparsity: f64

Mean per-item active-axis fraction: |active axes| / dim, averaged over items. An axis is active when |v_i| > active_threshold.

axis_utilization_entropy: f64

Entropy of how often each axis is active across the corpus, normalized by log(dim). High = all axes used similarly; low = a few axes dominate.

noise_estimate: f64

Median of |v_i| across inactive entries (|v_i| ≤ threshold), averaged across items. A proxy for the noise floor.

mean_intra_category_similarity: f64

Mean intra-category cosine similarity in embedding space. High = items within a category are tightly clustered.

mean_inter_category_similarity: f64

Mean inter-category cosine similarity in embedding space. High = categories overlap heavily; low = categories are semantically distinct.

category_separation_ratio: f64

mean_intra / max(mean_inter, eps). A ratio-based separation signal: values > 1 mean categories separate well in embedding space; values near 1 mean the corpus is difficult to partition.

Implementations

impl CorpusFeatures

pub fn feature_names() -> [&'static str; 10]

Stable feature names aligned with Self::to_vec. Useful for logging, feature importance reports, and CSV headers.

Note: category_separation_ratio is deliberately excluded — it’s a derived ratio of two features already named here, so including it would double-count under any distance metric. See Self::to_vec.

pub fn to_vec(&self) -> [f64; 10]

Fixed-length flattened representation in the order declared by Self::feature_names. Suitable as input to any nearest-neighbor or regression meta-model. category_separation_ratio is intentionally excluded because it’s a derived ratio of two features already in the vector — keeping it in would double-count.

pub fn extract( categories: &[String], embeddings: &[Vec<f64>], ) -> Result<Self, String>

Extract features from a corpus using default Laplacian config (for the active_threshold used in sparsity/noise estimation).

Returns an error if the inputs are invalid (empty corpus, mismatched lengths, zero-dim embeddings, or ragged rows).

pub fn extract_with_threshold( categories: &[String], embeddings: &[Vec<f64>], active_threshold: f64, ) -> Result<Self, String>

Extract features with an explicit active-axis threshold. Use this when you want feature values comparable across different Laplacian configurations.

Returns an error if the inputs are invalid (empty corpus, mismatched lengths, zero-dim embeddings, or ragged rows).

Trait Implementations

impl Clone for CorpusFeatures

fn clone(&self) -> CorpusFeatures

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for CorpusFeatures

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for CorpusFeatures

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for CorpusFeatures

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.