Struct TextIntegrityMetrics 

pub struct TextIntegrityMetrics {
    pub sentence_integrity: f32,
    pub word_integrity: f32,
    pub chunk_quality: f32,
    pub overall: f32,
    pub chunk_count: usize,
    pub avg_chunk_length: usize,
}

Text integrity metrics for embedding quality assessment.

Target: >90% overall integrity score before indexing.

Metrics

• Sentence Integrity: % of complete sentences preserved (not cut mid-sentence)
• Word Integrity: % of complete words (not truncated mid-word)
• Chunk Quality: How close chunks are to optimal size

Example

let metrics = TextIntegrityMetrics::compute(original_text, &chunks);
if !metrics.passes_threshold() {
    warn!("Text integrity below 90%: {:.1}%", metrics.overall * 100.0);
}

Fields

sentence_integrity: f32

Percentage of complete sentences preserved (0.0 - 1.0)

word_integrity: f32

Percentage of complete words (not truncated) (0.0 - 1.0)

chunk_quality: f32

Quality score based on chunk length distribution (0.0 - 1.0)

overall: f32

Combined overall score (0.0 - 1.0)

chunk_count: usize

Number of chunks analyzed

avg_chunk_length: usize

Average chunk length in characters

Implementations

impl TextIntegrityMetrics

pub const THRESHOLD: f32 = 0.90

Minimum acceptable overall score (90%)

pub const OPTIMAL_MIN: usize = 200

Optimal chunk length range (characters)

pub const OPTIMAL_MAX: usize = 800

pub fn compute(original: &str, chunks: &[String]) -> Self

Compute integrity metrics from original text and resulting chunks

pub fn passes_threshold(&self) -> bool

Check if metrics pass the minimum threshold (90%)

pub fn recommendation(&self) -> IntegrityRecommendation

Get recommendation based on metrics

Trait Implementations

impl Clone for TextIntegrityMetrics

fn clone(&self) -> TextIntegrityMetrics

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TextIntegrityMetrics

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for TextIntegrityMetrics

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Display for TextIntegrityMetrics

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Serialize for TextIntegrityMetrics

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.