Enum ContentType 

pub enum ContentType {
    Code,
    Prose,
    Mixed,
}

Classification of text content for adaptive token estimation

Variants

Code

Code-heavy content (>15% code indicators) Examples: source files, JSON, config files Token ratio: ~3.0 chars/token

Prose

Prose-heavy content (<5% code indicators) Examples: documentation, natural language, articles Token ratio: ~4.5 chars/token

Mixed

Mixed content (5-15% code indicators) Examples: markdown with code blocks, technical docs Token ratio: ~3.75 chars/token

Implementations

impl ContentType

pub fn chars_per_token(&self) -> f64

Get the estimated characters per token for this content type

Trait Implementations

impl Clone for ContentType

fn clone(&self) -> ContentType

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ContentType

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for ContentType

fn eq(&self, other: &ContentType) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for ContentType

impl Eq for ContentType

impl StructuralPartialEq for ContentType