Enum TokenizationBlock 

pub enum TokenizationBlock {
    ServerUsage {
        counts_special_tokens: bool,
        counts_prompt_echo: bool,
    },
    ClientTokenizer {
        tokenizer_sha256: String,
        counts_special_tokens: bool,
        counts_prompt_echo: bool,
    },
}

§4.4.6 — the tokenization block, required in every receipt.

method has no default (PP-11). The variants below make “client_tokenizer with no digest” unrepresentable rather than merely rejected, and counts_special_tokens / counts_prompt_echo are plain non-optional booleans the caller must state.

counts_* under server_usage is an operator declaration about the server’s counting semantics, not a client measurement — §4.4.6 is titled “Token counting must be declared”. It is required rather than defaulted precisely so it cannot be silently wrong.

Variants

ServerUsage

Server-reported counts.

Fields

counts_special_tokens: bool

Whether the count includes special tokens.

counts_prompt_echo: bool

Whether the count includes the echoed prompt.

ClientTokenizer

Client-side counts with the model’s own tokenizer.

Fields

tokenizer_sha256: String

Digest of the tokenizer actually used. 64 lowercase hex characters.

counts_special_tokens: bool

Whether the count includes special tokens.

counts_prompt_echo: bool

Whether the count includes the echoed prompt.

Implementations

impl TokenizationBlock

pub fn method(&self) -> TokenCountingMethod

The declared counting method.

pub fn validate(&self) -> Result<(), String>

Errors

When a client_tokenizer digest is not 64 lowercase hex characters.

pub fn require_counter(&self, has_client_counter: bool) -> Result<(), String>

Poka-yoke for transports: a declared method the transport cannot honour is refused at construction, not silently downgraded at measure time.

Errors

When the declared method and the available counting machinery disagree.

Trait Implementations

impl Clone for TokenizationBlock

fn clone(&self) -> TokenizationBlock

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenizationBlock

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for TokenizationBlock

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Eq for TokenizationBlock

impl PartialEq for TokenizationBlock

fn eq(&self, other: &TokenizationBlock) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl Serialize for TokenizationBlock

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for TokenizationBlock