Struct CacheLayerResidencyStats 

pub struct CacheLayerResidencyStats {

    pub logical_cached_tokens: u64,
    pub key_value_blocks: u64,
    pub compressed_latent_blocks: u64,
    pub device_blocks: u64,
    pub host_blocks: u64,
    pub disk_blocks: u64,
    pub current_device_bytes: u64,
    pub current_host_bytes: u64,
    pub current_disk_bytes: u64,
    pub mutable_tail_bytes: u64,
    pub in_flight_write_blocks: u64,
    pub in_flight_write_bytes: u64,
    pub in_flight_host_demotion_blocks: u64,
    pub in_flight_host_demotion_bytes: u64,
    pub protected_recent_blocks: u64,
    pub protected_prefix_blocks: u64,
    pub host_promotions: u64,
    pub disk_promotions: u64,
    pub host_demotions: u64,
    pub disk_demotions: u64,
    pub transfer_bytes: u64,
    pub transfer_wait: Duration,
    pub demand_hits: u64,
    pub demand_misses: u64,
    pub in_flight_waits: u64,
    pub failures: u64,
    pub prefill_full_attention_blocks: u64,
    pub prefill_full_attention_bytes: u64,
    pub decode_full_attention_blocks: u64,
    pub decode_full_attention_bytes: u64,
    pub attention_scratch_peak_bytes: u64,
}

Current residency and cumulative activity attributable to one layer or a bounded overflow group of layers.

Fields

logical_cached_tokens: u64

Logical cached tokens.

key_value_blocks: u64

Sealed key/value blocks.

compressed_latent_blocks: u64

Sealed compressed-latent/rotary blocks.

device_blocks: u64

Blocks cataloged on the execution device.

host_blocks: u64

Blocks cataloged in host memory.

disk_blocks: u64

Blocks cataloged on disk.

current_device_bytes: u64

Current logical device bytes, including mutable tails.

current_host_bytes: u64

Current physical host allocation capacity, including in-flight ownership.

current_disk_bytes: u64

Current logical disk bytes.

mutable_tail_bytes: u64

Current bytes in mutable tails.

in_flight_write_blocks: u64

Blocks whose host buffers are owned by background disk writes.

in_flight_write_bytes: u64

Physical host allocation capacity owned by background disk writes.

in_flight_host_demotion_blocks: u64

Blocks retaining both device and host allocations during demotion.

in_flight_host_demotion_bytes: u64

Physical host allocation capacity charged during device demotion.

protected_recent_blocks: u64

Recent device blocks protected from demotion.

protected_prefix_blocks: u64

Prefix or sink blocks protected for attention.

host_promotions: u64

Cumulative host promotions.

disk_promotions: u64

Cumulative disk promotions.

host_demotions: u64

Cumulative device demotions.

disk_demotions: u64

Cumulative demotions to disk.

transfer_bytes: u64

Cumulative logical bytes transferred between tiers.

transfer_wait: Duration

Cumulative host time at transfer ownership boundaries.

demand_hits: u64

Cumulative demand hits.

demand_misses: u64

Cumulative demand misses.

in_flight_waits: u64

Cumulative in-flight waits.

failures: u64

Cumulative residency or transfer failures.

prefill_full_attention_blocks: u64

Sealed and mutable blocks scanned by full attention during prefill.

prefill_full_attention_bytes: u64

Logical bytes scanned by full attention during prefill.

decode_full_attention_blocks: u64

Sealed and mutable blocks scanned by full attention during decode.

decode_full_attention_bytes: u64

Logical bytes scanned by full attention during decode.

attention_scratch_peak_bytes: u64

Peak logical scratch bytes used by this layer’s attention.

Implementations

impl CacheLayerResidencyStats

pub fn accumulate(&mut self, other: &Self)

Adds another exact row into this aggregate, preserving peak fields.

Trait Implementations

impl Clone for CacheLayerResidencyStats

fn clone(&self) -> CacheLayerResidencyStats

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for CacheLayerResidencyStats

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for CacheLayerResidencyStats

fn default() -> CacheLayerResidencyStats

Returns the “default value” for a type.

impl Eq for CacheLayerResidencyStats

impl PartialEq for CacheLayerResidencyStats

fn eq(&self, other: &CacheLayerResidencyStats) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for CacheLayerResidencyStats