Module cache 

Backend-neutral mutable-cache ownership, storage, and admission algorithms. Backend-neutral ownership and admission algorithms for live model caches.

A cache backend owns concrete tensors, transfer objects, and persistent files. This module owns both each session’s block/lease/tail lifecycle and the process-wide accounting boundary shared by independently managed cache sessions. Reservations are RAII tokens: an admitted resource remains charged until its exact backend transition either publishes the bytes into a registered manager or drops the reservation.

Structs

CacheBlockLifecycle

Canonical logical ownership state for one backend cache session.

CacheBlockStorage

Canonical physical-resource state machine for one cache block.

CacheHostPromotion

Exact rollback ownership returned by a host-to-device promotion.

CacheIoExecutionState

Canonical admission and cancellation state for a bounded cache I/O worker.

CacheIoOperationKey

Exact identity of one backing-store operation.

CacheIoSubmission

Prepared cache I/O that admits physical work only when explicitly enqueued.

CacheIoSubmissionOutcome

Physical admission observations for one submission.

CacheIoTicket

Exact completion ownership for one coalesced cache I/O operation.

CacheIoWorker

Generic bounded background worker over opaque backend task and output types.

CacheLayerResidencyReport

Bounded, individually identified per-layer residency observations.

CacheLayerResidencyStats

Current residency and cumulative activity attributable to one layer or a bounded overflow group of layers.

CachePoolLimits

Process-wide finite limits shared by independently owned live caches.

CachePoolMembership

Exact ownership of one manager’s aggregate pool contribution.

CachePoolReport

Aggregate process-pool occupancy and high-water marks.

CachePoolReservation

Exact ownership of a temporary aggregate cache admission.

CachePoolUsage

Occupancy on each independently admitted cache resource axis.

CacheResidencyPool

Shareable aggregate ownership boundary for scheduler and standalone caches.

CacheResidencyReport

Aggregated logical device/disk and physical host residency observations.

CacheResidencyTelemetry

Backend-neutral collector for bounded cache activity and snapshot assembly.

LiveCacheBlockPublication

Runtime-owned unique staging and atomic publication for one live-cache block.

MutableCacheTail

Device-resident mutable state that has not yet become an immutable block.

PagedCacheOptions

Validated finite limits for block-addressable mutable state.

PromptCachePublication

A runtime-owned staging directory that publishes one immutable cache atomically.

Enums

CacheIoAdmission

Result of requesting bounded queue admission for prepared cache I/O.

CacheIoCompletionDisposition

Publication action after a physically started operation completes.

CacheIoExecutionStateError

Invalid cache I/O executor lifecycle use.

CacheIoOperationKind

Kind of asynchronous backing-store operation.

CacheIoPreparation

Result of preparing an exact cache I/O key.

CacheIoStartDisposition

Action a physical executor takes after receiving one admitted operation.

CacheIoWorkerError

Failure in generic cache I/O worker coordination or task execution.

CacheLifecycleError

Invalid backend-neutral cache ownership transition.

CachePoolError

Backend-neutral aggregate cache ownership failure.

CachePoolResource

Resource axis named by an aggregate admission failure.

CacheResidencyConfigurationError

Invalid backend-neutral mutable-cache residency configuration.

CacheResidencyPolicy

Selects fully resident mutable state or bounded block residency.

CacheStorageError

Illegal cache storage phase transition or completion observation.

CacheStoragePhase

Stable phase of one cache block’s physical resources.

LiveCacheDiskPolicy

Controls optional disk backing for a live inference cache.

LiveCachePublicationError

Filesystem failure while publishing one ephemeral live-cache block.

PromptCachePersistenceError

Filesystem, catalog, and publication failures for a reusable prompt cache.

Constants

CACHE_RESIDENCY_LAYER_REPORT_LIMIT

Maximum number of individually identified layers in a residency report.

MAX_PROMPT_CACHE_SHARD_HEADER_BYTES

Maximum accepted safetensors metadata header size for a prompt-cache shard.

PROMPT_CACHE_CURRENT_FILE

Atomic pointer to the active immutable generation.

PROMPT_CACHE_GENERATIONS_DIRECTORY

Directory containing immutable replacement generations.

Traits

CacheHostDemotionOperation

Opaque backend host-demotion completion with stable identity.

CacheIoOperation

Opaque backend I/O completion with an exact neutral key.

Functions

finalize_prompt_cache_shard

Synchronizes a newly written shard and returns its exact payload SHA-256.

hash_prompt_cache_shard_payload

Hashes the safetensors payload bytes, excluding its bounded metadata header.

inspect_prompt_cache

Reads and validates a prompt-cache manifest without loading tensor arrays.

resolve_prompt_cache_root

Resolves the active immutable generation selected by the durable pointer.

safe_prompt_cache_shard_path

Resolves a manifest shard path while rejecting traversal and symlink escapes.

validate_prompt_cache_manifest

Validates manifest structure and the bounded metadata of every referenced shard.