Struct HttpCache 

pub struct HttpCache { /* private fields */ }

The state machine for http caching

This object is used to handle the state and transitions for HTTP caching through the life of a request.

Implementations

impl HttpCache

pub fn new() -> Self

Create a new HttpCache.

Caching is not enabled by default.

pub fn enabled(&self) -> bool

Whether the cache is enabled

pub fn bypassing(&self) -> bool

Whether the cache is being bypassed

pub fn phase(&self) -> CachePhase

Return the CachePhase

pub fn upstream_used(&self) -> bool

Whether anything was fetched from the upstream

This essentially checks all possible CachePhase who need to contact the upstream server

pub fn storage_type_is<T: 'static>(&self) -> bool

Check whether the backend storage is the type T.

pub fn lock_publish_fill_tokens(&self, tokens: &[u64])

Say something about this request’s cache fill, so readers coalescing behind it that cannot use it stop waiting. See lock::UnusableFills for the reader’s side.

No-op unless this request holds the write lock. Each call replaces the last, so publish the whole set each time.

pub fn release_write_lock(&mut self, reason: NoCacheReason)

Release the cache lock if the current request is a cache writer.

Generally callers should prefer using disable when a cache lock should be released due to an error to clear all cache context. This function is for releasing the cache lock while still keeping the cache around for reading, e.g. when serving stale.

pub fn disable(&mut self, reason: NoCacheReason)

Disable caching

pub fn bypass(&mut self)

Set the cache to bypass

Panic

This call is only allowed in CachePhase::CacheKey phase (before any cache lookup is performed). Use it in any other phase will lead to panic.

pub fn enable( &mut self, storage: &'static (dyn Storage + Sync), eviction: Option<&'static (dyn EvictionManager + Sync)>, predictor: Option<&'static (dyn CacheablePredictor + Sync)>, cache_lock: Option<&'static CacheKeyLockImpl>, option_overrides: Option<CacheOptionOverrides>, )

Enable the cache

• storage: the cache storage backend that implements storage::Storage
• eviction: optionally the eviction manager, without it, nothing will be evicted from the storage
• predictor: optionally a cache predictor. The cache predictor predicts whether something is likely to be cacheable or not. This is useful because the proxy can apply different types of optimization to cacheable and uncacheable requests.
• cache_lock: optionally a cache lock which handles concurrent lookups to the same asset. Without it such lookups will all be allowed to fetch the asset independently.

pub fn set_cache_lock( &mut self, cache_lock: Option<&'static CacheKeyLockImpl>, option_overrides: Option<CacheOptionOverrides>, )

Set the cache lock implementation.

Panic

Must be called before a cache lock is attempted to be acquired, i.e. in the cache_key_callback or cache_hit_filter phases.

pub fn set_admission_policy(&mut self, policy: &'static dyn AdmissionPolicy)

Set the AdmissionPolicy used to decide whether an absent key may fill the cache.

The policy is only consulted when storage reports a raw miss. Entries rejected by valid_after filtering still follow the normal miss path.

Panics

Panics after a cache lookup or fill has started.

pub fn enable_tracing(&mut self, parent_span: Span)

pub fn get_cache_span(&self) -> Option<SpanHandle>

pub fn get_miss_span(&self) -> Option<SpanHandle>

pub fn get_hit_span(&self) -> Option<SpanHandle>

pub fn set_cache_key(&mut self, key: CacheKey)

Set the cache key

Panic

Cache key is only allowed to be set in its own phase. Set it in other phases will cause panic.

pub fn cache_key(&self) -> &CacheKey

Return the cache key used for asset lookup

Panic

Can only be called after the cache key is set and the cache is not disabled. Panic otherwise.

pub fn max_file_size_bytes(&self) -> Option<usize>

Return the max size allowed to be cached.

pub fn set_max_file_size_bytes(&mut self, max_file_size_bytes: usize)

Set the maximum response body size in bytes that will be admitted to the cache.

Response header size should not contribute to the max file size.

To track body bytes, call track_bytes_for_max_file_size.

pub fn track_body_bytes_for_max_file_size(&mut self, bytes_len: usize) -> bool

Record body bytes for the max file size tracker.

The bytes_len input contributes to a cumulative body byte tracker.

Once the cumulative body bytes exceeds the maximum allowable cache file size (as configured by set_max_file_size_bytes), then the return value will be false.

Else the return value is true as long as the max file size is not exceeded. If max file size was not configured, the return value is always true.

pub fn exceeded_max_file_size(&self) -> bool

Check if the max file size has been exceeded according to max file size tracker.

Return true if max file size was exceeded.

pub fn cache_found( &mut self, meta: CacheMeta, hit_handler: HitHandler, hit_status: HitStatus, )

Set that cache is found in cache storage.

This function is called after Self::cache_lookup() which returns the CacheMeta and HitHandler.

The hit_status enum allows the caller to force expire assets.

pub fn cache_miss(&mut self)

Mark self to be cache miss.

This function is called after Self::cache_lookup() finds nothing or the caller decides not to use the assets found.

Panic

Panic in other phases.

pub fn hit_handler(&mut self) -> &mut HitHandler

Return the HitHandler

Panic

Call this after Self::cache_found(), panic in other phases.

pub fn miss_body_reader(&mut self) -> Option<&mut HitHandler>

Return the body reader during a cache admission (miss/expired) which decouples the downstream read and upstream cache write

pub fn support_streaming_partial_write(&self) -> Option<bool>

Return whether the underlying storage backend supports streaming partial write.

Returns None if cache is not enabled.

pub async fn finish_hit_handler(&mut self) -> Result<()>

Call this when cache hit is fully read.

This call will release resource if any and log the timing in tracing if set.

Panic

Panic in phases where there is no cache hit.

pub async fn set_miss_handler(&mut self) -> Result<()>

Set the MissHandler according to cache_key and meta, can only call once

pub fn miss_handler(&mut self) -> Option<&mut MissHandler>

Return the MissHandler to write the response body to cache.

None: the handler has not been set or already finished

pub async fn finish_miss_handler(&mut self) -> Result<()>

Finish cache admission

If self is dropped without calling this, the cache admission is considered incomplete and should be cleaned up.

This call will also trigger eviction if set.

pub fn set_cache_meta(&mut self, meta: CacheMeta)

Set the CacheMeta of the cache

Panics

Panics unless called in CachePhase::Miss or CachePhase::Stale. In stale phase, the stale metadata must still be present.

pub async fn revalidate_cache_meta(&mut self, meta: CacheMeta) -> Result<bool>

Set the CacheMeta of the cache after revalidation.

Certain info such as the original cache admission time will be preserved. Others will be replaced by the input meta.

pub fn revalidate_merge_header(&mut self, resp: &RespHeader) -> ResponseHeader

After a successful revalidation, update certain headers for the cached asset such as Etag with the fresh response header resp.

pub fn revalidate_uncacheable( &mut self, header: ResponseHeader, reason: NoCacheReason, )

Mark this asset uncacheable after revalidation

pub fn set_stale_updating(&mut self)

Mark this asset as stale, but being updated separately from this request.

pub fn update_variance(&mut self, variance: Option<HashBinary>)

Update the variance of the CacheMeta.

Note that this process may change the lookup key, and eventually (when the asset is written to storage) invalidate other cached variants under the same primary key as the current asset.

pub fn cache_meta(&self) -> &CacheMeta

Return the CacheMeta of this asset

Panic

Panic in phases which has no cache meta.

pub fn maybe_cache_meta(&self) -> Option<&CacheMeta>

Return the CacheMeta of this asset if any

Different from Self::cache_meta(), this function is allowed to be called in any phase and will not panic due to a wrong phase. It returns the cache meta in the phases where one may be set (CachePhase::Miss, CachePhase::Stale, CachePhase::StaleUpdating, CachePhase::Expired, CachePhase::Hit, CachePhase::Revalidated, and CachePhase::RevalidatedNoCache); in all other phases it returns None because no cache meta can exist.

pub fn maybe_cache_key(&self) -> Option<&CacheKey>

Return the CacheKey of this asset if any.

This is allowed to be called in any phase. If the cache key callback was not called, this will return None.

pub async fn cache_lookup(&mut self) -> Result<Option<(CacheMeta, HitHandler)>>

Perform the cache lookup from the given cache storage with the given cache key

A cache hit will return CacheMeta which contains the header and meta info about the cache as well as a HitHandler to read the cache hit body.

When an admission policy defers a raw storage miss, this returns Ok(None) and disables caching with NoCacheReason::Deferred. Callers must check Self::enabled() before calling Self::cache_miss().

Admission is observed at most once per HttpCache, on an initial CachePhase::CacheKey raw storage miss. Retried lookups and stale refills reuse the existing admission outcome or proceed without another observation.

Entries rejected by valid_after filtering are not raw storage misses and bypass admission. After an invalidation, admission therefore does not provide additional suppression for concurrent fills beyond the configured cache-lock behavior.

Panic

Panic in other phases.

pub fn cache_vary_lookup( &mut self, variance: HashBinary, meta: &CacheMeta, ) -> bool

Update variance and see if the meta matches the current variance

cache_lookup() -> compute vary hash -> cache_vary_lookup() This function allows callers to compute vary based on the initial cache hit. meta should be the ones returned from the initial cache_lookup()

• return true if the meta is the variance.
• return false if the current meta doesn’t match the variance, need to cache_lookup() again

pub fn is_cache_locked(&self) -> bool

Whether this request is behind a cache lock in order to wait for another request to read the asset.

pub fn is_cache_lock_writer(&self) -> bool

Whether this request is the leader request to fetch the assets for itself and other requests behind the cache lock.

pub fn cache_lock_max_retries(&self) -> Option<usize>

Maximum number of cache lock retries configured for this request.

pub fn take_write_lock(&mut self) -> (WritePermit, &'static CacheKeyLockImpl)

Take the write lock from this request to transfer it to another one.

Panic

Call is_cache_lock_writer() to check first, will panic otherwise.

pub fn set_write_lock(&mut self, write_lock: WritePermit)

Set the write lock, which is usually transferred from Self::take_write_lock()

Panic

Panics if cache lock was not originally configured for this request.

pub fn can_serve_stale_error(&self) -> bool

Whether this asset is staled and stale if error is allowed

pub fn can_serve_stale_updating(&self) -> bool

Whether this asset is staled and stale while revalidate is allowed.

pub async fn cache_lock_wait(&mut self) -> LockWaitOutcome

Wait for the cache read lock to be unlocked

A request carrying an lock::UnusableFills on its cache key can also stop early with LockWaitOutcome::Abandoned, which Self::lock_abandon keeps afterwards.

Panic

Check Self::is_cache_locked(), panic if this request doesn’t have a read lock.

pub fn lock_duration(&self) -> Option<Duration>

How long did this request wait behind the read lock

pub fn lock_abandon(&self) -> Option<LockAbandon>

The fill this request stopped waiting over, and why it could not use it.

Set only when Self::cache_lock_wait returned LockWaitOutcome::Abandoned. Absent for every other outcome, including LockWaitOutcome::GiveUp, which is the writer giving up rather than this request abandoning the wait.

pub fn lookup_duration(&self) -> Option<Duration>

How long did this request spent on cache lookup and reading the header

pub fn admission_decision(&self) -> Option<Decision>

Return the Decision made for an absent cache key.

pub async fn purge(&self) -> Result<bool>

Delete the asset from the cache storage

Panic

Need to be called after the cache key is set. Panic otherwise.

pub async fn expire(&self) -> Result<bool>

Mark the asset stale in the cache storage so the next read revalidates it.

Storage that cannot mark an asset stale deletes it instead, so this never leaves a fresh asset behind.

Panic

Need to be called after the cache key is set. Panic otherwise.

pub fn spawn_async_purge( &self, context: &'static str, ) -> JoinHandle<Result<bool>>

Delete the asset from the cache storage via a spawned task. Returns corresponding JoinHandle of that task.

Panic

Need to be called after the cache key is set. Panic otherwise.

pub fn cacheable_prediction(&self) -> bool

Check the cacheable prediction

Return true if the predictor is not set

pub fn predicted_uncacheable_reason(&self) -> Option<NoCacheReason>

The reason the predictor remembered for this key when Self::bypass ran.

None when the cache was not bypassed, when no predictor is configured, or when the predictor does not track reasons. Callers must treat None as “unknown” rather than as evidence about the previous response.

pub fn response_became_cacheable(&self)

Tell the predictor that this response, which is previously predicted to be uncacheable, is cacheable now.

pub fn response_became_uncacheable(&self, reason: NoCacheReason)

Tell the predictor that this response is uncacheable so that it will know next time this request arrives.

pub fn tag_as_subrequest(&mut self)

Tag all spans as being part of a subrequest.