Skip to main content

Module origin_probe

Module origin_probe 

Source
Expand description

Live-origin probe memo (#1130 pt3): a bounded, TTL’d cache of HEAD/HeadObject existence answers so a per-probe origin existence check does not hammer the backend.

rescan_origins (#1130) builds the zero-I/O origin_held index from enumerate() ∪ pins, which for http/s3 covers pinned hashes ONLY (neither backend lists). A non-pinned bucket object is therefore invisible to the probe. This memo backs the per-probe fallback that consults the origin directly (CacheEngine::origin_probe_size): it caches BOTH a positive answer (Present(size)) under a long positive TTL and a negative one (Absent) under a short negative TTL, so a flood of random-hash probes turns into at most one HeadObject per hash per TTL window rather than one per probe, while a stale Absent cannot hide newly-available own content for more than the short negative window.

The memo is deliberately a plain data structure with an injected clock (now: Instant) so its expiry and capacity behaviour are unit-testable without sleeping or a background sweeper. Expiry is lazy (on lookup); the capacity bound is enforced on insert.

Structs§

OriginProbeMemo
Bounded, lazily-expiring memo of origin-probe answers.
OriginProbePolicy
Live-origin probe policy: the three answer TTLs, the per-probe HEAD ceiling, and the memo’s capacity, resolved from the cache.origin_probe_* knobs.

Enums§

Presence
A memoised existence answer for a hash against the configured origins.

Constants§

DEFAULT_ORIGIN_PROBE_FAULT_TTL
Default fault TTL (cache.origin_probe_fault_ttl_sec). Longer than the negative TTL because a client retrying one hash against a failing origin is the common shape, and this memo is keyed per hash: it collapses the repeats of a hash, not the namespace — during an outage, N distinct hashes still cost N live HEADs per window. Shorter than the positive TTL because a memoised fault costs client-visible availability until it expires, so a recovered origin must be re-probed promptly.
DEFAULT_ORIGIN_PROBE_MEMO_CAPACITY
Default cap on distinct memoised hashes (cache.origin_probe_memo_capacity). Bounds memory under a random-hash probe flood.
DEFAULT_ORIGIN_PROBE_NEGATIVE_TTL
Default negative TTL (cache.origin_probe_negative_ttl_sec). Short on purpose: it bounds how long a stale Absent can hide newly-available own content. A random-hash flood never repeats a hash within any window, so a short negative TTL barely changes the flood cost while capping the fresh-content dark window.
DEFAULT_ORIGIN_PROBE_TIMEOUT
Default per-probe live-HEAD ceiling (cache.origin_probe_timeout_ms) — a slow origin must never stall the probe hot path.
DEFAULT_ORIGIN_PROBE_TTL
Default TTL for a memoised positive origin-probe answer (cache.origin_probe_ttl_sec). Canonical seconds value from decdn-config-types, wrapped as a Duration.