Module shadow 

v0.7.0 Form 5 — shadow-mode telemetry pipeline.

Per-recall observations land in confidence_shadow_observations when AI_MEMORY_CONFIDENCE_SHADOW=1, sampled at the rate carried by AI_MEMORY_CONFIDENCE_SHADOW_SAMPLE_RATE (0.0..=1.0; default 1.0 when shadow is enabled).

Audit-honest contract: shadow mode never silently overrides the caller’s confidence. The recall ranker still uses the caller value downstream; the derived value is only persisted for later calibration. This is the load-bearing property that lets operators safely turn the engine on in production.

Surface

• observe — write a single shadow row.
• should_sample — gate helper that consults the cached config (enabled flag + sample rate). The first call captures the env-var pair into a process-wide OnceLock; subsequent calls return the cached value without hitting the std::env syscall. This is the PERF-9 fix (Cluster G, issue #767): pre-Cluster-G, every recall touch re-read both env vars on the hot path.
• gc_observations — periodic GC sweep deleting rows older than the configured retention window. Wired into the daemon’s spawn_gc_loop from daemon_runtime.rs. PERF-4 fix.

Structs

ShadowConfig

Cached shadow config — captured on first access from the env-var pair. The recall hot path reads this OnceLock instead of calling std::env::var per touch (PERF-9).

ShadowObservation

One row of confidence_shadow_observations as exposed to readers.

Constants

DEFAULT_SHADOW_RETENTION_DAYS

Default retention window for the periodic GC sweep on confidence_shadow_observations. 30 days mirrors the Form 5 calibration window: the sweep runs against the table that the calibration sweep reads from, so an aligned default keeps the pipeline “what you see in the report is what you have on disk.”

ENV_SHADOW

Environment-variable opt-in for shadow mode.

ENV_SHADOW_SAMPLE_RATE

Optional sample rate (0.0..=1.0). When unset, defaults to 1.0 while shadow is enabled — every recall touch lands a row.

Functions

gc_observations

Delete confidence_shadow_observations rows whose observed_at is older than now - retention_days. Returns the number of rows removed. Called periodically from the daemon GC loop (daemon_runtime::spawn_gc_loop) to close PERF-4 (unbounded table growth on long-running shadow-mode deployments).

observations_since

Pull every shadow observation in namespace newer than since (RFC3339). When since is None, returns all rows. Used by tests and ad-hoc debugging; the calibration sweep itself uses crate::confidence::calibrate::calibrate_from_shadow which streams a SQL-side aggregation instead of materialising every row. The result vector is ordered by observed_at ASC for stable replays.

observe

Append one row to confidence_shadow_observations.

sample_rate

Resolve the configured sample rate. Reads the cached ShadowConfig — no env syscall on the hot path.

shadow_config

Returns the cached shadow config, capturing it from env vars on the first call. PERF-9: subsequent calls do NOT touch std::env — the returned reference points into a process-wide OnceLock.

shadow_enabled

Returns true when ENV_SHADOW was set to "1" at first-access time. Reads the cached ShadowConfig — no env syscall on the hot path.

should_sample

Decide whether to sample a recall touch for shadow observation.