pub struct WriterContentionDiagnostics {Show 31 fields
pub writer_acquisitions: u64,
pub pooled_writer_acquisitions: u64,
pub standalone_writer_acquisitions: u64,
pub writer_task_acquisitions: u64,
pub writer_acquisition_timeouts: u64,
pub writer_lease_timeouts: u64,
pub configured_guard_deadline_ms: Option<u64>,
pub configured_checkout_timeout_ms: u64,
pub effective_writer_wait_bound_ms: u64,
pub direct_writer_busy_refusals: u64,
pub writer_task_begin_busy: u64,
pub writer_task_begin_busy_absorbed: u64,
pub writer_task_begin_errors: u64,
pub writer_task_request_failures: u64,
pub writer_task_side_effects_unknown: u64,
pub audit_append_failures: Option<u64>,
pub audit_append_failures_unavailable_reason: Option<String>,
pub audit_obligation_append_failures: Option<u64>,
pub audit_obligation_append_failures_unavailable_reason: Option<String>,
pub audit_batch_flush_failures: Option<u64>,
pub audit_batch_flush_failures_unavailable_reason: Option<String>,
pub audit_degraded_rows: Option<u64>,
pub audit_degraded_rows_unavailable_reason: Option<String>,
pub audit_degraded: Option<bool>,
pub audit_degraded_unavailable_reason: Option<String>,
pub audit_admission_refused_obligations: Option<u64>,
pub audit_admission_refused_obligations_last_at_ms: Option<u64>,
pub audit_admission_refused_obligations_unavailable_reason: Option<String>,
pub audit_admission_unresolved_obligations: Option<u64>,
pub audit_admission_unresolved_obligations_last_at_ms: Option<u64>,
pub audit_admission_unresolved_obligations_unavailable_reason: Option<String>,
}Expand description
One typed snapshot of writer-contention signals.
writer_acquisitions is the aggregate of the three explicit connection
classes below. writer_acquisition_timeouts remains specific to the
finite-wait pool-mutex stage; standalone SQLite failures and writer-task
BEGIN failures have different ADR-135 F6 stages and are not mislabeled as
pool checkout timeouts. Those stages now carry their OWN failure counters
(writer_task_begin_busy, writer_task_begin_busy_absorbed,
writer_task_begin_errors) rather than being absent: refusing to mislabel
a failure is not a reason to omit it, and an omitted failure counter fails
toward looking healthy, which is the reading an operator believes.
audit_append_failures is supplied by the runtime
because the audit store lives above khive-db; direct khive-db callers
receive None plus an explicit reason instead of a fabricated zero.
Fields§
§writer_acquisitions: u64Successful acquisitions across pooled, standalone, and writer-task connection classes.
pooled_writer_acquisitions: u64Successful finite-wait main-pool mutex checkouts.
standalone_writer_acquisitions: u64Successful per-operation file-backed standalone writer opens.
writer_task_acquisitions: u64Successful writer-task ownership acquisitions.
writer_acquisition_timeouts: u64Main-pool writer checkouts that exhausted their finite deadline.
writer_lease_timeouts: u64Writer acquisitions refused because another writer held the volume
lease past the guard deadline (CapacityUnavailable, phase lock).
Every writer class takes the lease before its connection, so ordinary
same-volume writer contention is counted here, not in
writer_acquisition_timeouts.
configured_guard_deadline_ms: Option<u64>The guard deadline a writer waits for the volume lease, in ms; None
when the pool takes no lease (in-memory or read-only).
configured_checkout_timeout_ms: u64checkout_timeout, which bounds only the pool-mutex wait that comes
after the lease.
effective_writer_wait_bound_ms: u64The effective pooled-writer wait bound under contention: the guard
deadline for the lease, then checkout_timeout for the pool mutex. A
pool configured with a 50 ms checkout_timeout and the default 2000 ms
guard deadline can wait about 2050 ms before refusing.
direct_writer_busy_refusals: u64Final instrumented direct execution refusals retaining primary SQLITE_BUSY, once per operation; excludes SQLITE_LOCKED and task/reader/open/admission errors.
writer_task_begin_busy: u64Every writer-task BEGIN IMMEDIATE attempt refused busy or locked,
whether or not a subsequent bounded retry absorbed it. A refusal not
absorbed by a retry also surfaces to the caller as the retryable
writer_task_begin_busy stage.
writer_task_begin_busy_absorbed: u64Subset of writer_task_begin_busy absorbed by the bounded retry
before the request closure ran, so the caller never observed that
particular refusal. writer_task_begin_busy - writer_task_begin_busy_absorbed
is the count of refusals a caller actually observed.
writer_task_begin_errors: u64Writer-task BEGIN IMMEDIATE attempts that failed for a reason other
than busy or locked.
writer_task_request_failures: u64Dequeued writer-task requests that reached the writer seam and returned error, counted once per request. Sourced directly from the pool’s own acquisition-site counters, so — unlike the runtime-supplied fields below — it is populated identically for every caller.
writer_task_side_effects_unknown: u64Subset of writer_task_request_failures whose terminal state was
WriterTaskRequestState::SideEffectsUnknown.
audit_append_failures: Option<u64>Process-wide audit appends whose errors were logged and swallowed —
pure-observability rows only (config-lock rows, best-effort recall
telemetry). An obligation-bearing row’s commit failure (a gate
denial’s own audit row, a dispatch outcome, an unknown-verb row, or a
git.digest receipt) is never counted here: those either fail the
dispatch that produced them directly (visible to the caller as an
error, not as this counter moving) or, for a denial whose dispatch
already fails independent of the row, are tracked by the runtime’s
own separate obligation-failure counter instead. Summing this field
with audit_batch_flush_failures therefore does not double-count an
obligation-bearing generation failure against this one.
Why audit_append_failures is unavailable to this caller.
audit_obligation_append_failures: Option<u64>Process-wide obligation-bearing audit commit failures, including a
denial’s audit failure even though the denial already refuses the call.
This counts failed producer submissions, not failed batch generations;
do not add it to audit_batch_flush_failures as a disjoint total.
Why the runtime-owned obligation counter is unavailable to this caller.
audit_batch_flush_failures: Option<u64>Accepted audit-batch generations that reached a terminal non-commit
outcome after retry, including driver death; excludes preflight and
admission rejection. None for a direct khive-db caller, or when no
runtime audit-batch control has been wired into diagnostics.
Why audit_batch_flush_failures is unavailable to this caller.
audit_degraded_rows: Option<u64>Pure-observability audit rows released without a commit. None under
the same conditions as audit_batch_flush_failures.
Why audit_degraded_rows is unavailable to this caller.
audit_degraded: Option<bool>Monotonic process-lifetime flag set once any accepted row has been
released without a commit: a pure-observability row released degraded,
or a generation that failed to flush, whose rows are absent from the
audit trail whether or not their callers were told. None under the
same conditions as audit_batch_flush_failures.
Why audit_degraded is unavailable to this caller.
audit_admission_refused_obligations: Option<u64>Per-dispatch audit rows for an explicitly allowlisted, domain-write-free
read verb (VerbRegistry::ADMISSION_DEGRADE_SAFE_VERBS) that were
refused before they could be enqueued on the audit lane
(AuditTerminalReason::QueueAdmissionExhausted) while the dispatch
still reported its own successful result (ADR-103 Amendment 3, ADR-133
Amendment 1). This is a confirmed, terminal accounting loss: the row
never shared a generation with anyone and will never commit, so it
undercounts brain.event_counts’s cost totals for exactly the rows
counted here. Disjoint from audit_degraded_rows (a different reason:
persistent commit failure of a pure-observability row, not admission
pressure) and from audit_admission_unresolved_obligations (a row that
was enqueued and may still commit). None under the same conditions as
audit_batch_flush_failures.
CUMULATIVE since process start. Nothing decrements it, so a value that
holds steady under traffic means no refusal occurred in that window
rather than a stalled subsystem (#2791); read
audit_admission_refused_obligations_last_at_ms beside it to tell the
two apart.
audit_admission_refused_obligations_last_at_ms: Option<u64>Wall-clock milliseconds at which audit_admission_refused_obligations
last moved in the serving process, or None if it has never moved.
None alongside a count of zero is the ordinary quiet case; None
alongside a non-zero count cannot occur and would indicate the two are
being produced from different processes.
Why audit_admission_refused_obligations is unavailable to this caller.
audit_admission_unresolved_obligations: Option<u64>Per-dispatch audit rows for an explicitly allowlisted, domain-write-free
read verb (VerbRegistry::ADMISSION_DEGRADE_SAFE_VERBS) that were
already enqueued but had not resolved when the caller’s admission
wait deadline elapsed (AuditTerminalReason::AdmissionDeadlineExpired)
while the dispatch still reported its own successful result (ADR-103
Amendment 3, ADR-133 Amendment 1). Unlike
audit_admission_refused_obligations, a row counted here is not a
confirmed loss — it may still be committed by the generation driver
independently of the caller’s timeout — so this field is an upper
bound on the eventual undercount, not the undercount itself. None
under the same conditions as audit_batch_flush_failures.
CUMULATIVE since process start, despite the set-shaped name. There is no
live set of unresolved obligations and nothing resolves this counter:
each increment records one past deadline expiry, and the row it counted
most likely committed afterwards. A steady value under traffic means no
deadline expired in that window, which is the healthy reading (#2791).
Read audit_admission_unresolved_obligations_last_at_ms beside it: a
non-zero count whose mark is old is history, the same count with a
recent mark is an active condition.
audit_admission_unresolved_obligations_last_at_ms: Option<u64>Wall-clock milliseconds at which
audit_admission_unresolved_obligations last moved in the serving
process, or None if it has never moved.
Why audit_admission_unresolved_obligations is unavailable to this
caller.
Trait Implementations§
Source§impl Clone for WriterContentionDiagnostics
impl Clone for WriterContentionDiagnostics
Source§impl Debug for WriterContentionDiagnostics
impl Debug for WriterContentionDiagnostics
impl Eq for WriterContentionDiagnostics
impl StructuralPartialEq for WriterContentionDiagnostics
Auto Trait Implementations§
impl Freeze for WriterContentionDiagnostics
impl RefUnwindSafe for WriterContentionDiagnostics
impl Send for WriterContentionDiagnostics
impl Sync for WriterContentionDiagnostics
impl Unpin for WriterContentionDiagnostics
impl UnsafeUnpin for WriterContentionDiagnostics
impl UnwindSafe for WriterContentionDiagnostics
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
Source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
Source§fn equivalent(&self, key: &K) -> bool
fn equivalent(&self, key: &K) -> bool
key and return true if they are equal.Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§fn in_current_span(self) -> Instrumented<Self> ⓘ
fn in_current_span(self) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more