Skip to main content

Crate subms_otel

Crate subms_otel 

Source
Expand description

subms-otel - OpenTelemetry bridge for the subms perf harness.

Wires the harness’s SubMsObserver hook + the post-bench SubMsBenchSummary + the SubMsTimer timeline into OpenTelemetry Histogram / Span emission, without the harness itself ever pulling in the OTEL dependency tree.

§Feature flags

Capability features:

  • bridge (default) - always-available bridge: post-hoc helpers, CompositeObserver, counter + gauge surface constants, the reference-drift recorder.
  • observer - both [OtelObserver] (sync) and [OtelObserverAsync] (hand-rolled SPSC ring; target overhead < 300 ns per on_record).
  • exemplars - per-bucket tail-debug exemplar reservoir ([ExemplarReservoir]). Configurable K; default K = 5.
  • tracing - span emission per stage record + W3C TraceContext parent inheritance ([TracingObserver]).

Bootstrap:

  • autoconfig - env-driven one-line SDK wiring + runtime observer registry.

Exporter helpers (independent; pick what you ship):

  • exporter-otlp - [OtlpBuilder] + [ExporterOtlpHelper].
  • exporter-prometheus - [PrometheusBuilder] + [ExporterPrometheusHelper].
  • exporter-stdout - [ExporterStdoutHelper].

Both observers honour the same semantic-conventions table:

Attribute keySource
subms.workloadctx.workload / summary.workload
subms.langctx.lang / summary.lang
subms.stagectx.stage / stage name
subms.stage.kindctx.stage_kind.as_str()
subms.recipe.slugmeta["subms.recipe.slug"]
subms.recipe.categorymeta["subms.recipe.category"]
subms.workload.featuremeta["subms.workload.feature"]
subms.workload.entriesinputs["entries"]
subms.workload.seedinputs["seed"]
subms.hostmeta["host"]
subms.hardware.tiermeta["hardware_tier"]
subms.crate.versionmeta["crate_version"]

Inputs and meta arrive only via on_summarize (the harness’s ObservationCtx is intentionally minimal). The sync observer therefore emits with just workload, lang, stage, and stage.kind per record; on_summarize re-emits the whole histogram bucket set under the fuller attribute set. The async observer caches the latest summary’s inputs+meta between drain passes and applies them to drained samples once available.

Structs§

CompositeObserver
Holds a list of inner observers and forwards every event to each. Lets a consumer combine, e.g., an [crate::OtelObserver] with a custom Prometheus impl and a logging observer at once.
ObservationCtx
Context passed to SubMsObserver::on_record for each recorded sample.
ReferenceDivergenceRecorder
Hold a single Counter instrument so a recipe wiring repeated cross-checks doesn’t pay for a fresh u64_counter build per divergence. Internally cached; cloning is cheap (the counter is reference-counted in OTEL).
SubMsBenchSummary
Typed summary of one bench run. Mirrors the on-disk JSON downstream tooling persists per workload.
SubMsOtelResource
Static helper carrying the detected resource set.
SubMsStageSummary
Per-stage summary. The counterpart of one stages.<name> entry in the subms JSON contract.
SubMsTimer
Autostart stopwatch with named checkpoints. Not thread-safe; use one per task or synchronise externally.
SubMsTimerCheckpoint
A single named milestone captured by SubMsTimer::mark / SubMsTimer::lap / SubMsTimer::stop.

Enums§

SubMsStageKind
What kind of operation a stage records. Observers use this to choose histogram bucket boundaries that fit the measurement scale.

Constants§

DROPPED_TOTAL_COUNTER_NAME
Counter name for back-pressure-dropped samples. Promoted from the async observer’s private constant so dashboards can wire it directly.
EXEMPLARS_KEPT_COUNTER_NAME
Counter name incremented when a sample is retained by an [ExemplarReservoir].
HISTOGRAM_NAME
Canonical name for the per-stage latency histogram. Consumers filter on this in Grafana / dashboards.
HISTOGRAM_UNIT
Canonical unit for the latency histogram. Seconds is OTEL convention for duration measurements - we emit ns / 1e9 to satisfy it.
IN_FLIGHT_GAUGE_NAME
ObservableGauge name published by [OtelObserverAsync] tracking the current channel depth.
OPS_TOTAL_COUNTER_NAME
Counter name incremented on every on_record regardless of which observer surface received the sample.
REFERENCE_DIVERGENCE_COUNTER_NAME
Counter name surfaced to OTEL. Stable across versions.

Traits§

SubMsObserver
Hook a sibling crate (or downstream consumer) can register against a [SubMsPerfHarness] to react to every recorded sample plus the post-bench summary. Both methods default to a no-op so adding methods to the trait later is non-breaking.

Functions§

attributes_from_ctx
Build the OTEL attribute set for a hot-path record. Just the harness identity the ObservationCtx carries - inputs and meta arrive later via the summary.
attributes_from_summary
Build the full attribute set for a stage when we have the post-bench summary. Pulls workload + lang + stage from the summary and folds in any of the canonical inputs/meta keys that are present.
divergence_attributes
Build the attribute set carried by every divergence Counter emission.
export_summary
Export every stage of a SubMsBenchSummary as a Histogram emission on the given Meter. Each percentile (p50, p99, p999, max, mean, stddev) and the downsampled chronological samples are recorded as histogram values under the stage’s full attribute set.
export_timer
Export a SubMsTimer timeline as a parent span + one child span per checkpoint. Span start times are reconstructed as now - elapsed_ns so the timing offsets the timer captured are preserved end-to-end.
histogram_boundaries
Histogram bucket boundaries (in seconds) keyed off the stage kind. Adapter code applies these when first constructing the Histogram instrument for a given stage kind.
record_reference_divergence
One-shot helper for recipes that don’t want to hold a ReferenceDivergenceRecorder. Builds the counter on the fly; cheap enough for the cross-check call site but allocates a fresh instrument every call - if you call this in a tight loop, build a ReferenceDivergenceRecorder once and reuse it.