Skip to main content

Module telemetry

Module telemetry 

Source
Expand description

OTLP span emission (gateway L4 observability) — SDK-free tracing straight from the request path.

When [llm.telemetry] is enabled, EdgeGuard emits one OpenInference/OTLP span per metered LLM request to an OTLP/HTTP /v1/traces receiver (e.g. evald). Because the proxy sits in the request path, the span carries the correct model, per-tier tokens, computed cost, and server-side TTFT/TPOT/latency with no client SDK, no import-order fragility, and no per-framework instrumentor drift — the exact failure class that plagues in-process instrumentation (nested usage, dropped spans, async nesting breakage). Emission is fire-and-forget: it never blocks or fails the client response.

The wire format is OTLP-JSON posted with the crate’s existing HTTP client, so the data plane stays a single static binary (no protobuf codegen, no OpenTelemetry SDK). The span attribute keys are exactly the OpenInference / gen_ai.* keys a downstream store normalizes (llm.model_name, llm.token_count.*, input.value, …), so a gateway span round-trips into evald unchanged.

Structs§

SpanRecord
A metered LLM request rendered into span form. input/output stay None unless content capture is enabled (they are populated, already DLP-redacted, at the wiring site).
TelemetryRuntime
The compiled telemetry runtime carried on the proxy Runtime.
TraceContext
W3C trace context for one emitted span. parent_span_id is set when an inbound traceparent stitched this gateway span under an app-side span (so both land in one trace).

Functions§

build_export_json
Build the OTLP-JSON ExportTraceServiceRequest for one span. The attribute keys are the OpenInference / gen_ai.* keys a downstream OTel store normalizes, so the span round-trips. Ints are encoded as strings (the protobuf int64 → JSON mapping OTLP-JSON uses).
prepare_content
Lossy-UTF8 a captured body and truncate it to max_bytes on a char boundary (with a marker), so a large prompt/response can’t bloat the emitted span. Used at the content-capture wiring site.