Expand description
OTLP span emission (gateway L4 observability) — SDK-free tracing straight from the request path.
When [llm.telemetry] is enabled, EdgeGuard emits one OpenInference/OTLP span per metered LLM
request to an OTLP/HTTP /v1/traces receiver (e.g. evald). Because the proxy sits in the request
path, the span carries the correct model, per-tier tokens, computed cost, and server-side
TTFT/TPOT/latency with no client SDK, no import-order fragility, and no per-framework instrumentor
drift — the exact failure class that plagues in-process instrumentation (nested usage, dropped
spans, async nesting breakage). Emission is fire-and-forget: it never blocks or fails the
client response.
The wire format is OTLP-JSON posted with the crate’s existing HTTP client, so the data plane
stays a single static binary (no protobuf codegen, no OpenTelemetry SDK). The span attribute keys
are exactly the OpenInference / gen_ai.* keys a downstream store normalizes (llm.model_name,
llm.token_count.*, input.value, …), so a gateway span round-trips into evald unchanged.
Structs§
- Span
Record - A metered LLM request rendered into span form.
input/outputstayNoneunless content capture is enabled (they are populated, already DLP-redacted, at the wiring site). - Telemetry
Runtime - The compiled telemetry runtime carried on the proxy
Runtime. - Trace
Context - W3C trace context for one emitted span.
parent_span_idis set when an inboundtraceparentstitched this gateway span under an app-side span (so both land in one trace).
Functions§
- build_
export_ json - Build the OTLP-JSON
ExportTraceServiceRequestfor one span. The attribute keys are the OpenInference /gen_ai.*keys a downstream OTel store normalizes, so the span round-trips. Ints are encoded as strings (the protobuf int64 → JSON mapping OTLP-JSON uses). - prepare_
content - Lossy-UTF8 a captured body and truncate it to
max_byteson a char boundary (with a marker), so a large prompt/response can’t bloat the emitted span. Used at the content-capture wiring site.