Expand description
Per-run usage metering (#704): what a run moved and what it asked its backends to do, so the CLI can account for cost per pipeline / row / dataset and enforce run budgets (#703).
A UsageMeter is attached to a Pipeline with
Pipeline::with_usage_meter. The
observability decorators count every record and an estimate of its
serialized size; the pre-labelled RoundtripRecorder each connector
already receives tallies backend round trips by op and carries
cost signals — backend-reported usage such as BigQuery’s bytes
billed or a streaming insert’s payload size — into the same meter. A
library caller that attaches no meter pays nothing: counting is skipped
entirely.
Byte counts are estimates of the JSON serialization (no allocation,
one walk over each value) — good enough to attribute cost and enforce a
max_bytes budget, never a wire-accurate figure.
Structs§
- Cost
Signal - A backend-reported usage figure a connector learned during the run —
BigQuery’s
totalBytesBilled, the payload size of a streaming insert, a warehouse’s credits — attributed to the connector that reported it. - Usage
Meter - The live, thread-safe counters of one run. Cheap to share (
Arc). - Usage
Snapshot - The counters a run accumulated, frozen for reporting and storage.
Enums§
- Usage
Side - Which side of the pipeline a round trip or signal came from.
Functions§
- estimate_
json_ bytes - Approximate serialized JSON size of
v, in bytes, without allocating. - estimate_
page_ bytes - Estimated serialized size of a page of records.