Expand description
backbeat merge: combine several .bb dumps into one multi-instance .bb.
The dump format is inherently multi-instance — Meta, Intern, and Shard sections are each tagged
with the instance_id of the process that produced them, and the schema registry is content
addressed by a stable EventId. Merging therefore has two modes:
-
Default (dedup + trim). Decode each input’s records, drop duplicates — the same logged event re-captured by overlapping dumps (e.g. one process’s successive ring snapshots) — and re-pack the survivors into fresh, compact shards. The output is the smallest faithful dump: one row per distinct event. This is what you want before analysis, and it avoids paying to store/transfer redundant ring data.
-
--no-dedup(raw splice). Copy every input’s Meta/Intern/Shard bodies through verbatim and union the registries. Nothing is decoded, so it is cheap and lossless, but overlapping dumps keep their duplicates. Use this to quickly concatenate a host’s dumps for upload;convertalways dedups on the way out, so the duplicates never reach the final table.
Either way the registry is unioned by id (identical event types collapse to one entry) and
instance_ids are preserved, so converting the merged file yields exactly what converting the
inputs together would.
Functions§
- merge
- Merges
inputsinto one multi-instance dump written tooutput.