maxwell-cdc
maxwell-cdc turns the JSON that Maxwell's Daemon writes into typed
Rust messages. It parses bytes you already have, so it never touches MySQL. no_std, needs an
allocator.
Maxwell emits one JSON object per change, in twelve kinds: four carry a row, two bracket a bootstrap snapshot, six describe a schema change. Each kind is its own variant, so a field is optional here only when Maxwell may omit it.
use ;
let json = r#"{"database":"shop","table":"orders","type":"insert","ts":1477053217,"data":{"id":1,"total":"9.99"}}"#;
let message = parse?;
assert_eq!;
assert_eq!;
let Insert = &message else ;
assert_eq!;
assert_eq!;
# Ok::
Updates carry the previous values
Maxwell puts only the changed columns in old, so a diff is the intersection of the two maps.
use ;
let json = r#"{"database":"shop","table":"orders","type":"update","ts":1477053217,
"data":{"id":1,"status":"paid"},"old":{"status":"open"}}"#;
let Update = parse? else ;
let old = row.old.as_ref.expect;
assert_eq!;
assert_eq!;
# Ok::
Reading a stream
Maxwell's file and stdout producers write JSON Lines. parse_lines yields one result per line that
carries content, so one bad line does not end the stream, and the error names the line it came from.
use parse_lines;
let stream = concat!;
let : = parse_lines.partition;
assert_eq!;
let error = bad.as_ref.unwrap_err;
assert_eq!;
Telling a new Maxwell apart from a broken payload
Both arrive as a failure to deserialize, and serde_json reports them identically. ParseError
separates them, so a consumer can skip a message type it does not know while still failing loudly on
corrupt input.
use ;
// A type this crate does not model, most likely from a newer Maxwell.
let unknown = parse;
assert!;
// A type it does model, with a payload that is missing required fields.
let broken = parse;
assert!;
Nothing is silently dropped
Maxwell has many output flags, and its scripting hooks can inject arbitrary keys. Any field the model
does not name is kept verbatim in extra, so a message survives a round trip even when this crate is
older than the Maxwell that produced it.
use ;
let json = r#"{"database":"shop","table":"orders","type":"insert","data":{"id":1},
"injected_by_a_script":"keep me"}"#;
let Insert = parse? else ;
assert_eq!;
let written = to_value?;
assert_eq!;
# Ok::
Building a message
Every payload struct derives Default, so a test or a translation layer names the fields it cares
about and lets the rest default. This also means a field added in a later release does not break
construction.
use ;
let row = RowChange ;
let json = to_string?;
assert_eq!;
# Ok::
A defaulted value is a starting point, not a valid Maxwell message: the required fields come back empty.
Column order
data and old are a serde_json::Map, which by default sorts keys and therefore loses the column
order Maxwell emitted. Enable serde_json's preserve_order feature in your own build to keep it.
Types
Message and OpType are #[non_exhaustive], because Maxwell has added message types before and
will again. Match with a catch-all arm. Every payload struct derives Eq and Hash, so messages can
be deduplicated through a set.
Row timestamps (ts) are in seconds, schema-change timestamps in milliseconds. That is Maxwell's
choice. A MySQL DECIMAL wider than an f64 loses digits, because serde_json holds fractional
numbers as f64.