Crate spate_avro 

Avro deserialization for the Spate framework.

Decodes bare Avro datums as carried by Kafka messages — Confluent wire format (magic byte + schema id + datum), Avro single-object encoding, or raw datums with a fixed schema — into either dynamically-typed AvroValue records or your own serde types (AvroSerdeDeserializer).

Never block the pipeline thread

Registry schemas are fetched by an asynchronous task on the I/O runtime and cached per process. A payload whose schema is not cached yet reports “not ready”: the operator chain holds the batch (the source is paused by backpressure if needed) and replays the payload once the fetch lands — records are never dropped or duplicated, and the CPU-pinned pipeline threads never perform I/O. Ids the registry cannot serve are negatively cached (with a TTL) and handled by the deserializer’s ErrorPolicy like any other poison payload.

Schema evolution

Writer schemas come from the payload (registry id or fingerprint) or configuration; an optional reader_schema pins the shape records are resolved into, using Avro’s schema-resolution rules (field reordering, defaults, promotions, aliases). Registry schemas using references are not supported yet and are surfaced as unavailable.

Structs

AvroDeserializerBuilder

Builder produced from the opaque config section; hands out either the dynamically-typed or the serde-typed deserializer.

AvroSerdeDeserializer

Typed deserializer: decodes each datum into T via serde. The record type is plain Rust — no Avro types leak into the pipeline.

AvroSettings

The avro component configuration.

AvroValueDeserializer

Dynamically-typed deserializer: emits AvroValue records. Use when the schema is only known at runtime, or as the lower-allocation path — it decodes each datum exactly once. The typed AvroSerdeDeserializer keeps apache-avro types out of your pipeline, but it is not faster: it decodes to an AvroValue and then re-decodes that into T.

RegistrySection

Registry connection section.

SchemaSource

A schema provided inline or from a file — set exactly one field.

Enums

AvroConfigError

Construction errors.

AvroMode

Payload framing mode.

Functions

parse_confluent

Confluent framing: magic 0x00 + u32 big-endian schema id + datum.

parse_single_object

Single-object framing: magic 0xC3 0x01 + u64 little-endian Rabin fingerprint + datum.

Type Aliases

AvroValue

The generic Avro value type emitted by AvroValueDeserializer.