Crate spate_avro 

Avro deserialization for the Spate framework.

Decodes bare Avro datums as carried by Kafka messages (Confluent wire format of magic byte + schema id + datum, Avro single-object encoding, or raw datums with a fixed schema) three ways: single-pass into your own serde types (AvroDatumDeserializer, the throughput path, with optional zero-copy borrowed records), two-pass into serde types with Avro schema resolution (AvroSerdeDeserializer), or into dynamically-typed AvroValue records (AvroValueDeserializer).

Never block the pipeline thread

Registry schemas are fetched by an asynchronous task on the I/O runtime and cached per process. A payload whose schema is not cached yet reports “not ready”: the operator chain holds the batch (the source is paused by backpressure if needed) and replays the payload once the fetch lands. Records are never dropped or duplicated, and the CPU-pinned pipeline threads never perform I/O. Ids the registry cannot serve are negatively cached (with a TTL) and handled by the deserializer’s ErrorPolicy like any other poison payload. A registry that rejects the credentials or the TLS handshake, or whose certificate fails verification, makes the next cache miss stop the pipeline, whatever the ErrorPolicy.

Schema evolution

Writer schemas come from the payload (registry id or fingerprint) or configuration; an optional reader_schema pins the shape records are resolved into, using Avro’s schema-resolution rules (field reordering, defaults, promotions). Registry schemas using references are not supported yet and are surfaced as unavailable.

A reader field alias is not applied. A reader field that renames a writer’s field and lists the old name in aliases fails every payload; rename with #[serde(alias)] on the record type instead. Record names are not compared during resolution, so a record-level alias has no effect either way.

Structs

AvroDatumDeserializer

Single-pass typed deserializer: decodes each datum directly into the record family’s type via serde, never materialising the intermediate crate::AvroValue tree the other two deserializers build.

AvroDeserializerBuilder

Builder produced from the opaque config section; hands out either the dynamically-typed or the serde-typed deserializer.

AvroSerdeDeserializer

Typed deserializer: decodes each datum into T via serde. The record type is plain Rust; no Avro types leak into the pipeline.

AvroSettings

The avro component configuration.

AvroValueDeserializer

Dynamically-typed deserializer: emits AvroValue records. Use when the schema is only known at runtime, in pipelines that inspect or route on structure they cannot name at compile time. When the record type is known, crate::AvroDatumDeserializer decodes it in a single pass without materialising the AvroValue tree at all, and is the faster choice.

RegistrySection

Registry connection section.

SchemaSource

A schema provided inline or from a file. Set exactly one field.

TlsSection

Trust settings for an https:// registry.

Enums

AvroConfigError

Construction errors.

AvroMode

Payload framing mode.

Functions

parse_confluent

Confluent framing: magic 0x00 + u32 big-endian schema id + datum.

parse_single_object

Single-object framing: magic 0xC3 0x01 + u64 little-endian Rabin fingerprint + datum.

Type Aliases

AvroValue

The generic Avro value type emitted by AvroValueDeserializer.