Crate redpanda_transform_sdk

Redpanda Data Transforms Rust library.

Data transforms let you run common data streaming tasks, like filtering, scrubbing, and transcoding, within Redpanda. For example, you may have consumers that require you to redact credit card numbers or convert JSON to Avro.

Data transforms use a WebAssembly (Wasm) engine inside a Redpanda broker. Thus, transforms must compile to WebAssembly via --target=wasm32-wasi. A Wasm function acts on a single record in an input topic. You can develop and manage data transforms with rpk transform commands.

This crate provides a framework for writing transforms.

on_record_written: Transforms individual records after they have been written to an input topic. Written records are output to the destination topic.

Structs

BorrowedHeader

A zero-copy BorrowedRecord header.

BorrowedRecord

A zero-copy representation of a Record within Redpanda.

Record

A record in Redpanda.

RecordHeader

A key value pair attached to a Record.

RecordWriter

A struct that writes transformed records to the output topic..

WriteEvent

An event generated after a write event within the broker.

WriteOptions

Allows you to customize a RecordWriter’s write.

WrittenRecord

A written Record within Redpanda.

Enums

WriteError

An error that can occur when writing records to the output topic.

Traits

RecordSink

An internal trait that can receive a stream of records and output them a destination topic.

Functions

on_record_written

Register a callback to be fired when a record is written to the input topic.