Crate redpanda_transform_sdk

Redpanda Data Transforms Rust library.

Data transforms let you run common data streaming tasks, like filtering, scrubbing, and transcoding, within Redpanda. For example, you may have consumers that require you to redact credit card numbers or convert JSON to Avro.

Data transforms use a WebAssembly (Wasm) engine inside a Redpanda broker. Thus, transforms must compile to WebAssembly via --target=wasm32-wasi. A Wasm function acts on a single record in an input topic. You can develop and manage data transforms with rpk transform commands.

This crate provides a framework for writing transforms.

on_record_written: Transforms individual records after they have been written to an input topic. Written records are output to the destination topic.

Structs

• BorrowedHeader
  A zero-copy BorrowedRecord header.
• BorrowedRecord
  A zero-copy representation of a Record within Redpanda.
• Record
  A record in Redpanda.
• RecordHeader
  A key value pair attached to a Record.
• RecordWriter
  A struct that writes transformed records to the output topic..
• WriteEvent
  An event generated after a write event within the broker.
• WriteOptions
  Allows you to customize a RecordWriter’s write.
• WrittenRecord
  A written Record within Redpanda.

Enums

• WriteError
  An error that can occur when writing records to the output topic.

Traits

• RecordSink
  An internal trait that can receive a stream of records and output them a destination topic.

Functions

• on_record_written
  Register a callback to be fired when a record is written to the input topic.