Trait Postprocessor 

pub trait Postprocessor: Send + Sync {
    // Required method
    fn fork(&self) -> Box<dyn Postprocessor>;

    // Provided methods
    fn batch_start(&mut self, _step: Step, _batch_type: OutputBatchType) { ... }
    fn push_buffer(&mut self, buffer: &[u8]) -> AnyResult<Vec<u8>> { ... }
    fn push_key(
        &mut self,
        key: Option<&[u8]>,
        val: Option<&[u8]>,
        headers: &[(&str, Option<&[u8]>)],
    ) -> AnyResult<(Option<Vec<u8>>, Option<Vec<u8>>, Vec<(String, Option<Vec<u8>>)>)> { ... }
    fn batch_end(&mut self) { ... }
    fn memory(&self) -> usize { ... }
}

Trait for postprocessing encoded output data before transmission.

A postprocessor sits between the encoder and the transport. The user has to implement only one of the two possible APIs: push_buffer and push_key, depending on the encoder used: Keyed mode: is used encoders designed to work with message-based transports such as Kafka, where every message can have (key, value, header) components, and where the output is a stream of (key, value, headers) tuples. These encoder require the push_key method. Buffer mode: encoders where the output is simply a stream of buffers, where each buffer can contain an encoded representation of one or more output records. These encoders require the push_buffer method.

Required Methods

fn fork(&self) -> Box<dyn Postprocessor>

Create a new postprocessor with the same configuration as self.

Used when multiple parallel output pipelines are needed.

Provided Methods

fn batch_start(&mut self, _step: Step, _batch_type: OutputBatchType)

Called once for every output batch produced by the pipeline before any records are pushed to the postprocessor.

Arguments

• step — a monotonically-increasing sequence number assigned to this batch. Fault-tolerant endpoints use this to detect and discard duplicate output. Non-fault-tolerant postprocessors may ignore it.

• batch_type — indicates whether this batch is:

  • OutputBatchType::Delta: an incremental update (inserts and deletes since the last step).
  • OutputBatchType::Snapshot: a full snapshot of the materialized view at this point in time.

The default implementation is a no-op.

fn push_buffer(&mut self, buffer: &[u8]) -> AnyResult<Vec<u8>>

Transform a serialized buffer (buffer mode).

Called for each output chunk produced by the encoder. There can be many calls to this method between batch_start and batch_end notifications.

Arguments

• buffer — the raw bytes produced by the encoder.

Returns

The transformed byte buffer on success. On error the affected records are dropped and the error is reported to the controller; processing of subsequent records continues normally.

The default implementation returns the data unchanged.

fn push_key( &mut self, key: Option<&[u8]>, val: Option<&[u8]>, headers: &[(&str, Option<&[u8]>)], ) -> AnyResult<(Option<Vec<u8>>, Option<Vec<u8>>, Vec<(String, Option<Vec<u8>>)>)>

Transform a key/value/headers record (keyed mode).

Called for each key/value update generated by the encoder.

Arguments

• key — serialized key component of the message, or None if the key is absent.

• val — serialized value bytes, or None if the value is absent.

• headers — a slice of (name, value) pairs. Each header value is an optional byte slice; None means the header is present but has no value.

Returns

A tuple (key, val, headers) with the same shape as the arguments but owning the transformed bytes. On error the affected records are dropped and the error is reported to the controller; processing continues.

The default implementation returns the data unchanged.

fn batch_end(&mut self)

Called once at the end of each output batch. The default is a no-op.

fn memory(&self) -> usize

Returns the approximate amount of memory owned by this postprocessor.

The default returns 0; override when the implementation holds a significant internal buffer.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors