Skip to main content

AsyncRecordSink

wp_connector_api

Trait AsyncRecordSink 

Source 
pub trait AsyncRecordSink {
    // Required methods
    fn sink_record<'life0, 'life1, 'async_trait>(
        &'life0 mut self,
        data: &'life1 DataRecord,
    ) -> Pin<Box<dyn Future<Output = SinkResult<()>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait;
    fn sink_records<'life0, 'async_trait>(
        &'life0 mut self,
        data: Vec<Arc<DataRecord>>,
    ) -> Pin<Box<dyn Future<Output = SinkResult<()>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait;

    // Provided method
    fn sink_records_with_meta<'life0, 'async_trait>(
        &'life0 mut self,
        meta: BatchMeta,
        data: Vec<Arc<DataRecord>>,
    ) -> Pin<Box<dyn Future<Output = SinkResult<()>> + Send + 'async_trait>>
       where Self: Send + 'async_trait,
             'life0: 'async_trait { ... }
}
Expand description

Trait for sinking structured records.

Provides methods for writing parsed, typed data records to a destination. Implementations should handle batching efficiently when sink_records is called.

Required Methods§

Source

fn sink_record<'life0, 'life1, 'async_trait>( &'life0 mut self, data: &'life1 DataRecord, ) -> Pin<Box<dyn Future<Output = SinkResult<()>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Write a single data record to the sink.

§Arguments
  • data - Reference to the record to write
Source

fn sink_records<'life0, 'async_trait>( &'life0 mut self, data: Vec<Arc<DataRecord>>, ) -> Pin<Box<dyn Future<Output = SinkResult<()>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait,

Write multiple data records in batch.

Implementations should optimize for batch writes when possible. The order of records in the vector should be preserved.

§Arguments
  • data - Vector of records wrapped in Arc for shared ownership

Provided Methods§

Source

fn sink_records_with_meta<'life0, 'async_trait>( &'life0 mut self, meta: BatchMeta, data: Vec<Arc<DataRecord>>, ) -> Pin<Box<dyn Future<Output = SinkResult<()>> + Send + 'async_trait>>
where Self: Send + 'async_trait, 'life0: 'async_trait,

Write multiple data records with batch-level metadata.

Default implementation ignores BatchMeta and falls back to sink_records, so existing connectors compile unchanged. Sinks that need the metadata should override this method.

§Implementation Requirements

Sink implementations SHOULD consume meta.oml_name according to their output format:

Sink 类型oml_name 用途
Arrow (arrow_framed)写入 frame header tag（[tag_len][tag][IPC]）
Arrow (arrow_ipc)忽略（裸 IPC Stream 无 tag 位置）
JSON / CSV（文本格式）作为 wp_oml_name 字段追加到每条记录中

当 meta.oml_name 为空时，所有 sink 保持原有行为（Arrow 使用 connector 配置的固定 tag，文本格式不追加字段）。

§Arguments
  • meta - Batch-level runtime metadata (e.g. OML output name)
  • data - Vector of records wrapped in Arc for shared ownership

Dyn Compatibility§

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§