Struct ConnectorConfig

pub struct ConnectorConfig {
    pub transport: TransportConfig,
    pub format: Option<FormatConfig>,
    pub output_buffer_config: OutputBufferConfig,
    pub max_batch_size: u64,
    pub max_queued_records: u64,
    pub paused: bool,
}

A data connector’s configuration

Fields

transport: TransportConfig

Transport endpoint configuration.

format: Option<FormatConfig>

Parser configuration.

output_buffer_config: OutputBufferConfig

Output buffer configuration.

max_batch_size: u64

Maximum batch size, in records.

This is the maximum number of records to process in one batch through the circuit. The time and space cost of processing a batch is asymptotically superlinear in the size of the batch, but very small batches are less efficient due to constant factors.

This should usually be less than max_queued_records, to give the connector a round-trip time to restart and refill the buffer while batches are being processed.

Some input adapters might not honor this setting.

The default is 10,000.

max_queued_records: u64

Backpressure threshold.

Maximal number of records queued by the endpoint before the endpoint is paused by the backpressure mechanism.

For input endpoints, this setting bounds the number of records that have been received from the input transport but haven’t yet been consumed by the circuit since the circuit, since the circuit is still busy processing previous inputs.

For output endpoints, this setting bounds the number of records that have been produced by the circuit but not yet sent via the output transport endpoint nor stored in the output buffer (see enable_output_buffer).

Note that this is not a hard bound: there can be a small delay between the backpressure mechanism is triggered and the endpoint is paused, during which more data may be queued.

The default is 1 million.

paused: bool

Create connector in paused state.

The default is false.

Trait Implementations

impl Clone for ConnectorConfig

fn clone(&self) -> ConnectorConfig

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ConnectorConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for ConnectorConfig

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl PartialEq for ConnectorConfig

fn eq(&self, other: &ConnectorConfig) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for ConnectorConfig

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl<'__s> ToSchema<'__s> for ConnectorConfig

fn schema() -> (&'__s str, RefOr<Schema>)

Return a tuple of name and schema or reference to a schema that can be referenced by the name or inlined directly to responses, request bodies or parameters.

fn aliases() -> Vec<(&'__s str, Schema)>

Optional set of alias schemas for the ToSchema::schema.

impl Eq for ConnectorConfig

impl StructuralPartialEq for ConnectorConfig