Struct nakadion::components::connector::StreamParameters

pub struct StreamParameters {
    pub partitions: Vec<EventTypePartition, Global>,
    pub max_uncommitted_events: Option<StreamMaxUncommittedEvents>,
    pub batch_limit: Option<StreamBatchLimit>,
    pub stream_limit: Option<StreamLimit>,
    pub batch_flush_timeout_secs: Option<StreamBatchFlushTimeoutSecs>,
    pub batch_timespan_secs: Option<StreamBatchTimespanSecs>,
    pub stream_timeout_secs: Option<StreamTimeoutSecs>,
    pub commit_timeout_secs: Option<StreamCommitTimeoutSecs>,
}

Parameters for starting a new stream on a subscription

See also Nakadi Manual

Environment

When initialized/updated from the environment the following environment variable are used which by are by default prefixed with “NAKADION_” or a custom prefix “_”:

• “STREAM_MAX_UNCOMMITTED_EVENTS”
• “STREAM_BATCH_LIMIT”
• “STREAM_LIMIT”
• “STREAM_BATCH_FLUSH_TIMEOUT_SECS”
• “STREAM_BATCH_TIMESPAN_SECS”
• “STREAM_TIMEOUT_SECS”
• “STREAM_COMMIT_TIMEOUT_SECS”

Fields

partitions: Vec<EventTypePartition, Global>

List of partitions to read from in this stream. If absent or empty - then the partitions will be automatically assigned by Nakadi.

max_uncommitted_events: Option<StreamMaxUncommittedEvents>

The maximum number of uncommitted events that Nakadi will stream before pausing the stream. When in paused state and commit comes - the stream will resume.

batch_limit: Option<StreamBatchLimit>

Maximum number of Events in each chunk (and therefore per partition) of the stream.

stream_limit: Option<StreamLimit>

Maximum number of Events in this stream (over all partitions being streamed in this connection)

batch_flush_timeout_secs: Option<StreamBatchFlushTimeoutSecs>

Maximum time in seconds to wait for the flushing of each chunk (per partition).

batch_timespan_secs: Option<StreamBatchTimespanSecs>

Useful for batching events based on their received_at timestamp. For example, if batch_timespan is 5 seconds then Nakadi would flush a batch as soon as the difference in time between the first and the last event in the batch exceeds 5 seconds. It waits for an event outside of the window to signal the closure of a batch.

stream_timeout_secs: Option<StreamTimeoutSecs>

Maximum time in seconds a stream will live before connection is closed by the server.

If 0 or unspecified will stream for 1h ±10min.

commit_timeout_secs: Option<StreamCommitTimeoutSecs>

Maximum amount of seconds that Nakadi will be waiting for commit after sending a batch to a client. In case if commit does not come within this timeout, Nakadi will initialize stream termination, no new data will be sent. Partitions from this stream will be assigned to other streams. Setting commit_timeout to 0 is equal to setting it to the maximum allowed value - 60 seconds.

Implementations

impl StreamParameters

pub fn from_env() -> Result<StreamParameters, Error>

Initializes all fields from environment variables prefixed with “NAKADION_”

pub fn from_env_prefixed<T>(prefix: T) -> Result<StreamParameters, Error> where
    T: AsRef<str>, 

Initializes all fields from environment variables prefixed with “[prefix]_”

The underscore is omitted if prefix is empty

pub fn from_env_type_names() -> Result<StreamParameters, Error>

Initializes all fields from environment variables without any prefix

pub fn fill_from_env(&mut self) -> Result<(), Error>

Updates all not yet set fields from environment variables prefixed with “NAKADION_”

pub fn fill_from_env_type_names(&mut self) -> Result<(), Error>

Updates all not yet set fields from environment variables without any prefix

pub fn fill_from_env_prefixed<T>(&mut self, prefix: T) -> Result<(), Error> where
    T: AsRef<str>, 

Updates all not yet set fields from environment variables prefixed with “[prefix]_”

The underscore is omitted if prefix is empty

pub fn apply_defaults(&mut self)

pub fn partitions(
    self,
    partitions: Vec<EventTypePartition, Global>
) -> StreamParameters

List of partitions to read from in this stream. If absent or empty - then the partitions will be automatically assigned by Nakadi.

pub fn max_uncommitted_events<T>(self, value: T) -> StreamParameters where
    T: Into<StreamMaxUncommittedEvents>, 

The maximum number of uncommitted events that Nakadi will stream before pausing the stream.

When in paused state and commit comes - the stream will resume.

pub fn batch_limit<T>(self, value: T) -> StreamParameters where
    T: Into<StreamBatchLimit>, 

Maximum number of Events in each chunk (and therefore per partition) of the stream.

pub fn stream_limit<T>(self, value: T) -> StreamParameters where
    T: Into<StreamLimit>, 

Maximum number of Events in this stream (over all partitions being streamed in this connection)

pub fn batch_flush_timeout_secs<T>(self, value: T) -> StreamParameters where
    T: Into<StreamBatchFlushTimeoutSecs>, 

Maximum time in seconds to wait for the flushing of each chunk (per partition).

pub fn batch_timespan_secs<T>(self, value: T) -> StreamParameters where
    T: Into<StreamBatchTimespanSecs>, 

Useful for batching events based on their received_at timestamp.

For example, if batch_timespan is 5 seconds then Nakadi would flush a batch as soon as the difference in time between the first and the last event in the batch exceeds 5 seconds. It waits for an event outside of the window to signal the closure of a batch.

pub fn stream_timeout_secs<T>(self, value: T) -> StreamParameters where
    T: Into<StreamTimeoutSecs>, 

Maximum time in seconds a stream will live before connection is closed by the server..

pub fn commit_timeout_secs<T>(self, value: T) -> StreamParameters where
    T: Into<StreamCommitTimeoutSecs>, 

Maximum amount of seconds that Nakadi will be waiting for commit after sending a batch to a client.

In case if commit does not come within this timeout, Nakadi will initialize stream termination, no new data will be sent. Partitions from this stream will be assigned to other streams. Setting commit_timeout to 0 is equal to setting it to the maximum allowed value - 60 seconds.

pub fn effective_commit_timeout_secs(&self) -> u32

Returns the configured value or the Nakadi default

pub fn effective_max_uncommitted_events(&self) -> u32

Returns the configured value or the Nakadi default

pub fn effective_batch_limit(&self) -> u32

Returns the configured value or the Nakadi default

Trait Implementations

impl Clone for StreamParameters

pub fn clone(&self) -> StreamParameters

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for StreamParameters

pub fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Default for StreamParameters

pub fn default() -> StreamParameters

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for StreamParameters

pub fn deserialize<__D>(
    __deserializer: __D
) -> Result<StreamParameters, <__D as Deserializer<'de>>::Error> where
    __D: Deserializer<'de>, 

Deserialize this value from the given Serde deserializer.

impl Serialize for StreamParameters

pub fn serialize<__S>(
    &self,
    __serializer: __S
) -> Result<<__S as Serializer>::Ok, <__S as Serializer>::Error> where
    __S: Serializer, 

Serialize this value into the given Serde serializer.