Struct aws_sdk_glue::model::KafkaStreamingSourceOptions

#[non_exhaustive]
pub struct KafkaStreamingSourceOptions { /* private fields */ }

Additional options for streaming.

Implementations

impl KafkaStreamingSourceOptions

pub fn bootstrap_servers(&self) -> Option<&str>

A list of bootstrap server URLs, for example, as b-1.vpc-test-2.o4q88o.c6.kafka.us-east-1.amazonaws.com:9094. This option must be specified in the API call or defined in the table metadata in the Data Catalog.

pub fn security_protocol(&self) -> Option<&str>

The protocol used to communicate with brokers. The possible values are "SSL" or "PLAINTEXT".

pub fn connection_name(&self) -> Option<&str>

The name of the connection.

pub fn topic_name(&self) -> Option<&str>

The topic name as specified in Apache Kafka. You must specify at least one of "topicName", "assign" or "subscribePattern".

pub fn assign(&self) -> Option<&str>

The specific TopicPartitions to consume. You must specify at least one of "topicName", "assign" or "subscribePattern".

pub fn subscribe_pattern(&self) -> Option<&str>

A Java regex string that identifies the topic list to subscribe to. You must specify at least one of "topicName", "assign" or "subscribePattern".

pub fn classification(&self) -> Option<&str>

An optional classification.

pub fn delimiter(&self) -> Option<&str>

Specifies the delimiter character.

pub fn starting_offsets(&self) -> Option<&str>

The starting position in the Kafka topic to read data from. The possible values are "earliest" or "latest". The default value is "latest".

pub fn ending_offsets(&self) -> Option<&str>

The end point when a batch query is ended. Possible values are either "latest" or a JSON string that specifies an ending offset for each TopicPartition.

pub fn poll_timeout_ms(&self) -> Option<i64>

The timeout in milliseconds to poll data from Kafka in Spark job executors. The default value is 512.

pub fn num_retries(&self) -> Option<i32>

The number of times to retry before failing to fetch Kafka offsets. The default value is 3.

pub fn retry_interval_ms(&self) -> Option<i64>

The time in milliseconds to wait before retrying to fetch Kafka offsets. The default value is 10.

pub fn max_offsets_per_trigger(&self) -> Option<i64>

The rate limit on the maximum number of offsets that are processed per trigger interval. The specified total number of offsets is proportionally split across topicPartitions of different volumes. The default value is null, which means that the consumer reads all offsets until the known latest offset.

pub fn min_partitions(&self) -> Option<i32>

The desired minimum number of partitions to read from Kafka. The default value is null, which means that the number of spark partitions is equal to the number of Kafka partitions.

impl KafkaStreamingSourceOptions

pub fn builder() -> Builder

Creates a new builder-style object to manufacture KafkaStreamingSourceOptions.

Trait Implementations

impl Clone for KafkaStreamingSourceOptions

fn clone(&self) -> KafkaStreamingSourceOptions

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for KafkaStreamingSourceOptions

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq<KafkaStreamingSourceOptions> for KafkaStreamingSourceOptions

fn eq(&self, other: &KafkaStreamingSourceOptions) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

This method tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for KafkaStreamingSourceOptions