Trait nakadion::api::SubscriptionStreamApi [−][src]
pub trait SubscriptionStreamApi {
fn request_stream<T: Into<FlowId>>(
&self,
subscription_id: SubscriptionId,
parameters: &StreamParameters,
flow_id: T
) -> ApiFuture<'_, SubscriptionStreamChunks>;
}Required methods
fn request_stream<T: Into<FlowId>>(
&self,
subscription_id: SubscriptionId,
parameters: &StreamParameters,
flow_id: T
) -> ApiFuture<'_, SubscriptionStreamChunks>
fn request_stream<T: Into<FlowId>>(
&self,
subscription_id: SubscriptionId,
parameters: &StreamParameters,
flow_id: T
) -> ApiFuture<'_, SubscriptionStreamChunks>
Starts a new stream for reading events from this subscription.
Starts a new stream for reading events from this subscription. The minimal consumption unit is a partition, so it is possible to start as many streams as the total number of partitions in event-types of this subscription. The position of the consumption is managed by Nakadi. The client is required to commit the cursors he gets in a stream.
If you create a stream without specifying the partitions to read from - Nakadi will automatically assign partitions to this new stream. By default Nakadi distributes partitions among clients trying to give an equal number of partitions to each client (the amount of data is not considered). This is default and the most common way to use streaming endpoint.
It is also possible to directly request specific partitions to be delivered within the stream. If these partitions are already consumed by another stream of this subscription - Nakadi will trigger a rebalance that will assign these partitions to the new stream. The request will fail if user directly requests partitions that are already requested directly by another active stream of this subscription. The overall picture will be the following: streams which directly requested specific partitions will consume from them; streams that didn’t specify which partitions to consume will consume partitions that left - Nakadi will autobalance free partitions among these streams (balancing happens by number of partitions).
Specifying partitions to consume is not a trivial way to consume as it will require additional coordination effort from the client application, that’s why it should only be used if such way of consumption should be implemented due to some specific requirements.
Also, when using streams with directly assigned partitions, it is the user’s responsibility to detect, and react to, changes in the number of partitions in the subscription (following the re-partitioning of an event type). Using the GET /subscriptions/{subscription_id}/stats endpoint can be helpful.
See also Nakadi Manual