Crate nakadion

Nakadion

A client for the Nakadi Event Broker.

Nakadion uses the Subscription API of Nakadi.

Consuming

Nakadion supports two modes of consuming events. A sequuential one and a cuncurrent one.

Sequential Consumption

In this mode Nakadion will read a batch from Nakadi then call a handler on theese and afterwards try to commit the batch. This mode of operation is simple, straight forward and should be sufficient for most scenarios. No batches are buffered when consuming sequentially.

Concurrent Consumption

DO NOT USE FOR NOW

Configuration

Nakadion is configured by environment variables.

The environment variable NAKADION_SUBSCRIPTION_ID can be used to specify a subscription. The value can also be passed when creating the NakadiClient in case you want to cionsume multiple subscriptions.

Setting up the connector

• NAKADION_NAKADI_HOST: See ConnectorSettings::nakadi_host
• NAKADION_MAX_UNCOMMITED_EVENTS: See ConnectorSettings::max_uncommitted_events
• NAKADION_BATCH_LIMIT: See ConnectorSettings::batch_limit
• NAKADION_BATCH_FLUSH_TIMEOUT_SECS: See ConnectorSettings::batch_flush_timeout
• NAKADION_STREAM_TIMEOUT_SECS: See ConnectorSettings::stream_timeout
• NAKADION_STREAM_LIMIT: See ConnectorSettings::stream_limit
• NAKADION_STREAM_KEEP_ALIVE_LIMIT: See ConnectorSettings::stream_keep_alive_limit

Setting up the Sequential Worker

Just set NAKADION_USE_CONCURRENT_WORKER to false which is also the default.

Setting up the Concurent Worker:

Just set NAKADION_USE_CONCURRENT_WORKER to true.

Configure the worker:

• NAKADION_MAX_WORKERS: See ConcurrentWorkerSettings::max_workers
• NAKADION_WORKER_BUFFER_SIZE: See ConcurrentWorkerSettings::worker_buffer_size

In this mode Nakadion will spawn a number of worker threads and distribute work among them based on the partion id of a batch. The workers are not dedidacted to a partition. Work is rather distributed based on a hash of the partition id.

Performance

This library is not meant to be used in a high performance scenario. It uses synchronous IO.

Documentation

Documenatation can be found at docs.rs

License

Nakadion is distributed under the terms of both the MIT license and the Apache License (Version 2.0).

See LICENSE-APACHE and LICENSE-MIT for details.#![recursion_limit = "1024"]

Modules

metrics
worker	The components to consume from the stream.

Structs

BatchInfo	Information on a current batch. This might be useful for a Handler that wants to do checkpointing on its own.
ClientError	The Error type.
ConnectorSettings	Settings for establishing a connection to Nakadi.
ConnectorSettingsBuilder	Builder for ConnectorSettings.
Cursor	A Cursor describes a position in the stream. The cursor is used for checkpointing.
EventType	The Nakadi Event Type. Similiar to a topic.
EventTypeInfo	An EventType can be published on multiple partitions.
HyperClientConnector	A NakadiConnector using Hyper for dispatching requests.
NakadiClient	The client to consume events from Nakadi
PartitionId	A partition id that comes with a Cursor
PartitionInfo	Information on a partition
StreamId	A StreamId identifies a subscription. It must be provided for checkpointing with a Cursor.
StreamInfo	A stream can provide multiple EventTypes where each of them can have its own partitioning setup.
SubscriptionId	A SubscriptionId is used to guaratee a continous flow of events for a client.
Token	A token used for authentication against Nakadi.
TokenError	The Error type.

Enums

AfterBatchAction	Describes what to do after a batch has been processed.
ClientErrorKind	The kind of an error.
TokenErrorKind	The kind of an error.

Traits

Checkpoints	Checkpoints cursors
ClientResultExt	Additional methods for Result, for easy interaction with this crate.
Handler	Handles batches of events received from Nakadi.
NakadiConnector	Connects to Nakadi for checkpointing and consuming events.
ProvidesStreamInfo
ProvidesToken	Provides a Token.
ReadsStream	Connects to Nakadi and reads the stream-
TokenResultExt	Additional methods for Result, for easy interaction with this crate.

Type Definitions

ClientResult	Convenient wrapper around std::Result.
TokenResult	Convenient wrapper around std::Result.