rust_with_kafka_tls/lib.rs
1//! # Rust Kafka Publisher and Subscriber Demo with Strimzi Kafka and Client mTLS for encryption in transit
2//!
3//! ## Sources
4//!
5//! This crate was built from these awesome repos:
6//!
7//! - Rust Consumer and Producer examples from [rdkafka](https://github.com/fede1024/rust-rdkafka) with [examples](https://github.com/fede1024/rust-rdkafka/tree/master/examples)
8//! - [Using your own CA and TLS Assets with Strimzi](https://github.com/scholzj/strimzi-custom-ca-test)
9//!
10//! ## Optional - Custom TLS Assets
11//!
12//! By default the ``./kubernetes/deploy.sh`` script will use the included tls assets in the repo: [./kubernetes/tls](https://github.com/jay-johnson/rust-with-strimzi-kafka-and-tls/tree/main/kubernetes/tls). Before going into production with these, please change these to your own to prevent security issues.
13//!
14//! If you want to use your own tls assets you can set these environment variables:
15//!
16//! - ``CA_FILE`` - path to your Certificate Authority (CA) file
17//! - ``CA_KEY_FILE`` - path to your CA key file
18//! - ``TLS_CHAIN_FILE`` - path to your tls server chain file (ordered by: cert then CA)
19//! - ``TLS_KEY_FILE`` - path to your tls server key file
20//!
21//! ```bash
22//! ./kubernetes/deploy.sh
23//! ```
24//!
25//! ## Verify Client mTLS
26//!
27//! Clients must provide the tls key, cert and CAfile for establishing a valid mutual tls connection.
28//!
29//! For local testing you will need to add these entries to your ``/etc/hosts`` or set up a real nameserver for dns:
30//!
31//! - ``cluster-0-broker-0.redten.io``
32//! - ``cluster-0-broker-1.redten.io``
33//! - ``cluster-0-broker-2.redten.io``
34//!
35//! As an example on the local loopback device:
36//!
37//! ```bash
38//! # /etc/hosts
39//! 127.0.0.1 cluster-0-broker-0.redten.io cluster-0-broker-1.redten.io cluster-0-broker-2.redten.io
40//! ```
41//!
42//! For users on minikube you can use ``minikube ip -p CLUSTERNAME`` to get the ip address:
43//!
44//! ```bash
45//! # /etc/hosts
46//! 192.168.49.2 cluster-0-broker-0.redten.io cluster-0-broker-1.redten.io cluster-0-broker-2.redten.io
47//! ```
48//!
49//! ```bash
50//! echo "ssl test" | openssl s_client -connect \
51//! cluster-0-broker-0.redten.io:32151 \
52//! -key ./kubernetes/tls/client-key.pem \
53//! -cert ./kubernetes/tls/client.pem \
54//! -CAfile ./kubernetes/tls/ca.pem \
55//! -verify_return_error \
56//! && echo "strimzi kafka cluster is working with self-signed tls assets!"
57//! ```
58//!
59//! ## Create Kafka Topic for Rust Messaging
60//!
61//! ```bash
62//! cat <<EOL | kubectl apply -n dev -f -
63//! apiVersion: kafka.strimzi.io/v1beta2
64//! kind: KafkaTopic
65//! metadata:
66//! name: testing
67//! labels:
68//! strimzi.io/cluster: "dev"
69//! spec:
70//! partitions: 3
71//! replicas: 3
72//! EOL
73//! ```
74//!
75//! ## Rust Messaging
76//!
77//! ### Set TLS Paths
78//!
79//! You can either copy the TLS assets into the ``./tls`` directory or export the environment variables:
80//!
81//! - ``KAFKA_TLS_CLIENT_CA`` - path to the Certificate Authority file
82//! - ``KAFKA_TLS_CLIENT_KEY`` - path to the server key file
83//! - ``KAFKA_TLS_CLIENT_CERT`` - path to the server certificate file
84//!
85//! ### Set Broker Addresses
86//!
87//! Export this environment variable to the correct broker fqdns and ports:
88//!
89//! - ``KAFKA_BROKERS`` - comma delimited list of kafka brokers (format: ``cluster-0-broker-0.redten.io:32151,cluster-0-broker-1.redten.io:32152,cluster-0-broker-2.redten.io:32153``)
90//!
91//! ### Start Consumer
92//!
93//! ```bash
94//! # export KAFKA_BROKERS=cluster-0-broker-0.redten.io:32151,cluster-0-broker-1.redten.io:32152,cluster-0-broker-2.redten.io:32153
95//! cargo build --example run-consumer
96//! export RUST_BACKTRACE=1
97//! export RUST_LOG=info
98//! ./target/debug/examples/run-consumer -b $KAFKA_BROKERS -g rust-consumer-testing -t testing
99//! ```
100//!
101//! ### Start Producer
102//!
103//! ```bash
104//! # export KAFKA_BROKERS=cluster-0-broker-0.redten.io:32151,cluster-0-broker-1.redten.io:32152,cluster-0-broker-2.redten.io:32153
105//! cargo build --example run-producer
106//! export RUST_BACKTRACE=1
107//! export RUST_LOG=info
108//! ./target/debug/examples/run-producer -b $KAFKA_BROKERS -t testing
109//! ```
110
111pub mod consume_and_print;
112pub mod custom_context;
113pub mod log_utils;
114pub mod publish_messages;