# kiaproxy
Kiaproxy is a minimalistic and high performance TCP load balancer for the purpose of high availability.
The source code is minimalistic and understandable, fast compiling, built on [Tokio](https://crates.io/crates/tokio) async-io, and uses very few system resources.
It can run in a container, VM, or baremetal, and should compile to many OS targets.
Originally there was a single algorithm for backend selection (ordered), which is an ordered selection based on first available. As of version `0.2.0`, we now also have "round-robin', "least-conn", and "sticky-clients" algorithms.
The configuration comes from required environment variables. Set the SERVERS variable
to the backend endpoints to route traffic to, change them to fit the needs. DNS names or IPs can be used, but a colon and the port is required.
Set the LISTENER to the endpoint kiaproxy is to listen on, typically `0.0.0.0:443` would be used for TLS-passthrough, and is how the container port is exposed.
There is also an optional environment variable for LB_ALGO. If LB_ALGO is not set, then kiaproxy defaults to "ordered".
Example showing the environment variables, for running the program binary directly in a shell, such as on the command line, in systemd, or an rc service:
```
export SERVERS=192.168.1.33:443,192.168.1.34:443,192.168.1.55:443 &&
export LISTENER=0.0.0.0:443 &&
export LB_ALGO=round-robin
kiaproxy
```
### The "ordered" algorithm
The servers list will default to the first item and check each server starting from the first item.
The first server that responds over TCP for a client request is the one selected for use for that client request.
### The "round-robin" algorithm
The server list will be cycled through, routing requests to the next one on the list and then starting again with the first one after each server has had a request.
### The "least-conn" algorithm
The servers with the least amount of connections are prioritized. If it is a tie, then round-robin is used.
### The "sticky-clients" algorithm
The in-RAM connection tracking checks client source IP if existing entries are found for a previous connection to a backend, the same backend is preferred.
### health checks
Unlike many load balancers, the kiaproxy "health check" is done per request. There is no UP/DOWN shared state or control loop of health checks,
each request is only connected to a server that reports up in-flight and health checks are done per client request.
Kiaproxy is a TCP load balancer and can handle TLS passthrough (SSL/TLS/HTTPS backends), HTTP backends, and TCP/raw backends.
The connection is a bidirectional stream that works well for many types of network connections.
If no servers are available, the first selection will be tried (version 0.1.3 tries 28 times, version 0.1.2 tries 9 times), sleeping for 1 second between each attempt, before disconnecting the client.
If a server is selected for use because it is online and then goes offline, the connection will retry 28 times (version 0.1.3 has 28 retries, version 0.1.2 has 9 retries), sleeping for 1 second between each try.
When using the "ordered" algorithm, it is better to keep online servers near the "front" (left) of the server list - having many offline servers on the left still works, but the greater number of offline servers before reaching an online server,
the longer the connection build takes. That said, the connection build is still very fast in most uses, even with the health checks happening before the stream is established.
The typical use of kiaproxy is to provide one or more failover endpoints, so that if the primary endpoint is down, the secondary is used, etc etc.
Kiaproxy is especially useful for situations like maintenance or avoiding some types of outages.
With the expansion of algorithms in version `0.2.0`, kiaproxy can also be used to distribute (fan-out) traffic.
## Installation
Kiaproxy is available on [github](https://github.com/jpegleg/kiaproxy/), [crates.io](https://crates.io/crates/kiaproxy), and [docker hub](https://hub.docker.com/r/carefuldata/kiaproxy).
The container image is very small and hardened, with only a single statically linked Rust binary added to a minimized container "scratch" image.
Here is an example of pulling the image from docker hub and running via Podman or Docker:
```
podman pull docker.io/carefuldata/kiaproxy:latest
podman run -e SERVERS=192.168.1.120:443,192.168.1.121:443,192.168.1.122:443 -e LISTENER=0.0.0.0:443 -d -it --network=host carefuldata/kiaproxy
```
_Note that the default container image is set to use (EXPOSE) the 443 port in the container image, so we expect to use the 443 port for the listener when using the container but it still needs to be set as an environment variable.
The servers can use any ports, ip addresses, or DNS names in the container version._
Installing via Cargo:
```
cargo install kiaproxy
```
Kiaproxy can also be compiled from source or installed from precompiled release binaries via github.
Kiaproxy works well in Kubernetes, too, just specify the environment variables in the manifest.
This is a simplistic manifest example, just to show the general concept. There are of course many more advanced or refined
manifest possibilities.
```
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: kiaproxy
labels:
app.kubernetes.io/name: kiaproxy
spec:
replicas: 1
selector:
matchLabels:
app: kiaproxy
template:
metadata:
labels:
app: kiaproxy
app.kubernetes.io/name: kiaproxy
spec:
containers:
- name: kiaproxy
image: "carefuldata/kiaproxy:latest"
ports:
- name: tls-passthrough
containerPort: 443
env:
- name: LISTENER
value: "0.0.0.0:443"
- name: SERVERS
value: "192.168.1.120:443,192.168.1.121:443,192.168.1.122:443"
...
```
_Tip: add restrictions to deny disk access to the container if you like, because kiaproxy does not need the disk at all._
```
#include <tunables/global>
profile k8s-apparmor-deny-write flags=(attach_disconnected) {
#include <abstractions/base>
file,
# Deny all file writes.
deny /** w,
}
```
You can create your own container image easily as well. This example shows building a new image with a different exposed port set to 5000 and
is assuming a musl statically linked binary is already in $PWD for the image build. Compile kiaproxy on Alpine Linux, or extract the existing
one from the public container image, or download one from github, to get such a binary. The compile can obviously be added to the Dockefile
in an earlier step, or compiled in a dynamically linked way and used in an image with the right C libraries for your target.
```
FROM scratch
COPY ./kiaproxy /kiaproxy
EXPOSE 5000
CMD ["/kiaproxy"]
```
#### Non-features
Kiaproxy is so simple that it is easy to adapt the source code to your needs, but this program is unlikely to add many features.
This decision is in order to keep the program small, few dependencies, extremely light on system resources, and purpose built.
The following are _not_ features of kiaproxy:
- hot loading of configuration values
- TLS termination
- UDP support
- wasm32-unknown-unknown compile target
The choice to use environment variables also came from the desire to further reduce dependencies, size, and syscalls.
Also see [kiagateway](https://github.com/jpegleg/kiagateway/) regarding another small proxy service, kiagateway performs TCP domain-based routing.
## So, what about HA of kiaproxy itself?
Kiaproxy eliminates points of failure for the endpoints it proxies, but in order for kiaproxy itself to be highly available, we need a second kiaproxy instance, ideally on separate physical hardware.
The cheap and easy way is to have two different computers running kiaproxy (on separate hardware) and have DNS records for both, however that can still lead to outages.
A better solution is to use [GSLB](https://www.ibm.com/think/topics/global-server-load-balancing), and/or something like [CARP](https://www.openbsd.org/faq/pf/carp.html),
selecting which kiaproxy server to use in order to minimize downtime for kiaproxy itself.
<b>Important note for running kiaproxy on OpenBSD:</b> The file limits (the important default to change is defined in /etc/login.conf) should be raised from the OpenBSD defaults, otherwise a DoS condition is possible.
The other limit on OpenBSD is from (sysctl kern.maxfiles), which is the global limit that is usually more reasonable. I would ensure kiaproxy can get over 4,000 files, and wouldn't feel bad about going higher, especially
if internet facing. Further, kiaproxy version before `0.2.0` do not have complete BSD POSIX network error handling, so denial of service can be trivial unless carefully managed. It is better to use version 0.2.0, especially when running kiaproxy on a *BSD operating system.
Kiaproxy might be used _in front_ of Kubernetes clusters, but can be run within Kubernetes clusters, or maybe in it's own dedicated "load balancer cluster", etc etc.
## Project promises
This project will never use AI-slop. All code is reviewed, tested, and implemented by a human expert. This repository and the crates.io repository are carefully managed and protected.
This project will be maintained as best as is reasonable.