scalo

There's plenty of sage advice about running services in production at scale -- config cascades, structured logging, secret masking, Prometheus, OpenTelemetry, health probes, backpressure, graceful shutdown -- but almost none of it as code you can just install and use.

This is that code.

scalo is an integrated, self-regulating runtime for hyperscale-grade data-plane services. Config, logging and metrics come as one pre-wired trinity -- global singletons you just use, no plumbing, no init dance. Everything else leans on that same integration: the config cascade flows straight into the CLI so run/version/config-check just work; the metrics and health wiring feed the K8s probe trinity; and the deployment contract generates your Helm, Dockerfile and Argo manifests from the config the app already declares.

Attach scalo to your service and a whole class of production pain -- the kind done wrong a hundred times elsewhere -- just goes away. Battle-tested, and almost no code on your side to do it properly. It's not a bag of utility functions you wire up yourself; it's the wiring, done right, for free.

scalo comes in two halves that share one set of conventions, idiomatic in each language. scalo-rs (this crate) is the data plane -- the Rust hot path where every microsecond and byte counts (cargo add scalo). scalo-py is the control plane -- orchestration, APIs and integration in Python (pip install scalo).

Opinionated about correctness -- backpressure, memory safety and the probe trinity are on by default. Unopinionated about your domain -- no web framework, no ORM, no enforced transport. Built as the foundation for PB/hr data services.

This module exists because of this -- https://www.youtube.com/watch?v=xE9W9Ghe4Jk -- but for the backend. And of course, no microservices.

Quick Start

[dependencies]
scalo = "2"

Default features: config, logger. Add the others you want explicitly.

use scalo::{config, logger, env};

fn main() -> anyhow::Result<()> {
    let environment = env::Environment::detect();
    logger::setup_default()?;
    config::setup(config::ConfigOptions {
        env_prefix: "MYAPP".into(),
        ..Default::default()
    })?;

    tracing::info!("Running in {environment:?}");
    Ok(())
}

Features

Pick the slice you need; pay only for what you use.

Native System Dependencies

This crate dynamically links against system C libraries for several features. Both build hosts and deployment targets need the appropriate packages.

Build Host (CI / Development)

For librdkafka-dev >= 2.12.1, add the Confluent APT repo. The suite below is bookworm, which is what a Debian trixie host uses - Confluent publishes no trixie suite and the bookworm .deb installs cleanly on trixie. On an Ubuntu 24.04 host use noble:

curl -fsSL https://packages.confluent.io/clients/deb/archive.key \
  | sudo gpg --dearmor -o /usr/share/keyrings/confluent-clients.gpg
echo "deb [signed-by=/usr/share/keyrings/confluent-clients.gpg] \
  https://packages.confluent.io/clients/deb bookworm main" \
  | sudo tee /etc/apt/sources.list.d/confluent-clients.list
sudo apt-get update
sudo apt-get install -y librdkafka-dev libssl-dev libsasl2-dev pkg-config

Deployment Host (Runtime)

The compiled binary links against .so files at runtime. Install the runtime packages (not -dev) on deployment hosts or in Docker images.

Only install what you use. Check the features your binary enables to determine which runtime packages are needed.

Docker Example

This is the shape the deployment feature's generator emits, minus the generated LABEL and APT blocks. The WORKDIR /app in the build stage is what makes /app/target/release/... resolvable from the runtime stage.

# Build stage
FROM rust:1 AS builder
WORKDIR /app
RUN apt-get update && apt-get install -y \
    pkg-config libssl-dev librdkafka-dev libgit2-dev libzstd-dev
COPY . .
RUN cargo build --release

# Runtime stage - the contract's base_image (default: the org base, currently
# Debian trixie slim)
FROM debian:trixie-slim AS runtime
RUN apt-get update && apt-get install -y --no-install-recommends \
    librdkafka1 libssl3t64 libgit2-1.9 libzstd1 ca-certificates curl \
    && rm -rf /var/lib/apt/lists/*

COPY --from=builder /app/target/release/myapp /usr/local/bin/myapp
RUN chmod +x /usr/local/bin/myapp

RUN useradd --create-home --uid 1000 appuser
USER appuser

EXPOSE 9090

HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
    CMD curl -sf http://localhost:9090/healthz > /dev/null || exit 1

ENTRYPOINT ["myapp"]

libgit2-1.9 and libssl3t64 are the trixie package names; on a noble base they are libgit2-1.7 and libssl3t64, on bookworm libgit2-1.5 and libssl3. The deployment feature works these out for you from the contract's release - see the release table in docs/deployment/native-deps.md, which also covers adding the Confluent APT repo to both stages for librdkafka1. The generator also drops any pre-existing UID 1000 account (ubuntu bases ship one) before creating appuser; trixie slim does not, so the example skips it. Note that Kubernetes ignores HEALTHCHECK - it is there for plain Docker and Compose. K8s uses the probe paths above.

Health Check Endpoints

For services deployed to Kubernetes, the paths are the K8s-standard ones. Two routers can serve them and they do NOT carry the same set, so the "served by" column is the one to read before pointing a probe or a scrape at a port:

/health/live and /health/ready stay as aliases for consumer probes written before the rename, on both routers.

The deployment contract's metrics_path defaults to /metrics, and the generated Helm chart puts the Prometheus scrape annotations on the contract's metrics_port. That only answers if the METRICS server is the thing listening on that port -- an app that stands up only the http-server router there will serve the two health paths and 404 the scrape.

/startupz exists for the K8s startupProbe: 503 {"status":"starting"} until the app calls MetricsManager::mark_started(), 200 {"status":"started"} after. It is deliberately separate from readiness - a startup probe gets a long timeout for slow starters, readiness does not. Point a startupProbe at /startupz on the metrics server, or at /healthz if you only have the http-server router.

Liveness MUST NEVER check downstream dependencies (a DB outage shouldn't restart your replicas). Readiness checks dependencies AND requires an explicit set_ready() call - cleared during graceful shutdown.

Self-regulation (default vertical scaling)

A scalo data-plane app regulates its own intake. Sized for steady state, a pod slows down or speeds up WITHIN itself first -- the default, fast, local response to a burst, a stalled upstream, or a transform that balloons memory. Only when that vertical headroom is exhausted does it escalate to horizontal scale (KEDA adding pods), driven by the same pressure signal. Memory is the hard, never-OOM authority; CPU is left to the kernel scheduler (CFS), which the byte-budget loop reads through longer process times. It is ON by default and opt-out via self_regulation.enabled = false. See docs/self-regulation.md.

Architecture

See docs/ for the full documentation index - docs/architecture.md for the module map and layering, and docs/core-pillars/config.md for the 7-layer config cascade reference.

License

Apache-2.0.

Related

• scalo-py -- sister library for Python services. Same opinions, same patterns, expressive Python ergonomics for control planes, APIs, and integration layers.