Kalix — Declarative Kalman Filter with Language-Agnostic CLI Bridge
Mission
Kalix is a production-grade Kalman filter library in Rust with a CLI entry point
driven via stdin/stdout JSON — callable from any language that can spawn a process
(Python, TypeScript, Ruby, Go, shell, etc.). The crate name is kalix throughout —
in Cargo.toml, the binary name, and all documentation.
The filter is declaratively configured via TOML — the user writes dynamics as symbolic expressions, and the system automatically derives the F (transition) and H (observation) matrices, detects linearity, and selects the appropriate filter variant (linear KF or EKF).
The CLI supports two modes: live (low-latency streaming, minimal output) and backtest (full audit trail, named state fields, complete covariance matrices). Both modes read the same input format; output verbosity differs.
Repository Layout
kalman/
├── Cargo.toml
├── README.md
├── src/
│ ├── main.rs # CLI entry point
│ ├── lib.rs # Public library surface
│ ├── config.rs # TOML config parsing and validation
│ ├── expr/
│ │ ├── mod.rs
│ │ ├── ast.rs # Expression AST nodes
│ │ ├── parser.rs # String → AST parser
│ │ ├── diff.rs # Symbolic differentiation
│ │ └── eval.rs # AST evaluator (numeric, given variable bindings)
│ ├── filter/
│ │ ├── mod.rs
│ │ ├── linear.rs # Standard Kalman filter
│ │ ├── ekf.rs # Extended Kalman filter
│ │ └── traits.rs # KalmanFilter trait
│ ├── io/
│ │ ├── mod.rs
│ │ ├── input.rs # Input message deserialisation
│ │ └── output.rs # Output message serialisation (live vs backtest)
│ ├── matrix.rs # Matrix construction from derived Jacobians
│ └── log.rs # Structured JSON tracing setup
├── configs/
│ ├── trend_no_accel.toml
│ ├── trend_with_accel.toml
│ ├── constant_velocity.toml
│ └── pendulum.toml
└── tests/
├── test_expr.rs
├── test_diff.rs
├── test_config.rs
├── test_linear_kf.rs
└── test_ekf.rs
Dependencies (Cargo.toml)
[]
= "kalix"
= "0.2.1"
= "2021"
= "Declarative Kalman filtering from dynamics expressions. Write the physics, derive the filter."
= "MIT"
[[]]
= "kalix"
= "src/main.rs"
[]
= { = "0.33", = ["serde-serialize"] }
= { = "1", = ["derive"] }
= "1"
= "0.8"
= "0.1"
= { = "0.3", = ["json", "env-filter"] }
= "1"
= "1"
[]
= "0.5"
Do not use external expression or CAS crates. Implement the AST, parser, and symbolic differentiator from scratch — the supported operation set is deliberately small.
Expression Language
Expressions are polynomial in state variables and the special token dt. Supported:
| Syntax | Meaning |
|---|---|
pos, vel, x |
State variable reference |
dt |
Timestep — a runtime parameter, not a state var |
3.14, 0.5 |
Numeric literal |
a + b |
Addition |
a - b |
Subtraction |
a * b |
Multiplication |
a / b |
Division — right-hand side must be a literal |
a ^ 2 |
Integer power (positive integers only) |
sin(expr) |
Sine |
cos(expr) |
Cosine |
log(expr) |
Natural logarithm |
exp(expr) |
Exponential (e^expr) |
(expr) |
Grouping |
No division by state variables — the RHS of / must be a numeric literal.
Unsupported functions return a descriptive parse error.
AST Definition
Symbolic Differentiation Rules
Implement diff(expr: &Expr, var: &str) -> Expr:
d/dx(lit) = 0d/dx(x) = 1,d/dx(y) = 0fory != xd/dx(dt) = 0— dt is a parameter, not a state variable- Standard sum, difference, product, and power rules
- Chain rule for transcendental functions:
d/dx(sin(u)) = cos(u) * u',d/dx(cos(u)) = -sin(u) * u'd/dx(log(u)) = u' / u,d/dx(exp(u)) = exp(u) * u' - After differentiation, run a simplification pass:
0 + x -> x,x + 0 -> x,1 * x -> x,x * 1 -> x,0 * x -> 0,x * 0 -> 0,x ^ 1 -> x,x ^ 0 -> Lit(1)
Linearity Detection
After computing all partial derivatives df_i/dx_j, check whether any resulting
expression still contains a Var node whose name is a state variable (not dt).
If any partial is nonlinear -> EKF. Otherwise -> linear KF, F built once at startup.
TOML Config Format
[]
= "trend_with_accel"
= "Position/velocity/acceleration trend follower"
[]
= ["pos", "vel", "acc"]
[]
# One entry per state variable — how it evolves each timestep
= "pos + vel*dt + 0.5*acc*dt^2"
= "vel + acc*dt"
= "acc"
[]
# Names and expressions for each measurement channel
= ["z"]
= ["pos"] # z measures pos directly
[]
= [[0.01, 0, 0], [0, 0.01, 0], [0, 0, 0.01]] # Q — n x n
= [[1.0]] # R — m x m
[]
= [0.0, 0.0, 0.0]
= [[1, 0, 0], [0, 1, 0], [0, 0, 1]]
Config Validation (fail loudly at load time)
[dynamics]must have exactly one entry per state variable, in the declared order- Every variable in a dynamics expression must be in
[state].variablesor bedt - Every observation expression must reference only state variables (not
dt) - F ->
n x n, H ->m x n, Q ->n x n, R ->m x m— dimension mismatches are errors - Q and R diagonal entries must all be >= 0, with at least one > 0 each
KalmanFilter Trait
CLI
Invocation
# Live mode — reads from stdin, minimal output
# Backtest mode — reads from stdin, full audit output
# Backtest from file (--input implies --mode backtest)
# --input combined with --mode live is a validation error — exit 1
Startup Event (stdout, both modes)
Emitted once after successful config load, before reading any input:
For EKF: "variant": "ekf" and F is omitted (recomputed per step).
Input Format
One JSON object per line. Both modes share the same input format.
Normal step — observation available:
Predict-only step — sensor dropout or missing bar:
t is informational only — the filter does not infer dt from timestamps. Callers must
supply dt explicitly to handle irregular bars (weekends, missing ticks) correctly.
Input Validation
Validate every incoming message before processing. On error, emit a structured JSON line
to stderr and apply the --on-error policy (default skip; halt exits with code 1):
| Condition | Error message |
|---|---|
dt <= 0 |
"invalid dt: must be positive, got {dt}" |
dt is NaN or inf |
"invalid dt: non-finite value" |
wrong z length |
"z has {n} elements, expected {m}" |
| unparseable JSON | "malformed input: {reason}" |
Output Format
Live Mode
One compact JSON line per input line. State named using [state].variables:
For predict-only steps, shape is identical with "predict_only": true added:
Backtest Mode
One JSON object per input line. Full covariance matrices and named fields throughout.
P is a 2D array; variable order matches [state].variables.
Residual is named using [observation].variables.
For predict-only steps in backtest mode, "update" is omitted and "predict_only": true
is added at the top level.
Summary Event (backtest mode only)
Emitted to stdout when stdin reaches EOF or the input file is exhausted:
Logging (stderr)
All diagnostic output goes to stderr as structured JSON via tracing +
tracing-subscriber with the JSON formatter. stdout is the data channel exclusively.
Control verbosity with RUST_LOG:
RUST_LOG=info — startup summary only:
RUST_LOG=debug — per-step internals:
EKF additionally logs the Jacobian evaluated at the current state each step at DEBUG level.
Example Configs
configs/trend_no_accel.toml
[]
= "trend_no_accel"
[]
= ["pos", "vel"]
[]
= "pos + vel*dt"
= "vel"
[]
= ["z"]
= ["pos"]
[]
= [[0.01, 0], [0, 0.01]]
= [[1.0]]
[]
= [0.0, 0.0]
= [[1, 0], [0, 1]]
configs/trend_with_accel.toml
[]
= "trend_with_accel"
[]
= ["pos", "vel", "acc"]
[]
= "pos + vel*dt + 0.5*acc*dt^2"
= "vel + acc*dt"
= "acc"
[]
= ["z"]
= ["pos"]
[]
= [[0.01,0,0],[0,0.01,0],[0,0,0.01]]
= [[1.0]]
[]
= [0.0, 0.0, 0.0]
= [[1,0,0],[0,1,0],[0,0,1]]
configs/constant_velocity.toml
[]
= "constant_velocity"
[]
= ["pos", "vel"]
[]
= "pos + vel*dt"
= "vel"
[]
= ["z"]
= ["pos"]
[]
= [[0.1, 0], [0, 0.1]]
= [[5.0]]
[]
= [0.0, 0.0]
= [[10, 0], [0, 10]]
Pendulum (EKF with trig)
[]
= "pendulum"
[]
= ["theta", "omega"]
[]
= "theta + omega*dt"
= "omega - 9.81*sin(theta)*dt"
[]
= ["z"]
= ["theta"]
[]
= [[0.001, 0], [0, 0.001]]
= [[0.1]]
[]
= [0.1, 0.0]
= [[0.1, 0], [0, 0.1]]
Language Integration
The CLI communicates exclusively via stdin/stdout JSON lines — no language-specific bindings required. Any language that can spawn a child process works identically.
Python
=
# Read the ready event
=
# Normal observation
=
# Predict-only (sensor dropout)
=
Python — backtest from file
=
=
=
continue
break
# full per-step audit record
TypeScript
import { spawn } from "child_process";
import * as readline from "readline";
const proc = spawn("./target/release/kalix", [
"--config",
"configs/trend_with_accel.toml",
"--mode",
"live",
]);
const rl = readline.createInterface({ input: proc.stdout });
rl.on("line", (line) => {
const msg = JSON.parse(line);
if (msg.event === "ready") {
console.log("Filter ready:", msg.filter, msg.variant);
return;
}
console.log("pos:", msg.x.pos, "vel:", msg.x.vel);
});
// Normal observation
proc.stdin.write(
JSON.stringify({ t: Date.now() / 1000, dt: 1.0, z: [10.3] }) + "\n",
);
// Predict-only (sensor dropout)
proc.stdin.write(
JSON.stringify({ t: Date.now() / 1000 + 1, dt: 1.0, z: null }) + "\n",
);
Tests
Two clearly separated test concerns:
Section 1 — Design: verify TOML -> matrix derivation. Uses exact equality — safe because all F/H entries are integers or IEEE-754-exact fractions (0.5, 0.125).
Section 2 — Filter numerics: verify running filter output. Uses
approx::assert_abs_diff_eq! throughout. Never use exact float equality on filter
state. Shape assertions (lengths, matrix dimensions) may use assert_eq!.
Section 1 — Design: Matrix Derivation
tests/test_expr.rs — Parser
// Literal
parse // Variable
parse // Addition
parse // Multiplication
parse // Integer power
parse // Complex expression — must parse without error
parse // Trig and transcendental functions
parse // Unsupported — must return Err with descriptive message
parse // unknown function
parse // division by variable disallowed
tests/test_diff.rs — Symbolic Differentiation
Differentiate symbolically then evaluate at concrete bindings. All values below are
IEEE-754 exact — assert with epsilon = 1e-15.
let expr = "pos + vel*dt + 0.5*acc*dt^2";
// d/d(pos) = 1.0 — independent of bindings
eval == 1.0
eval == 1.0
// d/d(vel) = dt
eval == 1.0
eval == 0.5
// d/d(acc) = 0.5 * dt^2
eval == 0.5 // 0.5 * 1.0 — exact
eval == 0.125 // 0.5 * 0.25 — exact
// Simplification: d/d(pos) of "pos" must reduce to Lit(1), not a compound expr
matches!
// d/d(vel) and d/d(acc) of "vel + acc*dt"
eval == 1.0
eval == 1.0
eval == 0.5
// ── Transcendental chain rule ────────────────────────────────────────
eval == 1.0
eval == -0.5
eval == 0.5
eval == 1.0
eval == 2.0
tests/test_config.rs — Config Validation and Matrix Derivation
Exact equality on F and H is correct — all entries are 0, 1, or IEEE-754-exact fractions.
// ── trend_no_accel at dt=1.0 ──────────────────────────────────────────
// d(pos + vel*dt)/d(pos) = 1, d(...)/d(vel) = dt = 1
// d(vel)/d(pos) = 0, d(vel)/d(vel) = 1
let F = derive_F;
assert_eq!;
let H = derive_H;
assert_eq!;
assert_eq!;
// ── trend_with_accel at dt=1.0 ────────────────────────────────────────
// d(pos+vel*dt+0.5*acc*dt^2)/d(pos,vel,acc) = [1, 1, 0.5]
// d(vel+acc*dt)/d(pos,vel,acc) = [0, 1, 1.0]
// d(acc)/d(pos,vel,acc) = [0, 0, 1.0]
let F = derive_F;
assert_eq!;
// Same config at dt=0.5 — 0.5 and 0.125 are IEEE-754 exact
let F = derive_F;
assert_eq!;
let H = derive_H;
assert_eq!;
assert_eq!;
// ── EKF detection (nonlinear dynamics) ────────────────────────────────
// Drag: vel_next = vel - 0.01*vel^2*dt -> d/d(vel) = 1 - 0.02*vel*dt
assert_eq!;
// Pendulum: omega_next = omega - 9.81*sin(theta)*dt -> contains sin/theta
assert_eq!;
// Exponential growth: x_next = x + 0.1*exp(x)*dt -> contains exp
assert_eq!;
// ── Validation errors ─────────────────────────────────────────────────
config_with_bad_dynamics_key Section 2 — Filter Numerics
All assertions use approx::assert_abs_diff_eq!(actual, expected, epsilon = 1e-3)
unless stated otherwise. Shape assertions use assert_eq!. Never use exact equality
on filter state.
All expected values were computed analytically and verified with a Python reference implementation before inclusion in this prompt.
tests/test_linear_kf.rs
Config: trend_no_accel, x=[0,0], P=I, Q=0.01*I, R=[[1.0]], dt=1.0.
// ── Step 1: z = [10.0] ───────────────────────────────────────────────
let result = filter.step;
// Predicted (F*[0,0] = [0,0])
assert_abs_diff_eq!;
assert_abs_diff_eq!;
// Residual = z - H*x_predicted = 10.0 - 0.0
assert_abs_diff_eq!;
// Kalman gain shape: n x m = 2 x 1
assert_eq!;
assert_eq!;
// Kalman gain values: K ~= [0.6678, 0.3322]
// Gain is split across both state variables because vel feeds into pos prediction
assert_abs_diff_eq!;
assert_abs_diff_eq!;
// Updated state
assert_abs_diff_eq!;
assert_abs_diff_eq!;
// ── Step 2: z = [11.0] ───────────────────────────────────────────────
let result = filter.step;
// Predicted: pos = 6.6777 + 3.3223*1.0 = 10.0, vel = 3.3223
assert_abs_diff_eq!;
assert_abs_diff_eq!;
// Residual = 11.0 - 10.0 = 1.0
assert_abs_diff_eq!;
// Updated
assert_abs_diff_eq!;
assert_abs_diff_eq!;
// ── Predict-only step ─────────────────────────────────────────────────
// After step 2, call predict_only — state advances but P must grow (no measurement)
let p_post_diag = result.update.updated.P.diagonal.clone;
let predicted = filter.predict_only;
// pos_predicted ~= 10.6689 + 3.6567 = 14.3256
assert_abs_diff_eq!;
// Covariance diagonal must be >= post-update diagonal — uncertainty grows without data
for i in 0..2
// ── Convergence: 100 steps, z=10.0, fresh start ───────────────────────
// epsilon=0.01 — we care about convergence, not a precise settled value
assert_abs_diff_eq!;
assert_abs_diff_eq!;
// ── Named output — backtest serialisation ────────────────────────────
// Confirm JSON output uses state variable names, not integer indices
let json = to_value.unwrap;
assert!;
assert!;
assert!;
tests/test_ekf.rs
Define the config as an inline &str constant inside the test file. Parse it with the
normal config loader. Do not load from disk.
[]
= "drag"
[]
= ["pos", "vel"]
[]
= "pos + vel*dt"
= "vel - 0.01*vel^2*dt" # drag — nonlinear in vel
[]
= ["z"]
= ["pos"]
[]
= [[0.01, 0], [0, 0.01]]
= [[1.0]]
[]
= [0.0, 1.0]
= [[1, 0], [0, 1]]
// ── Variant detection ─────────────────────────────────────────────────
assert_eq!;
// ── Jacobian at state [10.0, 5.0], dt=1.0 ────────────────────────────
// pos_next = pos + vel*dt -> d/dpos=1, d/dvel=dt=1
// vel_next = vel - 0.01*vel^2*dt -> d/dpos=0, d/dvel = 1 - 0.02*vel*dt
// At vel=5.0, dt=1.0: F[1][1] = 1 - 0.02*5.0*1.0 = 0.9 (IEEE-754 exact)
// Use tight epsilon here — these are exact symbolic evaluations, not accumulated ops
let jac = ekf.jacobian;
assert_abs_diff_eq!;
assert_abs_diff_eq!;
assert_abs_diff_eq!;
assert_abs_diff_eq!;
// ── Single step — must not panic, all outputs must be finite ──────────
let result = ekf.step;
assert!;
assert!;
assert!;
// ── Stability: 50 steps, state must remain bounded ────────────────────
for _ in 0..50
assert!;
assert!;
assert!;
// ── Predict-only in EKF — Jacobian must be re-evaluated at current state
let predicted = ekf.predict_only;
assert!;
assert!;
// ── Pendulum (trigonometric dynamics) ─────────────────────────────────
// Jacobian at theta=0, dt=1: F = [[1, 1], [-9.81, 1]]
let jac = pendulum_ekf.jacobian;
assert_abs_diff_eq!;
// At theta=π/6: F[1][0] = -9.81*cos(π/6) ≈ -8.495
let jac = pendulum_ekf.jacobian;
assert_abs_diff_eq!;
// ── Exponential growth ───────────────────────────────────────────────
// x_next = x + 0.1*exp(x)*dt -> F = 1 + 0.1*exp(x)*dt
let jac = exp_ekf.jacobian;
assert_abs_diff_eq!;
Quality Bar
cargo test— zero failurescargo clippy -- -D warnings— zero warnings- All public types and trait methods must have doc comments
anyhowfor application-layer errors (main, CLI),thiserrorfor library errors (config, expr, filter) — do not mix- No
unwrap()orexpect()in library code — propagate errors explicitly - The expression evaluator and symbolic differentiator must be purely functional — no mutation of AST nodes, no interior mutability
src/io/owns all serialisation logic — filter structs must not contain serde derives; the IO layer maps them to output shapes for each mode