liner

liner is a lightweight, serverless, peer-to-peer message broker written in Rust. It provides a decentralized messaging mesh backed by Redis, SQLite, or PostgreSQL, with pure TCP between peers.

• Decentralized architecture — peer-to-peer mesh, no central broker process.
• Flexible storage — Redis, SQLite, or PostgreSQL (backends).
• At-least-once delivery — store-backed persistence and offline queues (per backend).
• Cross-language — Rust, Python, and C++ (C API).
• High message bandwidth — raw TCP; see benchmark.
• Messaging patterns — one-to-one, one-to-many, many-to-many, topic subscription.
• Unlimited payload size — not fixed by the wire format.
• Cross-platform — Linux and Windows.

Supported backends

• Redis — shared URL, default build. See using-redis.md.
• SQLite — per-process files; receivers_json seeds peers on isolated DBs. See using-sqlite.md.
• PostgreSQL — one shared database URL (Cargo feature postgres). See using-postgres.md.

Quick example (Redis)

Rust:

use liner_broker::Liner;

fn main() {
    let mut client1 = Liner::new("client1", "topic_client1", "localhost:2255", "redis://localhost/");
    let mut client2 = Liner::new("client2", "topic_client2", "localhost:2256", "redis://localhost/");

    client1.run(Box::new(|_to, from, _data| println!("receive_from {}", from)));
    client2.run(Box::new(|_to, _from, _data| {}));

    let payload = [0u8; 100];
    for _ in 0..10 {
        client1.send_to("topic_client2", &payload, true);
    }
}

Python:

import liner

liner.loadLib("./target/release/libliner_broker.so")

def on_msg(_to: str, from_: str, data: bytes):
    print(f"receive_from {from_}, data: {data}")

client1 = liner.Client("client1", "topic_client", "localhost:2255", "redis://localhost/")
client2 = liner.Client("client2", "topic_client", "localhost:2256", "redis://localhost/")

client1.run(on_msg)
client2.run(on_msg)

client1.send_to("topic_client", b"hello", True)

SQLite / PostgreSQL constructors: Liner::new_sqlite, Client.new_sqlite, Liner::new_postgres, Client.new_postgres — see docs below.

Build

Install Rust and Cargo, then:

cargo build --release

PostgreSQL backend:

cargo build --release --features postgres

Architecture

Examples of use

One to one: Python / CPP / Rust

One to one for many: Python / CPP / Rust

One to many: Python / CPP / Rust

Many to many: Python / CPP / Rust

Producer-consumer: Python / CPP / Rust

Benchmark

Three binaries run the same workload: two clients, send_to loop (10k messages × 100 cycles). Only the store differs (benchmark/).

cargo build --release --bin bench_pair_sendto_redis
./target/release/bench_pair_sendto_redis

cargo build --release --bin bench_pair_sendto_sqlite
./target/release/bench_pair_sendto_sqlite

export LINER_BENCH_POSTGRES_URL='postgresql://user:pass@127.0.0.1/liner_test'
cargo build --release --features postgres --bin bench_pair_sendto_postgres
./target/release/bench_pair_sendto_postgres

Sample output (liner + Redis, 10k messages per cycle):

$ ./bench_pair_sendto_redis
send_to 8 ms
receive_from 8 ms
send_to 5 ms
receive_from 5 ms
send_to 7 ms
receive_from 3 ms
send_to 11 ms
receive_from 3 ms
send_to 6 ms
receive_from 3 ms

About 10 ms on average for 10k messages.

Comparison with ZeroMQ on the same machine (make && ./compare_with_zmq):

$ make
g++ -Wall -O2 -std=c++17 -g -Wno-write-strings -o compare_with_zmq compare_with_zmq.cpp -lzmq
$ ./compare_with_zmq
Connecting to tcp://127.0.0.1:34079
send_to 20.198 ms
send_to 16.504 ms
send_to 11.5 ms
send_to 13.153 ms
send_to 10.964 ms
send_to 10.788 ms
send_to 10.785 ms
send_to 11.119 ms
send_to 11.348 ms
send_to 10.826 ms

For ZeroMQ it is similar (on the order of ~10 ms per 10k messages).

Tests

Rust unit tests:

cargo test
# PostgreSQL store tests (optional):
LINER_TEST_POSTGRES_URL='postgresql://user:pass@127.0.0.1/liner_test' \
  cargo test --features postgres

Python integration suites (build release library first):

cargo build --release
python3 test/redis/run_integration.py --list
python3 test/sqlite/run_integration.py

PostgreSQL (requires postgres feature and a running database):

export LINER_TEST_POSTGRES_URL='postgresql://user:pass@127.0.0.1/liner_test'
cargo build --release --features postgres
python3 test/postgres/run_integration.py

Redis tests under test/redis/ auto-start Redis via Docker when needed. Optional env:

LINER_TEST_REDIS_PORT=16379 LINER_TEST_REDIS_CONTAINER=liner-test-redis \
  python3 test/redis/run_integration.py --only offline,burst

Docs

• Using Redis — new_redis, shared URL, test/redis/
• Using SQLite — new_sqlite, receivers_json, isolated DB limits
• Using PostgreSQL — --features postgres, shared database
• Crate API on docs.rs
• Developer notes — errors, backends, C API, lifecycle
• C API compatibility and build

License

Licensed under the MIT License.