spectra (uf-spectra)

Typed metrics, structured event logs, and pluggable storage for Rust services. Depend on this as the primary crate; enable backend features explicitly on your dependency. At boot wire Spectra::builder() and use the prelude re-exports.

Role

• Re-exports spectra-core, spectra-runtime, and feature-gated backends
• default = ["mem"]; optional sqlite, tensorbase, clickhouse, telemetry-console
• CI demo schemas (platform_smoke_*) register via inventory; macros emit typed helpers and topics in linked modules

Configuration

Spectra has no global config file loader. Settings merge in this order (highest wins):

1. Explicit builder calls — .config(SpectraConfig { ... }), .sink(...), .persist_disabled(), backend constructors
2. SpectraConfig — programmatic overrides passed to .config()
3. Environment variables — read by SpectraConfig::from_env() when .config() is omitted
4. Schema defaults — per-metric/event levels and sample rates from the DSL
5. Library defaults — documented below

Cargo features

Enable backends at compile time on your spectra dependency. See the root README.

Builder composition

Publisher/consumer setup: cargo doc -p uf-spectra --open → Getting started → Publish-consume, then SpectraSink, topics, and examples quickstart_publish_only / quickstart_consume_forward.

Emit gate and sampling

Loaded by SpectraConfig::from_env() unless .config(...) overrides.

Event query paging is clamped in-library (MAX_EVENT_QUERY_LIMIT = 1000). Metric/table/field identifiers must match validate_spectra_ident. See repository SECURITY.md.

Emit buffer (embedded profile)

Set any of these to 0/false/no to disable.

Web note: prefer try_record_*_now / generated helpers (L2 enqueue) over request_scope. request_scope drops undrained emits on panic or early exit — avoid if you need failure telemetry.

Async storage persist (builder)

Configure L2 queue/batch on Spectra::builder() — not environment variables:

use std::time::Duration;
use spectra::{PersistConfig, Spectra};

Spectra::builder()
    // …backends…
    .persist(PersistConfig {
        queue_max: 8192,                      // overflow policy applies when full
        batch_max: 2048,                      // raise for DW firehose
        batch_wait: Duration::from_millis(5), // coalesce delay
        batch_enabled: true,                  // use batch insert APIs
    })
    .build()?;

After fire-and-forget *_now emits, scripts that need durability before exit call spectra.flush_persist().await.

Telemetry console (telemetry-console feature)

Use .telemetry_ndjson(dir) on the builder to write {dir}/metrics.ndjson and {dir}/events.ndjson.

Remote backends

Debug

Backend wiring

In-memory (default)

use spectra::{MemEventsBackend, MemMetricsBackend, Spectra};

let _spectra = Spectra::builder()
    .metrics_backend(std::sync::Arc::new(MemMetricsBackend::new()))
    .events_backend(std::sync::Arc::new(MemEventsBackend::new()))
    .embedded()
    .build()?;

SQLite (durable embedded)

use spectra::{SqliteEventsBackend, SqliteMetricsBackend, Spectra};

let metrics = SqliteMetricsBackend::new("/tmp/spectra-metrics.db")?;
let events = SqliteEventsBackend::new("/tmp/spectra-events.db")?;
let _spectra = Spectra::builder()
    .metrics_backend(std::sync::Arc::new(metrics))
    .events_backend(std::sync::Arc::new(events))
    .embedded()
    .build()?;

Requires features = ["sqlite"].

Remote (ClickHouse / TensorBase)

See spectra-backend-clickhouse/README.md and spectra-backend-tensorbase/README.md.

Schema collection

Your application owns telemetry DSL modules and links them with an explicit mod list. This repository demonstrates the contract with CI demo schemas under schemas/ and re-exports smoke helpers and topics from those expansions.

How to run examples

Navigational index: examples/README.md (when-to-use ladder + links to each .rs file).

Canonical teaching path (start here). Topology docs: Direct persist / Publish-consume / Dual-path.

export CARGO_BUILD_JOBS=1 CARGO_TARGET_DIR=target-spectra-extract

1. Embedded + schema emit — quickstart_schema_emit (standalone)

One process, in-memory store, typed helpers + query. No external services.

cargo run -p uf-spectra --example quickstart_schema_emit --features mem

Success: stderr prints schema emit OK: … metric point(s) persisted.

2. Publish-consume (two binaries)

Publisher and consumer are separate binaries in production. Spectra does not ship a message bus; your host owns that piece. The bus_* examples use a TCP JSON-lines adapter so you can run a real process boundary without Photon. Replace TcpJsonSink / the listener with your bus (Photon, NATS, …) in production.

export SPECTRA_BUS_ADDR=127.0.0.1:9809
# Terminal 1 — consumer
cargo run -p uf-spectra --example bus_consume --features mem
# Terminal 2 — publisher
cargo run -p uf-spectra --example bus_publish --features mem

Success: bus-consume OK: … metric point(s) in storage.

In-process sketches (no TCP): quickstart_publish_only, quickstart_consume_forward.

ClickHouse dual-path (sink + remote persist in one process):

docker compose -f docker-compose.dev.yml up -d clickhouse
export SPECTRA_ALLOW_INSECURE_REMOTE=1
export SPECTRA_CLICKHOUSE_URL=http://127.0.0.1:8123
cargo run -p uf-spectra --example clickhouse_dual_path --features clickhouse

3. Remote storage — quickstart_clickhouse_emit (standalone)

Direct persist into ClickHouse. Requires a live ClickHouse (Docker Compose below).

docker compose -f docker-compose.dev.yml up -d clickhouse
export SPECTRA_ALLOW_INSECURE_REMOTE=1   # local plaintext only
export SPECTRA_CLICKHOUSE_URL=http://127.0.0.1:8123
cargo run -p uf-spectra --example quickstart_clickhouse_emit --features clickhouse

Success: stderr prints clickhouse emit OK: … metric point(s), … event row(s).

Other examples

Status

Shipped in tag v0.1.0.