Crate oxirs_tsdb 

Time-series optimizations for OxiRS

Status: ✅ Production Ready (v0.1.0)

This crate provides high-performance time-series storage and query capabilities for IoT-scale RDF data.

Features

• ✅ Gorilla compression - 40:1 storage reduction (Facebook, VLDB 2015)
• ✅ Delta-of-delta timestamps - <2 bits per timestamp
• ✅ SPARQL temporal extensions - WINDOW, RESAMPLE, INTERPOLATE
• ✅ 500K+ writes/sec - High-throughput ingestion
• ✅ Hybrid storage - Seamless RDF + Time-Series integration
• ✅ Retention policies - Automatic downsampling and expiration
• ✅ Write-Ahead Log - Crash recovery and durability
• ✅ Background compaction - Automatic storage optimization
• ✅ Columnar storage - Disk-backed binary format
• ✅ Series indexing - Efficient chunk lookups

Architecture

┌─────────────────────────────────────────────┐
│         Hybrid Storage Model                │
├─────────────────────────────────────────────┤
│                                             │
│  ┌──────────────┐    ┌─────────────────┐  │
│  │  RDF Store   │◄──►│ Time-Series DB  │  │
│  │  (oxirs-tdb) │    │  (this crate)   │  │
│  └──────────────┘    └─────────────────┘  │
│        │                     │              │
│        │ Semantic            │ High-freq    │
│        │ metadata            │ sensor data  │
│        └──────────┬──────────┘              │
│                   │                         │
│        ┌──────────▼─────────┐               │
│        │ Unified SPARQL     │               │
│        │ Query Layer        │               │
│        └────────────────────┘               │
└─────────────────────────────────────────────┘

Quick Start

Basic Usage

use oxirs_tsdb::HybridStore;
use chrono::Utc;

// Create hybrid store (RDF + time-series)
let store = HybridStore::new()?;

// Direct time-series insertion
let series_id = 1;
let timestamp = Utc::now();
let value = 22.5;
store.insert_ts(series_id, timestamp, value)?;

// Query time range
let start = timestamp - chrono::Duration::hours(1);
let end = timestamp + chrono::Duration::hours(1);
let points = store.query_ts_range(series_id, start, end)?;

SPARQL Temporal Extensions

PREFIX ts: <http://oxirs.org/ts#>

# Moving average over 10-minute window
SELECT ?sensor (ts:window(?temp, 600, "AVG") AS ?avg_temp)
WHERE {
  ?sensor :temperature ?temp ;
          :timestamp ?time .
}

# Resample to hourly averages
SELECT ?hour (AVG(?power) AS ?avg_power)
WHERE {
  ?sensor :power ?power ;
          :timestamp ?time .
}
GROUP BY (ts:resample(?time, "1h") AS ?hour)

Compression

Gorilla Encoding (for float values)

Facebook’s Gorilla compression exploits temporal locality in sensor data:

• XOR with previous value
• Variable-length encoding for XOR result
• Typical compression: 30-50:1 for IoT data

Delta-of-Delta (for timestamps)

Exploits regularity in sensor sampling intervals:

• Store delta of consecutive deltas
• Variable-length encoding
• Typical compression: 32:1 for regular sampling

Performance

Targets (on AWS m5.2xlarge: 8 vCPUs, 32GB RAM)

• Write throughput: 1M+ data points/sec
• Query latency: <200ms for 1M points (p50)
• Compression ratio: 40:1 (average)
• Memory usage: <2GB for 100M points

Integration

Automatic integration with existing OxiRS components:

• ✅ oxirs-core::store::Store trait implementation (HybridStore)
• ✅ oxirs-stream ready for MQTT/Modbus ingestion
• ✅ oxirs-arq ready for SPARQL temporal extensions

CLI Commands

The oxirs CLI provides comprehensive time-series commands:

# Query with aggregation
oxirs tsdb query mykg --series 1 --aggregate avg

# Insert data point
oxirs tsdb insert mykg --series 1 --value 22.5

# Show compression statistics
oxirs tsdb stats mykg --detailed

# Manage retention policies
oxirs tsdb retention list mykg

Production Readiness

• ✅ 128/128 tests passing - Comprehensive test coverage
• ✅ Zero warnings - Strict code quality enforcement
• ✅ 10 examples - Complete usage documentation
• ✅ 3 benchmarks - Performance validation
• ✅ Production features - WAL, compaction, retention, caching

Re-exports

pub use config::AggregationFunction;

pub use config::TsdbConfig;

pub use error::TsdbError;

pub use error::TsdbResult;

pub use series::DataPoint;

pub use series::SeriesDescriptor;

pub use series::SeriesMetadata;

pub use storage::ChunkEntry;

pub use storage::ColumnarStore;

pub use storage::SeriesIndex;

pub use storage::DeltaOfDeltaCompressor;

pub use storage::DeltaOfDeltaDecompressor;

pub use storage::GorillaCompressor;

pub use storage::GorillaDecompressor;

pub use storage::TimeChunk;

pub use write::BufferConfig;

pub use write::BufferStats;

pub use write::CompactionConfig;

pub use write::CompactionStats;

pub use write::Compactor;

pub use write::RetentionEnforcer;

pub use write::RetentionStats;

pub use write::WalEntry;

pub use write::WriteAheadLog;

pub use write::WriteBuffer;

pub use query::Aggregation;

pub use query::AggregationResult;

pub use query::InterpolateMethod;

pub use query::Interpolator;

pub use query::QueryBuilder;

pub use query::QueryEngine;

pub use query::QueryResult;

pub use query::RangeQuery;

pub use query::ResampleBucket;

pub use query::Resampler;

pub use query::TimeRange;

pub use query::WindowFunction;

pub use query::WindowSpec;

pub use integration::Confidence;

pub use integration::DetectionResult;

pub use integration::HybridStore;

pub use integration::RdfBridge;

pub use sparql::interpolate_function;

pub use sparql::register_temporal_functions;

pub use sparql::resample_function;

pub use sparql::window_function;

pub use sparql::QueryRouter;

pub use sparql::RoutingDecision;

pub use sparql::TemporalFunctionRegistry;

pub use sparql::TemporalValue;

Modules

config

Configuration types for TSDB storage and query options.

error

Error types and result aliases for TSDB operations.

integration

Integration with oxirs-core RDF Store (hybrid storage). Integration with oxirs-core RDF Store

query

Query engine for time-series data with aggregations and window functions. Time-series query engine

series

Series definitions including data points and metadata.

sparql

SPARQL temporal extensions for time-series queries. SPARQL temporal extensions for time-series queries

storage

Storage layer with compression and chunk management. Columnar storage engine with compression

write

Write path with WAL and compaction for durable data ingestion. Write path with WAL and compaction