Crate oxirs_stream 

OxiRS Stream - Ultra-High Performance RDF Streaming Platform

Status: Alpha Release (v0.1.0-alpha.2) ⚠️ APIs may change. Not recommended for production use.

Real-time streaming support with Kafka/NATS/Redis I/O, RDF Patch, SPARQL Update delta, and advanced event processing capabilities.

This crate provides enterprise-grade real-time data streaming capabilities for RDF datasets, supporting multiple messaging backends with high-throughput, low-latency guarantees.

Features

• Multi-Backend Support: Kafka, NATS JetStream, Redis Streams, AWS Kinesis, Memory
• High Performance: 100K+ events/second, <10ms latency, exactly-once delivery
• Advanced Event Processing: Real-time pattern detection, windowing, aggregations
• Enterprise Features: Circuit breakers, connection pooling, health monitoring
• Standards Compliance: RDF Patch protocol, SPARQL Update streaming

Performance Targets

• Throughput: 100K+ events/second sustained
• Latency: P99 <10ms for real-time processing
• Reliability: 99.99% delivery success rate
• Scalability: Linear scaling to 1000+ partitions

Re-exports

pub use backend_optimizer::BackendOptimizer;

pub use backend_optimizer::BackendPerformance;

pub use backend_optimizer::BackendRecommendation;

pub use backend_optimizer::ConsistencyLevel;

pub use backend_optimizer::CostModel;

pub use backend_optimizer::OptimizationDecision;

pub use backend_optimizer::OptimizationStats;

pub use backend_optimizer::OptimizerConfig;

pub use backend_optimizer::PatternType;

pub use backend_optimizer::WorkloadPattern;

pub use bridge::BridgeInfo;

pub use bridge::BridgeStatistics;

pub use bridge::BridgeType;

pub use bridge::ExternalMessage;

pub use bridge::ExternalSystemConfig;

pub use bridge::ExternalSystemType;

pub use bridge::MessageBridgeManager;

pub use bridge::MessageTransformer;

pub use bridge::RoutingRule;

pub use circuit_breaker::CircuitBreakerError;

pub use circuit_breaker::CircuitBreakerMetrics;

pub use circuit_breaker::FailureType;

pub use circuit_breaker::SharedCircuitBreakerExt;

pub use connection_pool::ConnectionFactory;

pub use connection_pool::ConnectionPool;

pub use connection_pool::DetailedPoolMetrics;

pub use connection_pool::LoadBalancingStrategy;

pub use connection_pool::PoolConfig;

pub use connection_pool::PoolStats;

pub use connection_pool::PoolStatus;

pub use cqrs::CQRSConfig;

pub use cqrs::CQRSHealthStatus;

pub use cqrs::CQRSSystem;

pub use cqrs::Command;

pub use cqrs::CommandBus;

pub use cqrs::CommandBusMetrics;

pub use cqrs::CommandHandler;

pub use cqrs::CommandResult;

pub use cqrs::Query;

pub use cqrs::QueryBus;

pub use cqrs::QueryBusMetrics;

pub use cqrs::QueryCacheConfig;

pub use cqrs::QueryHandler;

pub use cqrs::QueryResult as CQRSQueryResult;

pub use cqrs::ReadModelManager;

pub use cqrs::ReadModelMetrics;

pub use cqrs::ReadModelProjection;

pub use cqrs::RetryConfig as CQRSRetryConfig;

pub use delta::BatchDeltaProcessor;

pub use delta::DeltaComputer;

pub use delta::DeltaProcessor;

pub use delta::ProcessorStats;

pub use event::EventCategory;

pub use event::EventMetadata;

pub use event::EventPriority;

pub use event::IsolationLevel;

pub use event::QueryResult as EventQueryResult;

pub use event::SchemaChangeType;

pub use event::SchemaType;

pub use event::SparqlOperationType;

pub use event::StreamEvent;

pub use event_sourcing::EventQuery;

pub use event_sourcing::EventSnapshot;

pub use event_sourcing::EventStore;

pub use event_sourcing::EventStoreConfig;

pub use event_sourcing::PersistenceBackend;

pub use event_sourcing::QueryOrder;

pub use event_sourcing::RetentionPolicy;

pub use event_sourcing::SnapshotConfig;

pub use event_sourcing::StoredEvent;

pub use event_sourcing::TimeRange as EventSourcingTimeRange;

pub use failover::ConnectionEndpoint;

pub use failover::FailoverConfig;

pub use failover::FailoverManager;

pub use multi_region_replication::ConflictResolution;

pub use multi_region_replication::ConflictType;

pub use multi_region_replication::GeographicLocation;

pub use multi_region_replication::MultiRegionReplicationManager;

pub use multi_region_replication::RegionConfig;

pub use multi_region_replication::RegionHealth;

pub use multi_region_replication::ReplicatedEvent;

pub use multi_region_replication::ReplicationConfig;

pub use multi_region_replication::ReplicationStats;

pub use multi_region_replication::ReplicationStrategy;

pub use multi_region_replication::VectorClock;

pub use patch::PatchParser;

pub use patch::PatchSerializer;

pub use performance_optimizer::AdaptiveBatcher;

pub use performance_optimizer::AggregationFunction;

pub use performance_optimizer::AutoTuner;

pub use performance_optimizer::BatchPerformancePoint;

pub use performance_optimizer::BatchSizePredictor;

pub use performance_optimizer::BatchingStats;

pub use performance_optimizer::EnhancedMLConfig;

pub use performance_optimizer::MemoryPool;

pub use performance_optimizer::MemoryPoolStats;

pub use performance_optimizer::PerformanceConfig as OptimizerPerformanceConfig;

pub use performance_optimizer::ProcessingResult;

pub use performance_optimizer::ProcessingStats;

pub use performance_optimizer::ProcessingStatus;

pub use performance_optimizer::TuningDecision;

pub use performance_optimizer::ZeroCopyEvent;

pub use schema_registry::CompatibilityMode;

pub use schema_registry::ExternalRegistryConfig;

pub use schema_registry::RegistryAuth;

pub use schema_registry::SchemaDefinition;

pub use schema_registry::SchemaFormat;

pub use schema_registry::SchemaRegistry;

pub use schema_registry::SchemaRegistryConfig;

pub use schema_registry::ValidationResult;

pub use schema_registry::ValidationStats;

pub use sparql_streaming::ContinuousQueryManager;

pub use sparql_streaming::QueryManagerConfig;

pub use sparql_streaming::QueryMetadata;

pub use sparql_streaming::QueryResultChannel;

pub use sparql_streaming::QueryResultUpdate;

pub use sparql_streaming::UpdateType;

pub use store_integration::ChangeDetectionStrategy;

pub use store_integration::ChangeNotification;

pub use store_integration::RealtimeUpdateManager;

pub use store_integration::StoreChangeDetector;

pub use store_integration::UpdateChannel;

pub use store_integration::UpdateFilter;

pub use store_integration::UpdateNotification;

pub use biological_computing::AminoAcid;

pub use biological_computing::BiologicalProcessingStats;

pub use biological_computing::BiologicalStreamProcessor;

pub use biological_computing::Cell;

pub use biological_computing::CellState;

pub use biological_computing::CellularAutomaton;

pub use biological_computing::ComputationalFunction;

pub use biological_computing::DNASequence;

pub use biological_computing::EvolutionaryOptimizer;

pub use biological_computing::FunctionalDomain;

pub use biological_computing::Individual;

pub use biological_computing::Nucleotide;

pub use biological_computing::ProteinStructure;

pub use biological_computing::SequenceMetadata;

pub use consciousness_streaming::ConsciousnessLevel;

pub use consciousness_streaming::ConsciousnessStats;

pub use consciousness_streaming::ConsciousnessStreamProcessor;

pub use consciousness_streaming::DreamSequence;

pub use consciousness_streaming::EmotionalContext;

pub use consciousness_streaming::IntuitiveEngine;

pub use consciousness_streaming::MeditationState;

pub use observability::AlertConfig;

pub use observability::AlertEvent;

pub use observability::AlertSeverity;

pub use observability::AlertType;

pub use observability::BusinessMetrics;

pub use observability::SpanLog;

pub use observability::SpanStatus;

pub use observability::StreamObservability;

pub use observability::StreamingMetrics;

pub use observability::TelemetryConfig;

pub use observability::TraceSpan;

pub use performance_utils::AdaptiveRateLimiter;

pub use performance_utils::IntelligentMemoryPool;

pub use performance_utils::IntelligentPrefetcher;

pub use performance_utils::ParallelStreamProcessor;

pub use performance_utils::PerformanceUtilsConfig;

pub use quantum_communication::BellState;

pub use quantum_communication::EntanglementDistribution;

pub use quantum_communication::QuantumCommConfig;

pub use quantum_communication::QuantumCommSystem;

pub use quantum_communication::QuantumOperation as QuantumCommOperation;

pub use quantum_communication::QuantumSecurityProtocol;

pub use quantum_communication::QuantumState as QuantumCommState;

pub use quantum_communication::Qubit;

pub use quantum_streaming::QuantumEvent;

pub use quantum_streaming::QuantumOperation;

pub use quantum_streaming::QuantumProcessingStats;

pub use quantum_streaming::QuantumState;

pub use quantum_streaming::QuantumStreamProcessor;

pub use reliability::BulkReplayResult;

pub use reliability::DlqStats;

pub use reliability::ReplayStatus;

pub use security::AuditConfig;

pub use security::AuditLogEntry;

pub use security::AuditLogger;

pub use security::AuthConfig;

pub use security::AuthMethod;

pub use security::AuthenticationProvider;

pub use security::AuthorizationProvider;

pub use security::AuthzConfig;

pub use security::Credentials;

pub use security::EncryptionConfig;

pub use security::Permission;

pub use security::RateLimitConfig;

pub use security::RateLimiter;

pub use security::SecurityConfig as StreamSecurityConfig;

pub use security::SecurityContext;

pub use security::SecurityManager;

pub use security::SecurityMetrics;

pub use security::SessionConfig;

pub use security::ThreatAlert;

pub use security::ThreatDetectionConfig;

pub use security::ThreatDetector;

pub use time_travel::AggregationType;

pub use time_travel::TemporalAggregations;

pub use time_travel::TemporalFilter;

pub use time_travel::TemporalOrdering;

pub use time_travel::TemporalProjection;

pub use time_travel::TemporalQuery;

pub use time_travel::TemporalQueryResult;

pub use time_travel::TemporalResultMetadata;

pub use time_travel::TemporalStatistics;

pub use time_travel::TimePoint;

pub use time_travel::TimeRange as TimeTravelTimeRange;

pub use time_travel::TimeTravelConfig;

pub use time_travel::TimeTravelEngine;

pub use time_travel::TimeTravelMetrics;

pub use time_travel::TimelinePoint;

pub use wasm_edge_computing::EdgeExecutionResult;

pub use wasm_edge_computing::EdgeLocation;

pub use wasm_edge_computing::OptimizationLevel;

pub use wasm_edge_computing::ProcessingSpecialization;

pub use wasm_edge_computing::WasmEdgeConfig;

pub use wasm_edge_computing::WasmEdgeProcessor;

pub use wasm_edge_computing::WasmPlugin;

pub use wasm_edge_computing::WasmResourceLimits;

pub use webhook::EventFilter as WebhookEventFilter;

pub use webhook::HttpMethod;

pub use webhook::RateLimit;

pub use webhook::RetryConfig as WebhookRetryConfig;

pub use webhook::WebhookConfig;

pub use webhook::WebhookInfo;

pub use webhook::WebhookManager;

pub use webhook::WebhookMetadata;

pub use webhook::WebhookSecurity;

pub use webhook::WebhookStatistics;

Modules

backend

Stream Backend Abstraction

backend_optimizer

Backend Optimization and Selection

biological_computing

Biological Computing Integration for RDF Streaming

bridge

Message Queue Bridge Module

circuit_breaker

Advanced Circuit Breaker Implementation

config

Advanced Configuration Management

connection_pool

Advanced Connection Pool Implementation

consciousness_streaming

Consciousness-Inspired Streaming Engine

consumer

Stream Consumer

cqrs

CQRS (Command Query Responsibility Segregation) Implementation

delta

SPARQL Update Delta Support

diagnostics

Stream Diagnostics Tools

error

Stream Error Types

event

Stream Events

event_sourcing

Event Sourcing Framework

failover

Failover mechanisms for connection pool

health_monitor

Connection health monitoring and tracking

join

Stream Join Operations

monitoring

Monitoring and Observability

multi_region_replication

Multi-Region Replication

observability

Advanced Observability and Telemetry for OxiRS Stream

patch

RDF Patch Support

performance_optimizer

Advanced Performance Optimizer

performance_utils

Advanced Performance Utilities for OxiRS Stream

processing

Advanced Event Processing Module

producer

Stream Producer

quantum_communication

Quantum Communication Module

quantum_processing

Quantum processing module - modular quantum computing integration for RDF streams

quantum_streaming

Quantum streaming module components

reconnect

Automatic reconnection logic with exponential backoff

reliability

Reliability Module

schema_registry

Schema Registry for Stream Events

security

Advanced Security Framework

serialization

Event Serialization Module

sparql_streaming

SPARQL Streaming Module

state

Stateful Stream Processing

store_integration

OxiRS Store Integration

time_travel

Time-Travel Query System

types

Stream Types

wasm_edge_computing

WebAssembly Edge Computing Module

webhook

Webhook Integration Module

Structs

AwsCredentials

AWS credentials configuration

CircuitBreakerConfig

Circuit breaker configuration

ConsumerStats

Consumer statistics for monitoring

MonitoringConfig

Monitoring configuration

NatsJetStreamConfig

NATS JetStream configuration

ProducerStats

Producer statistics for monitoring

PulsarAuthConfig

Pulsar authentication configuration

RdfPatch

RDF patch for atomic updates with full protocol support

RetryConfig

Retry configuration

SaslConfig

SASL authentication configuration

SecurityConfig

Security configuration

Stream

Unified Stream interface that combines producer and consumer functionality

StreamConfig

Enhanced stream configuration with advanced features

StreamConsumer

Enhanced stream consumer for receiving RDF changes with backend support

StreamPerformanceConfig

Performance tuning configuration

StreamProducer

Enhanced stream producer for publishing RDF changes with backend support

Enums

CompressionType

Compression types supported

PatchOperation

RDF patch operations with full protocol support

PulsarAuthMethod

Pulsar authentication methods

SaslMechanism

SASL authentication mechanisms

StreamBackendType

Enhanced streaming backend options

Functions

clear_memory_events

Clear the global memory storage (for testing)

get_memory_events