Crate oxirs_tdb

Expand description

§OxiRS TDB - Apache Jena TDB/TDB2 Compatible Storage Engine

Status: Production Release (v0.3.1) Stability: Public APIs are stable. Production-ready with comprehensive testing.

High-performance RDF triple store with B+Tree indexes, ACID transactions, and Apache Jena TDB/TDB2 API compatibility.

§Features

B+Tree Storage - Efficient range queries and sequential scans
Triple Indexes - SPO, POS, OSP for optimal query performance
ACID Transactions - Write-Ahead Logging (WAL) with 2PL concurrency control
Dictionary Encoding - Compress IRIs and literals to 8-byte NodeIDs
Buffer Pool - LRU caching for hot pages
100M+ Triples - Scalable to large datasets

§Quick Start

use oxirs_tdb::TdbStore;
// Create a new TDB store
let mut store = TdbStore::open("/path/to/data")?;

// Insert triples
// store.insert(...)?;

// Query triples
// let results = store.query(...)?;

§Architecture

┌─────────────────────────────────────────┐
│         TdbStore (High-level API)       │
└─────────────────┬───────────────────────┘
                  │
      ┌───────────┴───────────┐
      │                       │
┌─────▼─────┐         ┌──────▼──────┐
│  Indexes  │         │ Dictionary  │
│ SPO/POS/  │         │   Encoding  │
│    OSP    │         │             │
└─────┬─────┘         └──────┬──────┘
      │                      │
      └───────────┬──────────┘
                  │
           ┌──────▼──────┐
           │   B+Tree    │
           └──────┬──────┘
                  │
      ┌───────────┴───────────┐
      │                       │
┌─────▼─────┐         ┌──────▼──────┐
│  Buffer   │         │     WAL     │
│   Pool    │         │  (Logging)  │
└─────┬─────┘         └──────┬──────┘
      │                      │
      └───────────┬──────────┘
                  │
           ┌──────▼──────┐
           │ File Manager│
           │   (mmap)    │
           └─────────────┘

§See Also

oxirs-core - RDF data model
oxirs-arq - SPARQL query engine

Re-exports§

pub use backup_encryption::BackupEncryption;
pub use backup_encryption::EncryptedData;
pub use backup_encryption::EncryptionConfig;
pub use bulk_operations::BulkStats;
pub use bulk_operations::BulkTripleProcessor;
pub use bulk_operations::ParallelPipelineBuilder;
pub use bulk_operations::StreamingTripleIterator;
pub use compression::DeltaStoreConfig;
pub use compression::DeltaStoreStats;
pub use compression::EncodedDeltaLog;
pub use compression::TripleDelta;
pub use compression::TripleDeltaStore;
pub use connection_pool::ConnectionPool;
pub use connection_pool::ConnectionPoolConfig;
pub use connection_pool::ConnectionPoolStatsSnapshot;
pub use database_ops::CompactionStats;
pub use database_ops::DatabaseMetadata;
pub use database_ops::DatabaseOps;
pub use database_ops::DatabaseStatus;
pub use database_ops::RepairReport;
pub use error::Result;
pub use error::TdbError;
pub use index::BTreeTripleIndex;
pub use index::EncodedTriple;
pub use index::TripleIndexSet;
pub use index::TripleOrdering;
pub use loader::BulkLoadStats;
pub use loader::BulkLoader;
pub use loader::BulkLoaderConfig;
pub use loader::BulkLoaderFactory;
pub use loader::NodeDictionary;
pub use loader::ParallelBulkLoadConfig;
pub use loader::ParallelBulkLoadStats;
pub use loader::ParallelBulkLoader;
pub use loader::RawTriple;
pub use loader::RdfNode;
pub use loader::TripleSource;
pub use loader::VecTripleSource;
pub use materialized_views::MaterializedView;
pub use materialized_views::MaterializedViewConfig;
pub use materialized_views::MaterializedViewManager;
pub use materialized_views::MaterializedViewManagerStats;
pub use materialized_views::RefreshStrategy;
pub use materialized_views::ViewInfo;
pub use observability::HealthCheck;
pub use observability::HealthCheckResult;
pub use observability::HealthCheckResults;
pub use observability::HealthStatus;
pub use observability::MetricSnapshot;
pub use observability::ObservabilityConfig;
pub use observability::ObservabilityManager;
pub use observability::TraceSpan;
pub use observability::TraceSpanId;
pub use online_backup::OnlineBackupManager;
pub use online_backup::OnlineBackupStats;
pub use online_backup::Snapshot;
pub use online_backup::SnapshotConfig;
pub use online_backup::SnapshotId;
pub use online_backup::SnapshotStatus;
pub use query_resource_quota::QueryQuotaStats;
pub use query_resource_quota::QueryResourceQuotaConfig;
pub use query_resource_quota::QueryResourceQuotaManager;
pub use query_resource_quota::QueryResourceTracker;
pub use query_resource_quota::QueryResourceUsage;
pub use store::CompressionAlgorithm;
pub use store::ReplicationMode;
pub use store::StoreParams;
pub use store::StoreParamsBuilder;
pub use store::StorePresets;
pub use store::TdbConfig;
pub use store::TdbStats;
pub use store::TdbStore;
pub use wal_archive::WalArchiveConfig;
pub use wal_archive::WalArchiveMetadata;
pub use wal_archive::WalArchiver;
pub use wal_archive::WalArchiverStatsSnapshot;
pub use wal_shipping::ShippingConfig;
pub use wal_shipping::ShippingDestination;
pub use wal_shipping::ShippingRecord;
pub use wal_shipping::ShippingStats;
pub use wal_shipping::ShippingStatus;
pub use wal_shipping::WalShipper;
pub use temporal::ChangeEntry;
pub use temporal::ChangeLog;
pub use temporal::ChangeLogStats;
pub use temporal::ChangeOperation;
pub use temporal::TemporalDiff;
pub use temporal::TemporalSnapshot;
pub use temporal::TemporalVersionStore;
pub use temporal::TimeTravelQuery;
pub use temporal::TimeTravelQueryBuilder;
pub use temporal::TripleKey;
pub use temporal::TriplePattern;
pub use temporal::VersionedTriple;
pub use mvcc::deadlock::DeadlockDetector;
pub use mvcc::deadlock::DeadlockScanResult;
pub use mvcc::IsolationLevel;
pub use mvcc::MvccError;
pub use mvcc::MvccManager;
pub use mvcc::MvccStats;
pub use mvcc::TransactionContext;
pub use mvcc::TransactionState;
pub use mvcc::TxId;
pub use mvcc::VersionedEntry;
pub use mvcc::VersionedKey;
pub use mvcc::TX_ID_COMMITTED;
pub use tenant::TenantAuditEvent;
pub use tenant::TenantAuditLog;
pub use tenant::TenantConfig;
pub use tenant::TenantError;
pub use tenant::TenantHandle;
pub use tenant::TenantId;
pub use tenant::TenantIsolationLayer;
pub use tenant::TenantRegistry;
pub use tenant::TenantResult;
pub use tenant::TenantStats;
pub use tenant::TenantStore;
pub use tdb2::NodeTable;
pub use tdb2::RdfTerm;
pub use tdb2::Tdb2Database;
pub use tdb2::Tdb2Format;
pub use tdb2::TripleIndex;
pub use backup_engine::BackupCompression;
pub use backup_engine::BackupDelta;
pub use backup_engine::BackupEngine;
pub use backup_engine::BackupFormat;
pub use backup_engine::BackupManifest;
pub use backup_engine::IncrementalBackup;
pub use sla::SlaComplianceReport;
pub use sla::SlaMonitor;
pub use sla::SlaViolation;
pub use sla::StorageSla;
pub use sla::ViolationType;
pub use formats::CsvRdfMapper;
pub use formats::JsonLdImporter;
pub use formats::NQuadsExporter;
pub use formats::NQuadsImporter;
pub use formats::RdfColumnType;
pub use wal_compaction::BaseSnapshot;
pub use wal_compaction::CompactionConfig;
pub use wal_compaction::CompactionStats as WalCompactionStats;
pub use wal_compaction::PitrConfig;
pub use wal_compaction::PitrEngine;
pub use wal_compaction::RecoveryPlan;
pub use wal_compaction::WalCompactor;
pub use wal_compaction::WalRecord;
pub use wal_compaction::WalRecordType;
pub use wal_compaction::WalSegment;

Modules§

adaptive_execution: Adaptive Query Execution
advanced_diagnostics: Advanced diagnostic tools for TDB store monitoring and analysis
advanced_diagnostics_engine: Core engine for advanced TDB diagnostics.
advanced_diagnostics_types: Data types for advanced TDB diagnostics.
backup: Backup and Restore Utilities for OxiRS TDB
backup_encryption: Backup Encryption for Data at Rest
backup_engine: BackupEngine and IncrementalBackup for OxiRS TDB
bloom_filter: Space-efficient Bloom filter for triple existence checks.
bloom_index: Bloom filter index for fast triple existence checks: FNV double-hashing, capacity-tuned bit array, TripleBloomIndex wrapper (v1.1.0 round 16) Bloom filter index for fast RDF triple membership tests (v1.1.0 round 16).
btree: B+Tree implementation for key-value storage
btree_index: B-tree based index for triple pattern matching.
bulk_operations: Bulk operations module for high-throughput RDF processing
checkpoint_manager: Database checkpoint management for crash recovery and WAL truncation (v1.9.0). Database checkpoint management for TDB storage.
cloud_storage: Cloud storage integration for TDB backups and data storage
compaction_scheduler: Background compaction task scheduler (v1.7.0). Background compaction task scheduler for OxiRS TDB.
compression: Advanced Compression Module for TDB Storage
connection_pool: Connection pooling for multi-client RDF database access
connection_pool_optimizer: Advanced connection pool optimization features
consensus: Consensus Algorithms for Distributed Coordination
database_ops: Database operations and management utilities
diagnostics: Advanced diagnostic tools for TDB store.
diagnostics_collectors: Diagnostic engine and built-in check collectors.
diagnostics_tests: Unit tests for the diagnostics subsystem.
diagnostics_types: Diagnostic data types: levels, severities, results, reports, and repair recommendations.
dictionary: Dictionary encoding for IRIs and Literals
distributed: Distributed Coordination and Replication
error: Error types for OxiRS TDB storage engine
formats: Dataset import/export format converters for OxiRS TDB.
index: Triple and quad index structures (SPO, POS, OSP, GSPO, GPOS, GOSP)
index_rebuilder: B-tree index rebuild from raw triple data: SPO/POS/OSP/GSPO/GPOS/GOSP orders, sorted IndexEntry generation, rebuild statistics, and sortedness verification (v1.1.0 round 14) B-tree index rebuild from raw triple data (v1.1.0 round 14).
index_statistics: Triple store cardinality estimator for query optimization.
journal: Write-ahead journal for crash recovery
loader: Data loading utilities for OxiRS TDB
lsm_compaction: LSM-tree Compaction Strategy
materialized_views: Materialized Views for Query Acceleration
mvcc: MVCC (Multi-Version Concurrency Control) Transaction Manager for OxiRS TDB
observability: Observability module for comprehensive monitoring and metrics
online_backup: Online Backup Without Downtime
performance: Performance optimization module using SciRS2-Core
prefix_table: Compressed Prefix Table for IRI storage in triple indexes.
production: Production Hardening Features for OxiRS TDB
query_cache: Query result caching for improved performance
query_hints: Query hint support for optimization
query_join_optimizer: Join Order Optimization for SPARQL Query Planning
query_monitor: Query monitoring for timeout enforcement and slow query logging
query_optimizer: Advanced query optimizer for TDB storage
query_resource_quota: Query Resource Quota Management
query_timeout: Query Timeout Enforcement
rdf_star: RDF-star support for quoted triples
recovery: Crash recovery and corruption detection
six_index_store: Six-index triple store (SPO/POS/OSP and derived patterns) for O(log n) pattern matching — used in TDB2 parity verification. Six-index (SPO/SOP/PSO/POS/OSP/OPS) triple store.
sla: Advanced SLA monitoring and enforcement for OxiRS TDB storage layer.
slow_query_log: Slow Query Logging and Analysis
statistics: Statistics collection for cost-based query optimization
storage: Low-level storage layer
store: High-level TDB store API
tdb2: TDB2-Compatible Data Format
temporal: Temporal RDF – Version Control and Time-Travel Queries
tenant: Multi-Tenant Isolation for OxiRS TDB
text_search: Full-text search index for RDF literal values using Tantivy.
transaction: Transaction support for TDB storage engine
triple_buffer: Write buffer for batching triple insertions (v1.6.0).
triple_cache: Triple pattern cache: memoize repeated pattern lookups using LRU eviction (v1.1.0 round 15) Triple pattern cache: memoize repeated triple pattern lookups using LRU eviction.
vacuum: Database vacuuming and compaction: dead-space detection, fragmentation analysis, online vacuum (v2.0.0). Database vacuuming and compaction for TDB storage.
wal_archive: Write-Ahead Log Archiving for Point-in-Time Recovery
wal_compaction: WAL Compaction and Point-in-Time Recovery
wal_shipping: Continuous WAL Shipping for Disaster Recovery
write_batch: Atomic batch write operations with WAL integration (v1.8.0). Atomic batch write operations for TDB store.

Constants§

DEFAULT_BUFFER_POOL_SIZE: Default buffer pool size (1000 pages = 4MB)
DEFAULT_PAGE_SIZE: Default page size (4KB)
VERSION: TDB storage engine version

Crate oxirs_tdb

Crate oxirs_tdb Copy item path

§OxiRS TDB - Apache Jena TDB/TDB2 Compatible Storage Engine

§Features

§Quick Start

§Architecture

§See Also

Re-exports§

Modules§

Constants§

Crate oxirs_tdb