Crate rivven_core 

Core storage, concurrency, and data-plane primitives for Rivven.

Synchronization: parking_lot vs tokio::sync

This crate deliberately uses parking_lot::{Mutex, RwLock} for in-memory data structures that are guarded by O(1) critical sections: HashMap lookups/inserts, atomic swaps, counter increments, and similar operations that complete in bounded constant time.

Why not tokio::sync? A tokio::sync::Mutex is optimised for holding a lock across .await points. When the lock body is purely synchronous and O(1), parking_lot avoids the future-boxing overhead and provides deterministic low-latency locking (spin → OS wait).

Ground rules enforced throughout the crate:

1. A parking_lot lock is never held across an .await boundary.
2. Every critical section executes in bounded constant time.
3. Where a lock must span an .await (e.g., file I/O), tokio::sync is used instead (see Segment::log_file).

Re-exports

pub use cedar_authz::AuthzContext;

pub use cedar_authz::AuthzDecision;

pub use cedar_authz::CedarAuthorizer;

pub use cedar_authz::CedarError;

pub use cedar_authz::CedarResult;

pub use cedar_authz::RivvenAction;

pub use cedar_authz::RivvenResource;

pub use backpressure::AdaptiveRateLimiter;

pub use backpressure::AdaptiveRateLimiterConfig;

pub use backpressure::AdaptiveStatsSnapshot;

pub use backpressure::BackpressureChannel;

pub use backpressure::ChannelStatsSnapshot;

pub use backpressure::CircuitBreaker;

pub use backpressure::CircuitBreakerConfig;

pub use backpressure::CircuitBreakerStatsSnapshot;

pub use backpressure::CircuitState;

pub use backpressure::CreditFlowControl;

pub use backpressure::CreditStatsSnapshot;

pub use backpressure::TokenBucket;

pub use backpressure::TokenBucketStatsSnapshot;

pub use backpressure::WindowedRateTracker;

pub use bloom::AdaptiveBatcher;

pub use bloom::BatchConfig;

pub use bloom::BatcherStats;

pub use bloom::BloomFilter;

pub use bloom::CountingBloomFilter;

pub use bloom::HyperLogLog;

pub use bloom::OffsetBloomFilter;

pub use buffer_pool::BufferChain;

pub use buffer_pool::BufferPool;

pub use buffer_pool::BufferPoolConfig;

pub use buffer_pool::PooledBuffer;

pub use buffer_pool::SizeClass;

pub use concurrent::AppendLogConfig;

pub use concurrent::AppendOnlyLog;

pub use concurrent::ConcurrentHashMap;

pub use concurrent::ConcurrentSkipList;

pub use concurrent::LockFreeQueue;

pub use concurrent::QueueStats;

pub use config::Config;

pub use error::Error;

pub use error::Result;

pub use idempotent::IdempotentProducerManager;

pub use idempotent::IdempotentProducerStats;

pub use idempotent::PartitionProducerState;

pub use idempotent::ProducerEpoch;

pub use idempotent::ProducerId;

pub use idempotent::ProducerMetadata;

pub use idempotent::SequenceNumber;

pub use idempotent::SequenceResult;

pub use idempotent::NO_SEQUENCE;

pub use message::Message;

pub use offset::OffsetManager;

pub use partition::Partition;

pub use quota::EntityQuotaStats;

pub use quota::QuotaConfig;

pub use quota::QuotaEntity;

pub use quota::QuotaEntityType;

pub use quota::QuotaManager;

pub use quota::QuotaResult;

pub use quota::QuotaStats;

pub use quota::QuotaStatsSnapshot;

pub use quota::QuotaType;

pub use quota::DEFAULT_CONSUME_BYTES_RATE;

pub use quota::DEFAULT_PRODUCE_BYTES_RATE;

pub use quota::DEFAULT_REQUEST_RATE;

pub use quota::UNLIMITED;

pub use storage::ColdStorageBackend;

pub use storage::ColdStorageConfig;

pub use storage::HotTier;

pub use storage::HotTierStats;

pub use storage::LocalFsColdStorage;

pub use storage::SegmentMetadata;

pub use storage::SegmentSyncPolicy;

pub use storage::StorageTier;

pub use storage::TieredStorage;

pub use storage::TieredStorageConfig;

pub use storage::TieredStorageStats;

pub use storage::TieredStorageStatsSnapshot;

pub use storage::WarmTier;

pub use storage::WarmTierStats;

pub use topic::Topic;

pub use topic::TopicManager;

pub use topic_config::CleanupPolicy;

pub use topic_config::CompressionType;

pub use topic_config::ConfigValue;

pub use topic_config::TopicConfig;

pub use topic_config::TopicConfigManager;

pub use topic_config::DEFAULT_MAX_MESSAGE_BYTES;

pub use topic_config::DEFAULT_RETENTION_BYTES;

pub use topic_config::DEFAULT_RETENTION_MS;

pub use topic_config::DEFAULT_SEGMENT_BYTES;

pub use topic_config::DEFAULT_SEGMENT_MS;

pub use transaction::AbortedTransaction;

pub use transaction::AbortedTransactionIndex;

pub use transaction::IsolationLevel;

pub use transaction::PendingWrite;

pub use transaction::Transaction;

pub use transaction::TransactionCoordinator;

pub use transaction::TransactionId;

pub use transaction::TransactionMarker;

pub use transaction::TransactionOffsetCommit;

pub use transaction::TransactionPartition;

pub use transaction::TransactionResult;

pub use transaction::TransactionState;

pub use transaction::TransactionStats;

pub use transaction::TransactionStatsSnapshot;

pub use transaction::DEFAULT_TRANSACTION_TIMEOUT;

pub use transaction::MAX_PENDING_TRANSACTIONS;

pub use vectorized::BatchDecoder;

pub use vectorized::BatchEncoder;

pub use vectorized::BatchMessage;

pub use vectorized::BatchProcessor;

pub use vectorized::RecordBatch;

pub use vectorized::RecordBatchIter;

pub use wal::GroupCommitWal;

pub use wal::RecordType;

pub use wal::SyncMode;

pub use wal::TxnWalPayload;

pub use wal::WalConfig;

pub use wal::WalReader;

pub use wal::WalRecord;

pub use wal::WalStatsSnapshot;

pub use zero_copy::BufferRef;

pub use zero_copy::BufferSlice;

pub use zero_copy::ConsumedMessage;

pub use zero_copy::SmallVec;

pub use zero_copy::ZeroCopyBuffer;

pub use zero_copy::ZeroCopyBufferPool;

pub use zero_copy::ZeroCopyConsumer;

pub use zero_copy::ZeroCopyProducer;

pub use async_io::AsyncFile;

pub use async_io::AsyncIo;

pub use async_io::AsyncIoConfig;

pub use async_io::AsyncSegment;

pub use async_io::BatchBuilder;

pub use auth::AclEntry;

pub use auth::AuthConfig;

pub use auth::AuthError;

pub use auth::AuthManager;

pub use auth::AuthResult;

pub use auth::AuthSession;

pub use auth::PasswordHash;

pub use auth::Permission;

pub use auth::Principal;

pub use auth::PrincipalType;

pub use auth::ResourceType;

pub use auth::Role;

pub use auth::SaslPlainAuth;

pub use auth::SaslScramAuth;

pub use auth::ScramState;

pub use auth::Session;

pub use io_uring::is_io_uring_available;

pub use io_uring::AsyncReader;

pub use io_uring::BatchExecutor;

pub use io_uring::BatchReadResult;

pub use io_uring::BatchStats;

pub use io_uring::BlockingWriter;

pub use io_uring::IoBatch;

pub use io_uring::IoOperation;

pub use io_uring::IoUringConfig;

pub use io_uring::IoUringStats;

pub use io_uring::IoUringStatsSnapshot;

pub use io_uring::PortableWalWriter;

pub use io_uring::SegmentReader;

pub use service_auth::ApiKey;

pub use service_auth::AuthMethod;

pub use service_auth::ServiceAccount;

pub use service_auth::ServiceAuthConfig;

pub use service_auth::ServiceAuthError;

pub use service_auth::ServiceAuthManager;

pub use service_auth::ServiceAuthRequest;

pub use service_auth::ServiceAuthResponse;

pub use service_auth::ServiceAuthResult;

pub use service_auth::ServiceSession;

pub use validation::ValidationError;

pub use validation::Validator;

pub use tls::certificate_fingerprint;

pub use tls::generate_self_signed;

pub use tls::load_certificates;

pub use tls::load_private_key;

pub use tls::CertificateSource;

pub use tls::CertificateWatcher;

pub use tls::MtlsMode;

pub use tls::PrivateKeySource;

pub use tls::TlsAcceptor;

pub use tls::TlsClientStream;

pub use tls::TlsConfig;

pub use tls::TlsConfigBuilder;

pub use tls::TlsConnector;

pub use tls::TlsError;

pub use tls::TlsIdentity;

pub use tls::TlsResult;

pub use tls::TlsSecurityAudit;

pub use tls::TlsServerStream;

pub use tls::TlsVersion;

Modules

async_io

Cross-Platform Async I/O Layer

auth

Authentication and Authorization (RBAC/ACL) for Rivven

backpressure

Backpressure and Flow Control

bloom

High-Performance Bloom Filters and Probabilistic Data Structures

buffer_pool

Zero-Copy Buffer Pool

cedar_authz

Cedar-based Authorization Engine for Rivven

compression

High-performance compression layer for Rivven

concurrent

Lock-Free Concurrent Data Structures

config

consumer_group

Consumer Group Coordination

crypto

Shared cryptographic primitives for Rivven encryption subsystems.

encryption

Encryption at rest for Rivven data

error

hash

Canonical hash functions shared across client and server.

idempotent

Idempotent Producer Support

io_uring

High-performance async I/O with optional io_uring backend

message

metrics

Unified metrics infrastructure for Rivven

offset

partition

quota

Per-Principal Quotas (Kafka Parity)

serde_utils

service_auth

Service-to-Service Authentication for Rivven

storage

tls

Production-grade TLS/mTLS infrastructure for Rivven.

topic

topic_config

Topic Configuration Management

transaction

Native Transaction Support

validation

Input validation for Rivven operations

vectorized

Vectorized Batch Processing

wal

Group Commit Write-Ahead Log (WAL)

zero_copy

Zero-Copy Producer/Consumer API