Crate kimberlite

Expand description

§Kimberlite

Compliance-native database for regulated industries.

Kimberlite is built on a replicated append-only log with deterministic projection to a custom storage engine. This provides:

Correctness by design - Ordered log → deterministic apply → snapshot
Full audit trail - Every mutation is captured in the immutable log
Point-in-time recovery - Replay from any offset
Compliance by construction - Built-in durability and encryption

§Architecture

┌─────────────────────────────────────────────────────────────┐
│                          Kimberlite                             │
│  ┌─────────┐   ┌───────────┐   ┌──────────┐   ┌──────────┐ │
│  │   Log   │ → │  Kernel   │ → │  Store   │ → │  Query   │ │
│  │(append) │   │(pure FSM) │   │(B+tree)  │   │  (SQL)   │ │
│  └─────────┘   └───────────┘   └──────────┘   └──────────┘ │
└─────────────────────────────────────────────────────────────┘

§Quick Start

use kimberlite::{Kimberlite, TenantId, DataClass};

// Open database
let db = Kimberlite::open("./data")?;

// Get tenant handle
let tenant = db.tenant(TenantId::new(1));

// Create a stream
let stream_id = tenant.create_stream("events", DataClass::Public)?;

// Append events
tenant.append(stream_id, vec![b"event1".to_vec(), b"event2".to_vec()], Offset::ZERO)?;

// Query (point-in-time support)
let results = tenant.query("SELECT * FROM events LIMIT 10", &[])?;

§Modules

SDK Layer: Kimberlite, TenantHandle - Main API
Foundation: Types, crypto, storage primitives
Query: SQL subset for compliance lookups

Structs§

AbacDecision: The result of evaluating an access request against a policy.
AbacPolicy: An Attribute-Based Access Control policy.
AbacRule: A single access control rule within a policy.
AccessPolicy: Access control policy.
AuditQuery: Query filter for the audit log.
BTreeStore: B+tree-based projection store implementation.
BreachDetector: Automated breach detector implementing HIPAA Section 164.404 and GDPR Article 33.
BreachEvent: A single breach detection event with full audit trail metadata.
BreachReport: Comprehensive breach report for regulatory notification.
BreachThresholds: Configurable thresholds for breach detection indicators.
ChainHash: A 32-byte SHA-256 hash used for chaining records.
ColumnDef: Definition of a table column.
ColumnFilter: Filter for column-level access control (field-level security).
ColumnName: SQL column name.
ComplianceAuditEvent: A single audit event with full context.
ComplianceAuditLog: Immutable, append-only audit log for compliance operations.
ComplianceReport: Complete compliance report
ConsentRecord: A single consent record
ConsentTracker: Consent tracker manages all consent records
Directory: Routes stream placements to VSR replication groups.
EnvironmentAttributes: Attributes describing the environment/context of the access request.
ErasureEngine: Engine managing the lifecycle of GDPR Article 17 erasure requests.
ErasureRequest: An erasure request tracking the lifecycle of a right-to-erasure invocation.
ExportEngine: Engine for GDPR Article 20 data portability exports
ExportRecord: A single record included in a portability export
FieldKey: A key for encrypting a specific field, derived from a tenant key.
FieldMask: Describes how a single column should be masked.
GroupId: Unique identifier for a replication group.
KAnonymityResult: Result of a k-anonymity check.
Key: A key in the projection store.
Kimberlite: The main Kimberlite database handle.
KimberliteConfig: Configuration for opening a Kimberlite database.
MaskingPolicy: A collection of field masks forming a complete masking policy.
Offset: Position of an event within a stream.
PermissionSet: Set of permissions granted to a role.
PolicyEnforcer: Policy enforcement engine.
PortabilityExport: Metadata about a completed data portability export
ProofCertificate: Proof certificate embedding verification metadata
QueryEngine: Query engine for executing SQL against a projection store.
QueryResult: Result of executing a query.
Record: A single record in the event log.
Requirement: A single compliance requirement
ResourceAttributes: Attributes describing the resource being accessed.
ReversibleToken: Encrypted token that can be reversed with the key.
RowFilter: Filter for row-level security (RLS).
Schema: Schema registry mapping SQL names to store types.
SchemaBuilder: Builder for constructing schemas fluently.
StandardPolicies: Standard policies for each role.
State: The kernel’s in-memory state.
Storage: Append-only event log storage with checkpoint support and segment rotation.
StreamFilter: Filter for stream-level access control.
StreamId: Unique identifier for a stream within the system.
StreamMetadata: Metadata describing a stream’s configuration and current state.
StreamName: Human-readable name for a stream.
TableDef: Definition of a table in the schema.
TableId: Unique identifier for a table within the store.
TableName: SQL table name.
TenantHandle: A tenant-scoped handle for database operations.
TenantId: Unique identifier for a tenant (organization/customer).
Token: A deterministic token for consistent pseudonymization.
UserAttributes: Attributes describing the user making the access request.
WriteBatch: A batch of write operations to apply atomically.

Enums§

BreachError
BreachIndicator: Indicator that triggered a breach detection event.
BreachSeverity: Severity level for a breach event, ordered from lowest to highest.
BreachStatus: Status of a breach event through its lifecycle.
Command: A command to be applied to the kernel.
ComplianceAuditAction: Extended audit actions covering all compliance modules.
ComplianceError
ComplianceFramework: Compliance framework identifiers
ConsentError
ConsentScope: Scope of consent (what data is covered)
DataClass: Classification of data for compliance purposes.
DataType: SQL data types supported by the query engine.
DatePrecision: Precision level for date truncation.
DirectoryError: Errors that can occur during directory lookups.
Effect: An effect to be executed by the runtime.
EnforcementError: Error type for policy enforcement.
ErasureError
ErasureStatus: Current status of an erasure request.
ExecuteResult: Result of executing a DDL/DML statement.
ExemptionBasis: Legal basis for exemption from the right to erasure.
ExportError
ExportFormat: Machine-readable export format (GDPR Article 20 compliance)
GeoLevel: Geographic hierarchy levels for generalization.
KernelError: Errors that can occur when applying commands to the kernel.
KimberliteError: Errors that can occur during Kimberlite operations.
MaskStyle: Style for masking sensitive values.
MaskingError: Errors that can occur during masking operations.
MaskingStrategy: Strategy used to mask a field value.
Permission: Permission that can be granted to a role.
Placement: Placement policy for a stream.
PolicyEffect: The effect of a policy rule: allow or deny access.
ProofStatus: Status of a compliance requirement
Purpose: Lawful basis for data processing (GDPR Article 6)
QueryError: Errors that can occur during query parsing and execution.
RedactPattern: Pattern for partial redaction of known data formats.
Region: Geographic region for data placement.
Role: Role in the access control system.
RowFilterOperator: Operator for row-level security filters.
StorageError: Errors that can occur during storage operations.
StoreError: Errors that can occur during store operations.
Value: A typed SQL value.
WriteOp: A single write operation within a batch.

Traits§

CommandRouter: Hook installed by replicated runtimes (kimberlite-server in cluster mode) so Kimberlite::submit can route writes through VSR before they touch the local projection.
ProjectionStore: Trait for projection stores that maintain derived state from the log.

Functions§

apply_committed: Applies a committed command to the state, producing new state and effects.
chain_hash: Computes the next hash in the chain.
check_k_anonymity: Checks if a set of quasi-identifier combinations achieves k-anonymity.
decrypt_field: Decrypts a field value encrypted with encrypt_field.
encrypt_field: Encrypts a field value with randomized encryption.
generalize_age
generalize_numeric: Generalizes a numeric value into a range.
generalize_zip: Generalizes a ZIP code by preserving only the first N digits.
mask: Masks a string value according to the specified style.
redact: Returns None to represent a fully redacted value.
tokenize: Creates a deterministic token from a value.
truncate_date: Truncates a date to the specified precision.

Type Aliases§

Result: Result type for Kimberlite operations.
Row: A single result row.

Crate kimberlite

Crate kimberlite Copy item path

§Kimberlite

§Architecture

§Quick Start

§Modules

Structs§

Enums§

Traits§

Functions§

Type Aliases§

Crate kimberlite