crdt-lite 0.5.0

A lightweight, column-based CRDT implementation in Rust
Documentation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
# CRDT-Lite

> [!WARNING]
> This project is in early development and not intended for production use.

A lightweight implementation of Conflict-free Replicated Data Types (CRDTs) in both **Rust** and **C++**. CRDT-Lite provides generic CRDT structures for building distributed systems requiring eventual consistency.

CRDT-Lite is currently being used in [Formabble](https://formabble.com), a collaborative game engine, and will be integrated into a derived product that we will announce soon.

## What's Inside

This library includes two CRDT implementations:

1. **Column-Based CRDT** (Rust: `src/lib.rs`, C++: `crdt.hpp`) - Generic key-value store with fine-grained column-level conflict resolution
2. **Text CRDT** (C++: `text_crdt.hpp`) - Line-based collaborative text editor with fractional positioning

Both Rust and C++ implementations share the same core algorithms and maintain API compatibility.

## Features

### Column-Based CRDT

- **Generic over key/value types** - Use any type that meets the requirements
-**Fine-grained conflict resolution** - Column-level (field-level) versioning
-**Last-Write-Wins semantics** - Deterministic conflict resolution using logical clocks
-**Tombstone-based deletion** - Proper handling of deleted records across nodes
-**Parent-child hierarchies** - Support for layered CRDT structures
-**Custom merge rules** - Implement your own conflict resolution strategies
-**Change compression** - Optimized transmission by removing redundant changes
-**Efficient sync** - Track version boundaries to skip unchanged records

### Text CRDT

- **Line-based editing** - Collaborative text with line granularity
-**Fractional positioning** - Infinite density between any two lines
-**Multiple merge strategies** - Last-Write-Wins and Both-Writes-Win support
-**Dual indexing** - Fast position-ordered iteration and ID-based lookup
-**Conflict detection** - Preserve or resolve concurrent edits
-**Change streaming** - Real-time collaboration support

### Persistence Layer (Rust)

- **Write-Ahead Log (WAL)** - Durable operation log with automatic rotation
-**Snapshots** - Periodic full-state checkpoints with configurable thresholds
-**Crash Recovery** - Automatic recovery from snapshot + WAL replay
-**Hook System** - Callbacks for post-operation, snapshot, and WAL sealing events
-**Batch Collector** - Accumulate changes for efficient network broadcasting
-**Upload Tracking** - Track uploaded snapshots/WAL segments for safe cleanup
-**Auto-cleanup** - Configurable retention policies to prevent disk bloat
-**Windows Support** - Proper file handle management for strict locking

### Platform Support

- **`no_std` compatible** - Works in embedded systems and other `no_std` environments (requires `alloc`)
-**Zero-cost abstractions** - Uses hashbrown (same HashMap as std) in `no_std` mode
-**Optional std** - std is enabled by default for backwards compatibility

#### Using in `no_std` Environments

The Rust implementation supports `no_std` environments with allocator support.

**Requirements:**
- **`alloc` crate required**: This library needs an allocator for `Vec`, `HashMap`, `Arc`, etc.
- **Example setup**:
  ```rust
  #![no_std]
  extern crate alloc;

  use crdt_lite::CRDT;
  // ... use as normal
  ```

**Cargo.toml configuration:**

```toml
[dependencies]
# For no_std with basic CRDT functionality (requires alloc feature)
crdt-lite = { version = "0.5", default-features = false, features = ["alloc"] }

# For no_std with JSON serialization
crdt-lite = { version = "0.5", default-features = false, features = ["alloc", "json"] }

# For no_std with binary serialization (bincode)
crdt-lite = { version = "0.5", default-features = false, features = ["alloc", "binary"] }

# For standard environments (default, uses std::collections::HashMap)
crdt-lite = { version = "0.5", features = ["json"] }
```

**Implementation Notes:**
- `no_std` mode uses `hashbrown::HashMap`, which is the same underlying implementation that `std::collections::HashMap` uses (identical performance)
- The `std` feature is enabled by default for backwards compatibility
- The `alloc` feature is required for `no_std` environments and pulls in `hashbrown` only when needed
- When using std (default), `hashbrown` is not compiled, reducing dependency bloat
- Binary serialization uses `bincode` 2.0 which has full `no_std` support

### NodeId Type Configuration

By default, `NodeId` is `u64`. For applications using UUID-based node identifiers, enable the `node-id-u128` feature:

```toml
[dependencies]
crdt-lite = { version = "0.5", features = ["node-id-u128"] }
```

**Why u128?**
- `u64` provides 2^64 (~18 quintillion) unique IDs
- With random UUID generation, birthday paradox makes collisions likely after ~4 billion IDs
- `u128` provides 2^128 unique IDs, eliminating collision concerns for UUID-based systems
- C++ implementation allows customizing `CrdtNodeId` via preprocessor (see `crdt.hpp`)

## Quick Start

### Rust Implementation

```bash
cargo add crdt-lite  # (when published)
# or add to Cargo.toml: crdt-lite = "0.5"
```

```rust
use crdt_lite::{CRDT, DefaultMergeRule};

// Create two CRDT nodes
let mut node1: CRDT<String, String, String> = CRDT::new(1, None);
let mut node2: CRDT<String, String, String> = CRDT::new(2, None);

// Node 1: Insert data
let changes1 = node1.insert_or_update(
    &"user1".to_string(),
    vec![
        ("name".to_string(), "Alice".to_string()),
        ("age".to_string(), "30".to_string()),
    ],
);

// Node 2: Insert conflicting data
let changes2 = node2.insert_or_update(
    &"user1".to_string(),
    vec![
        ("name".to_string(), "Bob".to_string()),
        ("age".to_string(), "25".to_string()),
    ],
);

// Merge changes bidirectionally
let merge_rule = DefaultMergeRule;
node1.merge_changes(changes2, &merge_rule);
node2.merge_changes(changes1, &merge_rule);

// Both nodes converge to same state (node2 wins due to higher node_id)
assert_eq!(node1.get_data(), node2.get_data());
```

**Run tests:**
```bash
cargo test
```

### C++ Implementation

**Compile and run:**
```bash
# Column CRDT tests
g++ -std=c++20 -o crdt tests.cpp && ./crdt

# Text CRDT tests
g++ -std=c++20 -o text_crdt test_text_crdt.cpp && ./text_crdt
```

**Column CRDT example:**
```cpp
#include "crdt.hpp"

CRDT<std::string, std::string> node1(1);
CRDT<std::string, std::string> node2(2);

// Node 1: Insert data
std::vector<Change<std::string, std::string>> changes1;
node1.insert_or_update("user1", changes1,
    std::make_pair("name", "Alice"),
    std::make_pair("age", "30")
);

// Node 2: Insert conflicting data
std::vector<Change<std::string, std::string>> changes2;
node2.insert_or_update("user1", changes2,
    std::make_pair("name", "Bob"),
    std::make_pair("age", "25")
);

// Merge changes bidirectionally
node1.merge_changes(std::move(changes2));
node2.merge_changes(std::move(changes1));

// Both nodes converge
assert(node1.get_data() == node2.get_data());
```

**Text CRDT example:**
```cpp
#include "text_crdt.hpp"

TextCRDT<std::string> doc1(1);
TextCRDT<std::string> doc2(2);

// Both nodes insert lines concurrently
auto id1 = doc1.insert_line_at_end("Line from Node 1");
auto id2 = doc2.insert_line_at_end("Line from Node 2");

// Sync changes
uint64_t sync_version = 0;
auto changes1 = doc1.get_changes_since(sync_version);
auto changes2 = doc2.get_changes_since(sync_version);

doc2.merge_changes(changes1);
doc1.merge_changes(changes2);

// Both nodes have both lines
assert(doc1.line_count() == 2);
assert(doc2.line_count() == 2);
```

## Core Concepts

### Column-Based Design

Records are stored as maps of columns (field names) to values. Each column has independent version tracking:

```rust
// Rust
pub struct Record<C, V> {
  pub fields: HashMap<C, V>,
  pub column_versions: HashMap<C, ColumnVersion>,
  // Version boundaries for efficient sync
  pub lowest_local_db_version: u64,
  pub highest_local_db_version: u64,
}
```

**Why column-based?**
- Conflicts resolved per-field, not per-record
- Only changed columns need syncing
- Natural fit for structured data (forms, database records)

### Conflict Resolution

Conflicts are resolved deterministically using a three-tier comparison:

1. **Column Version** (per-field edit counter) - Higher wins
2. **Database Version** (global logical clock) - Higher wins
3. **Node ID** (unique node identifier) - Higher wins (tie-breaker)

This ordering ensures:
- **Field-level granularity**: Each field resolves independently
- **Causal ordering**: Logical clocks prevent out-of-order updates
- **Determinism**: All nodes converge to identical state

### Logical Clocks

Maintains causality using Lamport-style logical clocks:

```rust
impl LogicalClock {
  // Local operation
  pub fn tick(&mut self) -> u64 {
    self.time += 1;
    self.time
  }

  // Receive remote change
  pub fn update(&mut self, received_time: u64) -> u64 {
    self.time = self.time.max(received_time);
    self.time += 1;
    self.time
  }
}
```

**Important:** Always update clock on merge, even for rejected changes (prevents clock drift).

### Tombstone-Based Deletion

Deleted records are marked with tombstones rather than immediately removed:

```rust
pub struct TombstoneInfo {
  pub db_version: u64,
  pub node_id: NodeId,
  pub local_db_version: u64,
}
```

**⚠️ Critical: Tombstone Management**

Tombstones accumulate indefinitely unless compacted. To prevent memory exhaustion:

1. Track which versions have been acknowledged by **ALL nodes**
2. Call `compact_tombstones(min_acknowledged_version)` periodically
3. **Never compact early** - deleted records will reappear on nodes that haven't seen the deletion yet (zombie records)

### Field Deletion

Individual fields can be deleted from records without removing the entire record:

```rust
// Rust
let change = crdt.delete_field(&record_id, &"email".to_string());
```

```cpp
// C++
std::vector<Change<std::string, std::string>> changes;
bool success = crdt.delete_field(record_id, "email", changes);
```

**How it works:**
- Field is removed from `record.fields` map
- `ColumnVersion` entry **remains** in `record.column_versions` (acts as implicit field tombstone)
- Syncs to other nodes as `Change { col_name: Some("field"), value: None }`
- Field deletion is versioned like field updates (increments `col_version`)

**Distinguished from null values:**
- **Field deletion**: Field removed entirely from map
- **Null value**: Field exists in map with null/None value (if V = Option<T>)

### Fractional Positioning (Text CRDT)

Each line in the text CRDT has a position defined by a path of integers:

```cpp
struct FractionalPosition {
  std::vector<uint64_t> path;

  // Examples:
  // [10000]           - First line
  // [10000, 5000]     - Between first and second level
  // [20000]           - After [10000]
};
```

**Properties:**
- Total ordering with infinite density
- Can always insert between any two positions
- Automatically extends depth when space runs out

## Persistence Layer

The Rust implementation includes an optional persistence layer with WAL (Write-Ahead Log) and snapshots for durability and crash recovery.

### Quick Start

```toml
[dependencies]
crdt-lite = { version = "0.5", features = ["persist"] }
```

```rust
use crdt_lite::persist::{PersistedCRDT, PersistConfig};
use std::path::PathBuf;

// Open or create a persisted CRDT
let mut pcrdt = PersistedCRDT::<String, String, String>::open(
    PathBuf::from("./data"),
    1, // node_id
    PersistConfig::default(),
)?;

// Use like a normal CRDT - all operations are automatically persisted
pcrdt.insert_or_update(
    &"user1".to_string(),
    vec![
        ("name".to_string(), "Alice".to_string()),
        ("email".to_string(), "alice@example.com".to_string()),
    ],
)?;

// Changes are persisted to WAL and automatically recovered on crash
```

### Configuration

```rust
let config = PersistConfig {
    snapshot_threshold: 1000,      // Create snapshot every 1000 changes
    auto_cleanup_snapshots: Some(3), // Keep 3 most recent snapshots
    max_batch_size: Some(10000),    // Auto-flush batch at 10k changes
};
```

### Hook System

The persistence layer provides three types of hooks for integration with backup systems, network layers, etc:

#### Post-Operation Hook

Called after changes are written to WAL (before fsync). Use for broadcasting changes to other nodes:

```rust
use std::sync::mpsc::Sender;

let (tx, rx) = std::sync::mpsc::channel();

pcrdt.add_post_hook(Box::new(move |changes| {
    // Send to network layer for broadcast
    let _ = tx.send(changes.to_vec());
}));
```

#### Snapshot Hook

Called after a snapshot is created and sealed. Use for uploading to cloud storage:

```rust
pcrdt.add_snapshot_hook(Box::new(move |snapshot_path| {
    let path = snapshot_path.clone();

    // Spawn async upload task
    tokio::spawn(async move {
        let data = tokio::fs::read(&path).await.unwrap();
        // Upload to R2, S3, etc.
        // r2.put(format!("snapshots/{}", path.file_name()?), data).await?;

        // Mark as uploaded for safe cleanup
        // pcrdt.lock().unwrap().mark_snapshot_uploaded(path);
    });
}));
```

#### WAL Segment Hook

Called after a WAL segment is sealed (rotated). Use for archival:

```rust
pcrdt.add_wal_segment_hook(Box::new(move |segment_path| {
    let path = segment_path.clone();

    tokio::spawn(async move {
        let data = tokio::fs::read(&path).await.unwrap();
        // Archive to cloud storage
        // r2.put(format!("wal/{}", path.file_name()?), data).await?;
    });
}));
```

### Batch Collector

Accumulate changes for efficient network broadcasting:

```rust
// Perform multiple operations
pcrdt.insert_or_update(&"user1".to_string(), fields1)?;
pcrdt.insert_or_update(&"user2".to_string(), fields2)?;

// Collect all changes since last call
let batch = pcrdt.take_batch();

// Broadcast to other nodes
broadcast_to_peers(batch);
```

**Auto-flush protection:** By default, the batch is cleared when it reaches 10,000 changes to prevent OOM. Call `take_batch()` regularly to avoid losing changes.

### Upload Tracking and Cleanup

For cloud backup workflows, track which files have been uploaded before deleting them:

```rust
// After successful upload
pcrdt.mark_snapshot_uploaded(snapshot_path);
pcrdt.mark_wal_segment_uploaded(segment_path);

// Cleanup only uploaded files (safe - won't lose data)
pcrdt.cleanup_old_snapshots(2, true)?; // Keep 2, require uploaded
pcrdt.cleanup_old_wal_segments(5, true)?; // Keep 5, require uploaded

// Or cleanup all old files (after snapshot creation)
pcrdt.cleanup_old_snapshots(2, false)?; // Unconditional cleanup
```

### Crash Recovery

Recovery is automatic on `open()`:

```rust
// After crash, simply open again
let pcrdt = PersistedCRDT::<String, String, String>::open(
    PathBuf::from("./data"),
    1,
    PersistConfig::default(),
)?;

// All data is recovered from snapshot + WAL replay
```

### Durability Guarantees

**Design choice:** WAL writes are flushed to OS but NOT fsynced per-operation for CRDT performance:

- ✅ Changes written to WAL before broadcast (minimizes network latency)
- ✅ Changes fsynced during snapshot creation (periodic durability)
- ⚠️ Crash before snapshot = possible local data loss
- ✅ Peers have the data from broadcast (system-wide consistency maintained)

This design prioritizes CRDT convergence over local durability. For stronger durability guarantees, decrease `snapshot_threshold`.

### Tombstone Compaction

After all nodes acknowledge a version, compact tombstones to reclaim memory:

```rust
// Track acknowledgments from all nodes
let min_acknowledged = get_min_ack_from_all_nodes();

// Atomically compact tombstones and cleanup WAL
pcrdt.compact_tombstones(min_acknowledged)?;
```

**Critical:** Only compact after ALL nodes acknowledge the version, or deleted records will reappear (zombie records).

### Examples

See working examples in the repository:
- `examples/persistence_example.rs` - Basic persistence with hooks
- `examples/r2_backup_example.rs` - Cloud backup workflow with R2

Run with:
```bash
cargo run --example persistence_example --features persist
cargo run --example r2_backup_example --features persist
```

## Synchronization

### Basic Sync Protocol

```rust
// Rust
let changes = node1.get_changes_since(last_db_version);
node2.merge_changes(changes, &DefaultMergeRule);

// Optionally exclude changes from specific nodes
let excluding = HashSet::from([node2_id]);
let changes = node1.get_changes_since_excluding(last_db_version, &excluding);
```

```cpp
// C++
auto changes = node1.get_changes_since(last_db_version);
node2.merge_changes(std::move(changes));

// Or use the helper function
uint64_t last_sync = 0;
sync_nodes(node1, node2, last_sync);
```

### Change Compression

When syncing with parent-child CRDTs or after accumulating many changes:

```rust
// Rust
CRDT::<String, String, String>::compress_changes(&mut changes);
```

```cpp
// C++
CRDT<K, V>::compress_changes(changes);
```

This removes redundant operations (O(n log n)):
- Superseded field updates (same record+column, older version)
- Field updates replaced by record deletions

## Advanced Features

### Parent-Child Hierarchies

Create temporary overlays or transaction isolation:

```rust
// Rust
use std::sync::Arc;

let parent = Arc::new(CRDT::<String, String, String>::new(1, None));
let child = CRDT::new(2, Some(parent.clone()));

// Child sees parent data but maintains separate modifications
```

```cpp
// C++
auto parent = std::make_shared<CRDT<K, V>>(1);
CRDT<K, V> child(2, parent);

// Generate inverse changes to undo child's work
auto revert_changes = child.revert();

// Compute difference between two CRDTs
auto diff = child.diff(other_crdt);
```

### Custom Merge Rules

```rust
// Rust
struct CustomMergeRule;

impl<K, C, V> MergeRule<K, C, V> for CustomMergeRule {
    fn should_accept(
        &self,
        local_col: u64, local_db: u64, local_node: NodeId,
        remote_col: u64, remote_db: u64, remote_node: NodeId,
    ) -> bool {
        // Custom logic here
        remote_col > local_col
    }
}

crdt.merge_changes(changes, &CustomMergeRule);
```

```cpp
// C++
template <typename K, typename V, typename Context = void>
struct CustomMergeRule {
  constexpr bool operator()(
      const Change<K, V> &local,
      const Change<K, V> &remote) const {
    // Custom logic here
    return remote.col_version > local.col_version;
  }
};

crdt.merge_changes<false, void, CustomMergeRule<K, V>>(
    std::move(changes), false, CustomMergeRule<K, V>()
);
```

### Text CRDT Merge Strategies

**Last-Write-Wins (default):**
```cpp
doc.merge_changes(remote_changes);
```

**Both-Writes-Win (preserve conflicts):**
```cpp
BothWritesWinMergeRule<std::string, std::string> bww;
doc.merge_changes_with_rule(remote_changes, bww);

// Check for conflicts
auto line = doc.get_line(line_id);
if (line->has_conflicts()) {
  auto all_versions = line->get_all_versions();
  // Display all concurrent edits to user
}
```

**⚠️ Auto-Merge (EXPERIMENTAL - DO NOT USE):**
The `AutoMergingTextRule` is currently broken and violates CRDT convergence guarantees. See `CLAUDE.md` for details.

## Security and DoS Protection

### Trust Model

⚠️ **This is a data structure library**, not a complete distributed system. Security must be implemented at higher layers:

- No authentication (accepts any changes)
- No authorization (no access control)
- Assumes all nodes are non-malicious

### DoS Mitigation Strategies

1. **Tombstone Accumulation**
   - Track tombstone count: `crdt.tombstone_count()`
   - Set application-level limits
   - Compact periodically after all nodes acknowledge a version

2. **Resource Exhaustion**
   - Implement rate limiting on operations
   - Validate and limit key/value sizes
   - Set maximum records/columns per record

3. **Clock Manipulation**
   - Malicious nodes can set high `db_version` to win all conflicts
   - Use authenticated logical clocks in production

### Production Recommendations

1. **Network Layer**: Use TLS/encryption for change transmission
2. **Authentication**: Verify node identity (HMAC, digital signatures)
3. **Rate Limiting**: Per-node operation limits
4. **Input Validation**: Sanitize and limit sizes
5. **Monitoring**: Track tombstone growth and memory usage
6. **Thread Safety**: Use `Arc<Mutex<CRDT>>` in Rust, external locks in C++

## Performance Characteristics

### Time Complexity

| Operation | Average Case | Notes |
|-----------|-------------|-------|
| `insert_or_update` | O(n) | n = number of fields |
| `delete_field` | O(1) | HashMap field removal |
| `delete_record` | O(1) | HashMap record removal |
| `merge_changes` | O(c) | c = number of changes |
| `get_changes_since` | O(r × f) | r = records, f = fields (optimized with version bounds) |
| `compress_changes` | O(n log n) | Uses unstable sort for better performance |
| `compact_tombstones` | O(t) | t = number of tombstones |

### Memory Efficiency

- HashMap-based storage: O(1) average case lookups
- Version boundaries: Skip unchanged records during sync
- Change compression: Remove redundant operations
- Tombstone compaction: Prevent unbounded growth

## Limitations

- **Thread Safety**: Not thread-safe; external synchronization required
- **Network Transport**: Not included; implement your own sync protocol
- **Text CRDT**: Auto-merge feature is incomplete (use BWW instead)
- **No Encryption**: Implement at application/network layer

## Migration Between Languages

| C++ | Rust |
|-----|------|
| `CRDT<K, V> crdt(node_id);` | `let mut crdt = CRDT::<K, C, V>::new(node_id, None);` |
| `crdt.insert_or_update(id, changes, pair1, pair2);` | `let changes = crdt.insert_or_update(&id, vec![pair1, pair2]);` |
| `crdt.delete_field(id, "field", changes);` | `if let Some(change) = crdt.delete_field(&id, &"field") { ... }` |
| `crdt.delete_record(id, changes);` | `if let Some(change) = crdt.delete_record(&id) { ... }` |
| `crdt.merge_changes(std::move(changes));` | `crdt.merge_changes(changes, &DefaultMergeRule);` |
| `auto* record = crdt.get_record(id);` | `let record = crdt.get_record(&id);` |

## Documentation

- **`CLAUDE.md`** - Comprehensive technical documentation for developers (and Claude!)
- See inline documentation in source files

## Future Enhancements

### Planned
- [ ] Tombstone garbage collection improvements
- [ ] Custom merge functions for specialized use cases
- [ ] Text CRDT: Fix auto-merge algorithm
- [ ] Text CRDT: Implement move operations
- [ ] Text CRDT: Position rebalancing

### Rust-Specific Possibilities
- [ ] `async` support for network operations
- [ ] Optional resource limits (max records, max tombstones)

### Completed
- [x] `serde` integration for serialization (v0.2.0)
- [x] WebAssembly support via `no_std` + `alloc` (v0.2.0)
- [x] Persistence layer with WAL and snapshots (v0.5.0)
- [x] Hook system for post-operation, snapshot, and WAL events (v0.5.0)
- [x] Batch collector for efficient network broadcasting (v0.5.0)

## Testing

```bash
# Rust - All tests
cargo test

# Rust - Persistence layer tests
cargo test --features persist

# C++ - Column CRDT
g++ -std=c++20 tests.cpp -o tests && ./tests

# C++ - Text CRDT
g++ -std=c++20 test_text_crdt.cpp -o test_text_crdt && ./test_text_crdt
```

## Contributing

Contributions are welcome! Please ensure:

**Rust:**
- All tests pass (`cargo test`)
- Code is formatted (`cargo fmt`)
- No clippy warnings (`cargo clippy`)
- Maintain feature parity with C++

**C++:**
- Requires C++20 compatible compiler
- All tests pass
- Maintain feature parity with Rust

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

## Related Projects

- **[crdt-sqlite]https://github.com/sinkingsugar/crdt-sqlite** - SQLite wrapper with CRDT synchronization
  - Persistent storage for CRDT-backed applications
  - Automatic change tracking via SQLite triggers
  - Normal SQL INSERT/UPDATE/DELETE syntax (no special APIs required)
  - Cross-platform support (Linux, macOS, Windows)
  - Use crdt-lite for: in-memory state, game sync, real-time collaboration
  - Use crdt-sqlite for: persistent storage, database-backed apps, SQLite integration

## References

### CRDT Theory
- [A comprehensive study of CRDTs - Shapiro et al.]https://hal.inria.fr/inria-00555588
- [Conflict-free Replicated Data Types]https://arxiv.org/abs/1805.06358

### Text CRDTs
- [LSEQ: Adaptive Structure for Sequences]https://hal.archives-ouvertes.fr/hal-00921633
- [Logoot: Scalable Optimistic Replication]https://hal.inria.fr/inria-00432368
- [Fractional Indexing - Figma]https://www.figma.com/blog/realtime-editing-of-ordered-sequences/

### Logical Clocks
- [Time, Clocks, and Ordering - Lamport]https://lamport.azurewebsites.net/pubs/time-clocks.pdf

---

**CRDT-Lite** offers a streamlined approach to conflict-free replicated data types, balancing simplicity and efficiency. By focusing on fine-grained conflict resolution and deterministic merge semantics, CRDT-Lite is well-suited for applications requiring scalability and low overhead in distributed environments.