commy-sdk-rust 0.1.0

# Hybrid SDK Migration Guide


## Overview


The Commy Rust SDK has been extended with hybrid architecture support. **Old code still works** - this is a backwards-compatible enhancement. New code can optionally use the hybrid API for better performance and unified local/remote experience.

## Key Changes


### What Changed

- ✅ **New:** Hybrid local/remote abstraction  
- ✅ **New:** SIMD-accelerated change detection
- ✅ **New:** Virtual variable file model
- ✅ **New:** File watcher integration
- ✅ **Preserved:** All existing API methods
- ✅ **Preserved:** Backward compatibility

### What Didn't Change

- ❌ Connection model (still WSS)
- ❌ Authentication (still same)  
- ❌ Wire protocol (still backwards-compatible)
- ❌ Variable storage (still same memory layout)

## Migration Paths


### Path 1: No Changes Required (Lazy Migration)


Your existing code continues to work exactly as before:

```rust
// Old code - still works unchanged!
let client = Client::new("wss://server:9000");
client.connect().await?;
client.authenticate("tenant", creds).await?;

let data = client.read_variable("tenant", "service", "variable").await?;
client.write_variable("tenant", "service", "variable", &new_data).await?;

// All original methods still available
client.list_services("tenant").await?;
client.get_service_info("tenant", "service").await?;
```

**Pros:**
- Zero code changes
- Gradual adoption possible
- Lower risk

**Cons:**
- No access to hybrid benefits
- May have higher latency for local clients
- Misses SIMD optimization

**Best for:** Existing deployments, third-party code

### Path 2: Minimal Adoption (Recommended)


Add hybrid infrastructure initialization but keep existing code:

```rust
// Set up hybrid layer once
let client = Client::new("wss://server:9000");
client.connect().await?;
client.authenticate("tenant", creds).await?;
client.init_file_watcher().await?;  // ← NEW: Initialize hybrid
client.start_file_monitoring().await?; // ← NEW: Enable monitoring

// Keep using old API - now optimized!
let data = client.read_variable("tenant", "service", "variable").await?;
client.write_variable("tenant", "service", "variable", &new_data).await?;

// Optional: Listen for changes proactively
if let Some(event) = client.try_get_file_change().await? {
    println!("Service {} updated: {:?}", event.service_id, event.changed_variables);
}
```

**Pros:**
- Minimal code changes (3 new lines)
- Automatic local optimization when available
- File monitoring available if needed
- Backward compatible

**Cons:**
- Doesn't fully leverage hybrid model
- Still using old API

**Best for:** Existing codebases, quick optimization

### Path 3: Full Migration (Best Performance)


Adopt new hybrid API for maximum benefits:

```rust
// Setup - exactly like Path 2
let client = Client::new("wss://server:9000");
client.connect().await?;
client.authenticate("tenant", creds).await?;
client.init_file_watcher().await?;

// Now use new hybrid API
let vf = client.get_virtual_service_file("tenant", "service").await?.unwrap();

// Register variables once
let meta = VariableMetadata::new("user_id".to_string(), 0, 8, 1);
vf.register_variable(meta).await?;

// High-performance variable access
vf.write_variable("user_id", &[0, 0, 0, 0, 0, 0, 0, 123]).await?;
let data = vf.read_variable_slice("user_id").await?;

// Monitor changes
client.start_file_monitoring().await?;
while let Some(event) = client.wait_for_file_change().await? {
    for var_name in event.changed_variables {
        let updated = vf.read_variable_slice(&var_name).await?;
        process_update(&var_name, &updated);
    }
}
```

**Pros:**
- Maximum performance (especially local)
- Full hybrid capabilities
- SIMD acceleration
- Change monitoring

**Cons:**
- Requires API refactoring
- Different abstraction level
- More setup code

**Best for:** New code, performance-critical paths

## API Comparison


### Old API


```rust
// Single-call operations
async fn read_variable(
    &self,
    tenant: &str,
    service: &str,
    variable: &str,
) -> Result<Vec<u8>>;

async fn write_variable(
    &self,
    tenant: &str,
    service: &str,
    variable: &str,
    data: &[u8],
) -> Result<()>;

async fn list_variables(
    &self,
    tenant: &str,
    service: &str,
) -> Result<Vec<String>>;
```

### New Hybrid API


```rust
// Setup: Get virtual file once
async fn get_virtual_service_file(
    &self,
    tenant: &str,
    service: &str,
) -> Result<Option<Arc<VirtualVariableFile>>>;

// Register variables with metadata
async fn register_variable(
    &self,
    metadata: VariableMetadata,
) -> Result<()>;

// High-performance operations
async fn read_variable_slice(
    &self,
    variable: &str,
) -> Result<Vec<u8>>;

async fn write_variable(
    &self,
    variable: &str,
    data: &[u8],
) -> Result<()>;

// Change detection
async fn changed_variables(&self) -> Vec<String>;
async fn find_changed_variables_from_diff(
    &self,
    byte_ranges: &[(u64, u64)],
) -> Result<Vec<String>>;
```

## Common Refactoring Patterns


### Pattern 1: Replace Read/Write Loop


**Before:**
```rust
for var_name in variable_names {
    let val = client.read_variable(tenant, service, &var_name).await?;
    process_value(&var_name, &val);
}
```

**After:**
```rust
let vf = client.get_virtual_service_file(tenant, service).await?.unwrap();

for var_name in variable_names {
    let val = vf.read_variable_slice(&var_name).await?;
    process_value(&var_name, &val);
}
```

**Benefit:** Gets cached VirtualVariableFile, reduces connection overhead

### Pattern 2: Polling to Event-Driven


**Before:**
```rust
loop {
    let val = client.read_variable(tenant, service, "status").await?;
    if val != last_value {
        process_change(&val);
        last_value = val;
    }
    tokio::time::sleep(Duration::from_millis(100)).await;
}
```

**After:**
```rust
client.start_file_monitoring().await?;

while let Some(event) = client.wait_for_file_change().await? {
    if event.changed_variables.contains(&"status".to_string()) {
        let val = vf.read_variable_slice("status").await?;
        process_change(&val);
    }
}
```

**Benefit:** No polling, immediate change notification, uses file watcher

### Pattern 3: Multi-Variable Transactions


**Before:**
```rust
// Three separate network roundtrips
let user_id = client.read_variable(tenant, service, "user_id").await?;
let status = client.read_variable(tenant, service, "status").await?;
let count = client.read_variable(tenant, service, "count").await?;

if should_update {
    // Another three roundtrips
    client.write_variable(tenant, service, "user_id", &new_user_id).await?;
    client.write_variable(tenant, service, "status", &new_status).await?;
    client.write_variable(tenant, service, "count", &new_count).await?;
}
```

**After:**
```rust
let vf = client.get_virtual_service_file(tenant, service).await?.unwrap();

// All local or batch updated
let user_id = vf.read_variable_slice("user_id").await?;
let status = vf.read_variable_slice("status").await?;
let count = vf.read_variable_slice("count").await?;

if should_update {
    // Same API, optimized by hybrid layer
    vf.write_variable("user_id", &new_user_id).await?;
    vf.write_variable("status", &new_status).await?;
    vf.write_variable("count", &new_count).await?;
}
```

**Benefit:** Significantly lower latency for local clients, unified API

## Compatibility Matrix


| Feature           | Old API    | Old + Hybrid Init | New Hybrid API |
| ----------------- | ---------- | ----------------- | -------------- |
| Remote access     | ✅          | ✅                 | ✅              |
| Local access      | ⚠️ WSS only | ✅ Optimized       | ✅ Optimized    |
| Change detection  | ❌ Polling  | ✅ Events          | ✅ Events       |
| SIMD acceleration | ❌          | ⚠️ Limited         | ✅ Full         |
| Performance       | Baseline   | Improved          | Best           |
| Code changes      | None       | Minimal           | Moderate       |

## Migration Checklist


### For Path 1 (No Changes)

- [ ] No code changes needed
- [ ] Test existing functionality
- [ ] Consider future migration

### For Path 2 (Minimal)

- [ ] Add `client.init_file_watcher().await?;`
- [ ] Add `client.start_file_monitoring().await?;`
- [ ] Test existing functionality works
- [ ] Optionally add change event handling
- [ ] Validate performance improvement

### For Path 3 (Full)

- [ ] Audit all variable access patterns
- [ ] Create metadata for all variables
- [ ] Refactor read/write loops to use VirtualVariableFile
- [ ] Convert polling to event-driven where applicable
- [ ] Update tests to use new API
- [ ] Benchmark before/after performance
- [ ] Update documentation

## Testing During Migration


### Regression Testing


```rust
#[tokio::test]

async fn test_old_api_still_works() {
    let client = setup_client().await;
    
    // Old API should work
    let result = client.read_variable("tenant", "svc", "var").await;
    assert!(result.is_ok());
}
```

### Hybrid API Testing


```rust
#[tokio::test]

async fn test_new_hybrid_api() {
    let client = setup_client().await;
    client.init_file_watcher().await.unwrap();
    
    let vf = client.get_virtual_service_file("tenant", "svc")
        .await
        .unwrap()
        .unwrap();
    
    let meta = VariableMetadata::new("var".to_string(), 0, 8, 1);
    assert!(vf.register_variable(meta).await.is_ok());
}
```

### Equivalence Testing


```rust
#[tokio::test]

async fn test_old_new_api_equivalence() {
    let client = setup_client().await;
    client.init_file_watcher().await.unwrap();
    
    // Read via old API
    let old_result = client.read_variable("tenant", "svc", "var")
        .await
        .unwrap();
    
    // Read via new API
    let vf = client.get_virtual_service_file("tenant", "svc")
        .await
        .unwrap()
        .unwrap();
    let new_result = vf.read_variable_slice("var")
        .await
        .unwrap();
    
    // Should be identical
    assert_eq!(old_result, new_result);
}
```

## Performance Expectations


### After Minimal Migration (Path 2)


```
Old API (remote):        1-100 ms per variable (network)
Old API + Init (remote): 1-100 ms per variable (optimized routing)
Old API + Init (local):  < 100 μs per variable (direct mapping!)
```

**Expected improvement for local clients: 10,000x faster**

### After Full Migration (Path 3)


```
New API (remote):  10-50 μs overhead (better batching)
New API (local):   < 1 μs per variable (zero-copy)
SIMD comparison:   1-10 μs (vs 100+ μs byte-by-byte)
```

**Expected improvement for local: 10,000x faster than old remote**
**Expected improvement for remote: 100x faster than polling**

## Troubleshooting


### Issue: "VariableNotFound" error


**Cause:** Variable wasn't registered with metadata

**Fix:**
```rust
// Must register before accessing
let meta = VariableMetadata::new("variable_name".to_string(), offset, size, 1);
vf.register_variable(meta).await?;
```

### Issue: File watcher not detecting changes


**Cause:** Monitor not started or service not registered

**Fix:**
```rust
// Ensure monitoring is started
client.start_file_monitoring().await?;

// Ensure virtual file is registered
let vf = client.get_virtual_service_file("tenant", "svc").await?.unwrap();
client.init_file_watcher().await?; // Initialize before getting file
```

### Issue: Performance not improving on local


**Cause:** Still using remote API path

**Fix:**
Check that:
1. File watcher initialized: `client.init_file_watcher().await?`
2. Using new API: `vf.read_variable_slice()` not `client.read_variable()`
3. Server returns local file path for direct mapping
4. File permissions allow memory mapping

### Issue: Old tests failing after migration


**Cause:** Breaking changes in async signatures or return types

**Fix:**
- Update test setup to initialize file watcher
- Adjust assertions for new return types
- Consider adding both old and new API tests

## Decision Tree


```
Do you need maximum performance?
├─ YES → Use Path 3 (Full Migration)
│  └─ Pros: Best latency, SIMD acceleration, event-driven
│  └─ Cons: Requires refactoring
│
├─ MAYBE → Use Path 2 (Minimal)
│  └─ Pros: Quick wins, backward compatible
│  └─ Cons: Doesn't leverage full benefits
│
└─ NO → Use Path 1 (No Changes)
   └─ Pros: Zero effort, zero risk
   └─ Cons: Misses optimization
```

## Next Steps


1. **Choose your path** based on requirements
2. **Test thoroughly** with regression tests
3. **Benchmark** before/after for Path 2 & 3
4. **Update docs** for your organization
5. **Monitor** production for issues
6. **Gradually roll out** to production services
7. **Celebrate** performance improvements! 🚀