code-search-cli 0.2.3

# Caching Strategy

This document describes the caching architecture and strategy used in `cs` (Code Search) to optimize translation file parsing performance.

## Overview

`cs` implements a sophisticated two-tier caching system with cross-process cache sharing to dramatically improve search performance for large translation files (YAML/JSON).

## Performance Impact

### Without Cache
- Large YAML file (389KB, 8,689 lines): **0.455s**
- No match (grep prefilter): **0.025s**

### With Cache
- First run: **0.217s** (parse + cache write)
- Subsequent runs: **0.165s** (cache hit) - **24% faster**
- Combined speedup: **Cache + Bottom-up parsing = 95%+ faster than naive approach**

## Architecture

### Two-Tier Caching System

```
┌─────────────────────────────────────────┐
│         Cache Request                    │
└─────────────────┬───────────────────────┘
                  │
         ┌────────▼────────┐
         │  Front Cache    │ (In-Memory LRU)
         │  512 entries    │
         └────────┬────────┘
                  │ Miss
         ┌────────▼────────────────┐
         │    Backend Cache        │
         │  ┌──────────────────┐   │
         │  │ TCP Server Mode  │   │ (Cross-process)
         │  │ (Preferred)      │   │
         │  └──────────────────┘   │
         │         OR               │
         │  ┌──────────────────┐   │
         │  │  Local Mode      │   │ (Fallback)
         │  │  (Sled DB)       │   │
         │  └──────────────────┘   │
         └─────────────────────────┘
```

### Components

#### 1. Front Cache (In-Memory)
- **Type**: LRU (Least Recently Used)
- **Capacity**: 512 entries
- **Purpose**: Ultra-fast hot data access
- **Eviction**: Removes oldest accessed entries when full
- **Location**: Per-process memory

#### 2. Backend Cache (Persistent)

**TCP Server Mode** (Preferred):
- **Purpose**: Share cache across multiple `cs` processes
- **Implementation**: Background TCP server on `127.0.0.1:random_port`
- **Database**: Sled embedded database
- **Benefits**: 
  - Single cache shared by all `cs` instances
  - Avoids duplicate parsing across processes
  - Automatic server spawning

**Local Mode** (Fallback):
- **When**: TCP server fails or disabled via `CS_DISABLE_CACHE_SERVER`
- **Database**: Sled embedded database (per-process)
- **Location**: `~/.cache/cs/` (or platform equivalent)

## Cache Key Strategy

### Key Format
```
{file_path}|{normalized_query}
```

### Normalization Rules
- **Case-insensitive searches**: Query lowercased
- **Case-sensitive searches**: Query unchanged
- **File path**: Absolute path preserved

### Examples
```
/tmp/en.yml|log in          # Case-insensitive "Log In"
/tmp/en.yml|Log In          # Case-sensitive "Log In"
/tmp/fr.yml|connexion       # Different file
```

## Cache Value

Each cached entry stores:

```rust
struct CacheValue {
    mtime_secs: u64,              // File modification time
    file_size: u64,               // File size in bytes
    last_accessed: u64,           // Last access timestamp
    results: Vec<TranslationEntry>, // Parsed translation entries
}
```

## Invalidation Strategy

### File-Based Invalidation
Cache entries are invalidated when **either** changes:
1. **File modification time** (mtime)
2. **File size**

### Example Flow
```
1. Parse file.yml (mtime: 100, size: 1KB) → Cache entry A
2. File modified (mtime: 200, size: 1KB) → Cache miss, re-parse
3. Content added (mtime: 200, size: 2KB) → Cache miss, re-parse
```

### Why Both Checks?
- **mtime**: Detects content changes
- **size**: Backup check (mtime can be unreliable on some filesystems)

## Cleanup Strategy

### Automatic Cleanup Triggers
1. **On cache open** if last cleanup was >6 hours ago
2. **Periodic**: Every 6 hours of active use

### Cleanup Rules

#### Age-Based Cleanup
- **Max age**: 30 days
- **Action**: Remove entries not accessed in 30 days
- **Reason**: Translation files rarely change; 30 days provides good balance

#### Size-Based Cleanup
- **Max cache size**: 1GB
- **Action**: Remove least recently used entries until size < 1GB
- **Strategy**: Sort by `last_accessed`, remove oldest first

### Cleanup Algorithm
```
1. Scan all cache entries
2. Remove entries with age > 30 days
3. If total size > 1GB:
   - Sort remaining by last_accessed (oldest first)
   - Remove oldest until size <= 1GB
4. Flush database
5. Update cleanup timestamp
```

## Integration with Bottom-Up Parsing

The cache works synergistically with the bottom-up parsing optimization:

### Workflow
```
Search request
    │
    ├─> Check cache (mtime, size)
    │   ├─> Cache HIT → Return cached results (fastest)
    │   └─> Cache MISS → Continue
    │
    ├─> Grep prefilter
    │   ├─> No match → Skip file (fast)
    │   └─> Match found → Continue
    │
    ├─> Bottom-up parse (only matched lines)
    │   └─> Store in cache
    │
    └─> Return results
```

### Performance Cascade
1. **First search**: Grep prefilter + Bottom-up parse + Cache write
2. **Subsequent searches**: Cache hit (instant)
3. **File changes**: Invalidate → Repeat step 1

## Cache Server Architecture

### Server Lifecycle

```
Client Request
    │
    ├─> Read port file (~/.cache/cs/cache.port)
    │   ├─> File exists → Try connect
    │   │   ├─> Success → Use server
    │   │   └─> Fail → Spawn server
    │   └─> No file → Spawn server
    │
    ├─> Spawn server if needed
    │   ├─> Execute: cs --cache-server (background)
    │   ├─> Server binds to 127.0.0.1:random_port
    │   ├─> Server writes port to file
    │   └─> Wait 150ms for startup
    │
    └─> Fallback to local cache on failure
```

### Server Protocol

#### Message Format
Binary protocol using bincode serialization.

#### Request Types
```rust
enum CacheRequest {
    Get {
        file: PathBuf,
        query: String,
        case_sensitive: bool,
        mtime_secs: u64,
        file_size: u64,
    },
    Set {
        file: PathBuf,
        query: String,
        case_sensitive: bool,
        mtime_secs: u64,
        file_size: u64,
        results: Vec<TranslationEntry>,
    },
    Clear,
    Ping,
}
```

#### Response Types
```rust
enum CacheResponse {
    Get(Option<Vec<TranslationEntry>>),
    Ack(bool),
}
```

### Server Benefits
- **Cross-process sharing**: Multiple `cs` processes share one cache
- **Memory efficiency**: Single database instance
- **Build systems**: CI/CD with parallel jobs benefit significantly

## Visual Indicators

During search, `cs` shows cache status:

- **`C`** (cyan) - Cache hit
- **`.`** (green) - Successfully parsed
- **`-`** (dimmed) - Skipped (grep prefilter, no match)
- **`S`** (yellow) - Skipped (parse error)

### Example Output
```
...C..--.C..S...
```
Translation: 3 files parsed, 2 cache hits, 4 skipped (no match), 1 parse error

## Configuration

### Environment Variables

```bash
# Disable TCP cache server, use local cache only
export CS_DISABLE_CACHE_SERVER=1
```

### Cache Location

**Linux/macOS**: `~/.cache/cs/`
**Windows**: `%LOCALAPPDATA%\cs\cache\`

### Cache Files
```
~/.cache/cs/
├── cache.port        # TCP server address (if running)
├── meta.last         # Last cleanup timestamp
└── db/               # Sled database files
    ├── conf
    └── db
```

## Advanced Features

### Front Cache LRU Eviction
When the in-memory cache reaches 512 entries:
1. Find entry with oldest `last_accessed`
2. Remove that entry
3. Insert new entry

### Concurrent Access
- **Front cache**: Mutex-protected for thread safety
- **Sled database**: Handles concurrent access internally
- **TCP server**: Serial request handling (one connection at a time)

## Performance Tuning

### Constants (Tunable)
```rust
const FRONT_CACHE_CAP: usize = 512;              // In-memory entries
const MAX_CACHE_SIZE: u64 = 1_000_000_000;       // 1GB disk cache
const MAX_CACHE_AGE_SECS: u64 = 30 * 24 * 60 * 60; // 30 days
const CLEANUP_INTERVAL_SECS: u64 = 6 * 60 * 60;    // 6 hours
```

### Recommendations
- **Default settings**: Optimal for most use cases
- **Increase `FRONT_CACHE_CAP`**: If frequently searching same files
- **Increase `MAX_CACHE_SIZE`**: For very large projects with 1000+ translation files
- **Decrease `MAX_CACHE_AGE_SECS`**: For rapidly changing translation files

## Testing

### Manual Cache Testing
```bash
# Clear cache
cs --clear-cache

# First run (no cache)
time cs "search term" /path/to/files

# Second run (cache hit)
time cs "search term" /path/to/files

# Modify file to test invalidation
touch /path/to/files/en.yml

# Third run (cache miss, re-parse)
time cs "search term" /path/to/files
```

### Unit Tests
The cache implementation includes comprehensive tests:
- `test_cache_hit_local`: Verify cache retrieval
- `test_cache_invalidation_on_file_change_local`: Verify mtime/size invalidation
- `test_case_insensitive_normalization_local`: Verify query normalization

## Future Improvements

### Potential Enhancements
1. **Distributed cache**: Redis/Memcached for team sharing
2. **Compression**: Compress cached results for large translation files
3. **Cache warming**: Pre-populate cache for common queries
4. **Statistics**: Track hit rate, cache size, most accessed files
5. **Smart prefetching**: Predict and cache related files

### Why Not Implemented Yet
- **Simplicity**: Current design balances complexity vs. benefit
- **Performance**: Current cache provides 24%+ speedup, diminishing returns
- **Dependencies**: Avoiding external dependencies (Redis, etc.)

## Troubleshooting

### Cache Not Working
1. Check cache directory exists and is writable
2. Verify no `CS_DISABLE_CACHE_SERVER` environment variable
3. Check cache server process: `ps aux | grep "cs --cache-server"`

### Poor Cache Hit Rate
1. Files changing frequently → Expected behavior
2. Different queries → Each query gets separate cache entry
3. Case sensitivity → `"Log In"` vs `"log in"` (case-insensitive) share cache

### Cache Server Issues
1. Port already in use → Server will pick different port
2. Permission denied → Check `~/.cache/cs/` permissions
3. Server crashes → Falls back to local cache automatically

## Conclusion

The caching system provides significant performance improvements while maintaining correctness through robust invalidation. Combined with the bottom-up parsing optimization, `cs` achieves near-instant search results for large translation files on subsequent runs.

**Key Takeaway**: The cache is transparent, automatic, and requires zero configuration for optimal performance.