db-sync 0.1.1

# DB-Sync Database Backup Synchronization System


A secure and stable database backup file synchronization system that supports automatic synchronization and verification of backup files across multiple systems.

## System Architecture


- **Server (db-sync-server)**: Receives and stores backup files, provides API interfaces
- **Client (db-sync-client)**: Periodically checks and uploads backup files

## Features


### Security Features

- JWT token authentication
- MD5 file integrity check
- File size verification
- Secure HTTP transmission

### Stability Features

- Automatic retry mechanism
- Error recovery
- Logging
- Health checks

### Synchronization Strategies

- **Monthly Upload**: Automatically uploads all backup files on the 20th of each month
- **Daily Verification**: Verifies file integrity from the 20th to the 26th of each month
- **Incremental Sync**: Only uploads changed files

## Quick Start


### 1. Configure the Server


Edit `server.config.json`:

```json
{
  "listen_addr": "0.0.0.0:3030",
  "data": [
    {
      "system_name": "System Name",
      "system_ip": "192.168.1.100",
      "backup_dir": "./backups/System Name",
      "auth_token": "your-secret-token-here"
    }
  ]
}
```

### 2. Configure the Client


Edit `client.config.json`:

```json
{
  "data": [
    {
      "system_name": "System Name",
      "system_ip": "192.168.1.100",
      "backup_server_url": "http://server-ip:3030",
      "auth_token": "your-secret-token-here",
      "local_backup_dir": "./local/backups"
    }
  ]
}
```

### 3. Build the Project


```bash
cargo build --release
```

### 4. Generate Authentication Tokens


Use the built-in CLI tool to generate a 512-bit authentication token for each system:

#### Generate a Token for a Single System


```bash
cargo run --bin gen-auth-token generate --system-ip 192.168.1.100 --system-name "HR System"
```

Example output:
```
=== DB-Sync Authentication Token Generation ===
System IP: 192.168.1.100
System Name: HR System
Token Length: 512 bits
Generated Token: 1154e194e0668a3a37ce63a28c6fe72e0200b937fb1ebb6bbf80345a8bee00c6a612fe10dac35b7781f89a13116c3cf81941c316529b0e89b2ee8ebbf3ac4846
```

#### Batch Generate Tokens


1. First, create a configuration file `batch_config.json`:

```json
{
  "secret_key": "your_custom_secret_key",
  "systems": [
    {
      "system_ip": "192.168.1.100",
      "system_name": "HR System"
    },
    {
      "system_ip": "192.168.1.101",
      "system_name": "Asset Management System"
    }
  ]
}
```

2. Then run the batch generation command:

```bash
cargo run --bin gen-auth-token batch --config-file batch_config.json --format json
```

#### Validate Token Format


```bash
cargo run --bin gen-auth-token validate --token "your_token_here"
```

#### View All Available Commands


```bash
cargo run --bin gen-auth-token -- --help
```

#### Generate Example Configuration


```bash
cargo run --bin gen-auth-token example
```

### 5. Run the System


#### Start the Server

```bash
cargo run --bin db-sync-server
```

#### Start the Client

```bash
cargo run --bin db-sync-client
```

### 6. Test the System


Run the test script:
```bash
# Windows

test_system.bat

# Linux/Mac

bash test_system.sh
```

## API Endpoints


### Health Check

```
GET /health
```

### File Upload

```
POST /api/upload
Headers:
- authorization: Bearer <token>

Query Parameters:
- system_ip: System IP
- filename: Filename
- blake3: File blake3 hash
- size: File size

Body: File binary data
```

### blake3 Query

```
POST /api/blake3
Headers:
- authorization: Bearer <token>

Query Parameters:
- system_ip: System IP
- filename: Filename
```

## File Structure

```
db-sync/
├── src/
│   ├── db-sync-server.rs    # Server code
│   └── db-sync-client.rs    # Client code
├── server.config.json       # Server configuration
├── client.config.json       # Client configuration
├── test_data/              # Test data
├── test_system.bat         # Windows test script
├── test_system.sh          # Linux/Mac test script
└── README.md               # Documentation
```

## Security Recommendations

1. **Token Management**: Use strong passwords for auth_token and change them regularly.
2. **Network Security**: Use HTTPS in production environments.
3. **Access Control**: Restrict IP addresses for API access.
4. **Backup Encryption**: Encrypt sensitive backup files.

## Monitoring and Maintenance

### Viewing Logs
The system uses structured logging, and the log level can be controlled via environment variables:

```bash
export RUST_LOG=info
cargo run --bin db-sync-server
```

### Health Check

Periodically check the service status:
```bash
curl http://server-ip:3030/health
```

### File Verification

Manually verify file integrity:
```bash
# Calculate blake3 checksum for local file

blake3 backup_file.sql

# Query server file blake3 checksum

curl -X POST http://server-ip:3030/api/blake3 \
  -H "authorization: Bearer <token>" \
  -d "system_ip=192.168.1.100&filename=backup_file.sql"
```

## Troubleshooting


### Common Issues


1. **Connection Failure**: Check network connection and firewall settings.
2. **Authentication Failure**: Verify that the auth_token is configured correctly.
3. **File Checksum Failure**: Check file integrity during transmission.
4. **Insufficient Disk Space**: Regularly clean up old backup files.

### Debug Mode


Enable verbose logging:
```bash
export RUST_LOG=debug
cargo run --bin db-sync-server
```

## License


This project is licensed under the MIT License.

## Crates.io Information


**Package Name**: `db-sync`
**Version**: `0.1.0`
**Minimum Rust Version**: `1.70+`
**License**: `MIT`
**Keywords**: `backup`, `sync`, `database`, `file-transfer`, `security`

## Release Notes


This package is ready for publication on [crates.io](https://crates.io/), providing an enterprise-grade database backup synchronization solution.

### Features

- 🔐 Secure token authentication mechanism
- 📁 File integrity verification (blake3)
- ⏰ Scheduled task management
- 🔄 Automatic retry and error recovery
- 📊 Detailed logging and monitoring

### Use Cases

- Enterprise database backups
- Multi-system file synchronization
- Scheduled data backup tasks
- Secure file transfer

## Contributing


Issues and Pull Requests are welcome to improve this project.

## Changelog


### v0.1.0

- Initial release
- Implemented basic client-server architecture
- Support for file upload and blake3 verification
- Added scheduled task functionality
- Comprehensive error handling and logging