nntp-proxy 0.2.2

High-performance NNTP proxy server with connection pooling and authentication
Documentation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
# NNTP Proxy

A high-performance NNTP proxy server written in Rust, with intelligent hybrid routing, round-robin load balancing, and TLS support.

## Key Features

- 🧠 **Hybrid routing mode** - Intelligent per-command routing that auto-switches to stateful when needed (default)
- 🔄 **Round-robin load balancing** - Distributes connections across multiple backend servers
- 🔐 **TLS/SSL support** - Secure backend connections using rustls with system certificate store
-**High performance** - Lock-free routing, optimized response parsing, efficient I/O
- 🏥 **Health checking** - Automatic backend health monitoring with failure detection
- 📊 **Connection pooling** - Pre-authenticated connections with configurable limits and reservation
- 🛡️ **Type-safe protocol handling** - RFC 3977 compliant parsing with comprehensive validation
- ⚙️ **TOML configuration** - Simple, readable configuration with sensible defaults
- 🔍 **Structured logging** - Detailed tracing for debugging and monitoring
- 🧩 **Modular architecture** - Clean separation of concerns, well-tested codebase

## Table of Contents

- [Overview]#overview
- [Quick Start]#quick-start
- [Configuration]#configuration
- [Usage]#usage
- [Architecture]#architecture
- [Performance]#performance
- [Limitations]#limitations
- [Building]#building
- [Testing]#testing
- [License]#license

## Overview

This NNTP proxy offers three operating modes:

1. **Hybrid mode** (default) - Starts with per-command routing, automatically switches to stateful when needed
2. **Standard mode** (`--routing-mode standard`) - Full NNTP proxy with complete command support
3. **Per-command routing mode** (`--routing-mode per-command`) - Pure stateless routing for maximum efficiency

### Design Goals

- **Load balancing** - Distribute connections across multiple backend servers with health-aware routing
- **Performance** - Lock-free routing, optimized protocol parsing, efficient I/O with connection pooling
- **Security** - TLS/SSL support with certificate verification, pre-authenticated backend connections
- **Reliability** - Health monitoring, automatic failover, graceful connection handling
- **Flexibility** - Choose between full NNTP compatibility or resource-efficient per-command routing

### When to Use This Proxy

✅ **Hybrid mode (default) - Best for:**
- **Universal compatibility** - Works with any NNTP client automatically
- **Optimal performance** - Efficient per-command routing until stateful operations needed
- **Intelligent switching** - Automatically detects when clients need stateful mode (GROUP, XOVER, etc.)
- **Resource efficiency** - Uses per-command routing when possible, stateful only when necessary
- **Most deployments** - Recommended default that adapts to client behavior

✅ **Standard mode - Good for:**
- Traditional newsreaders requiring guaranteed stateful behavior
- Debugging or when you need predictable 1:1 connection mapping
- Legacy deployments where hybrid mode is not desired
- Maximum compatibility and simplicity

✅ **Per-command routing mode - Good for:**
- Message-ID based article retrieval workloads only
- Indexing and search tools that only need ARTICLE/BODY/HEAD by message-ID
- Specialized deployments where stateful operations are never needed
- Maximum resource efficiency when you control all clients

❌ **Not suitable for:**
- Scenarios requiring concurrent request processing (NNTP is inherently serial)
- Custom NNTP extensions not in RFC 3977 (unless in standard mode with compatible backend)

## Limitations

### Per-Command Routing Mode Restrictions

When running in **per-command routing mode** (`--per-command-routing` or `-r`), the proxy rejects stateful commands to maintain consistent routing:

**Rejected commands (require group context):**
- Group navigation: `GROUP`, `NEXT`, `LAST`, `LISTGROUP`
- Article by number: `ARTICLE 123`, `HEAD 123`, `BODY 123`, `STAT 123`
- Overview commands: `XOVER`, `OVER`, `XHDR`, `HDR`

**Always supported:**
- ✅ Article by Message-ID: `ARTICLE <message-id@example.com>`
- ✅ Metadata retrieval: `LIST`, `HELP`, `DATE`, `CAPABILITIES`
- ✅ Posting: `POST` (if backend supports)
- ✅ Authentication: `AUTHINFO USER/PASS` (handled by proxy)

**Rationale:** Commands requiring group context (current article number, group selection) cannot work reliably when each command routes to a different backend. Use standard mode or hybrid mode if you need these features.

### Hybrid Mode Advantage (Default)

**Hybrid mode** automatically handles the per-command routing limitations:
- **Starts efficiently** - Uses per-command routing for stateless operations (ARTICLE by message-ID, LIST, etc.)
-**Switches intelligently** - Detects stateful commands (GROUP, XOVER, etc.) and seamlessly switches to dedicated backend
-**Universal compatibility** - Works with any NNTP client without configuration
-**Resource efficient** - Uses shared pool when possible, dedicated connections only when needed
-**Best of both worlds** - Combines per-command efficiency with full protocol support

### Standard Mode

In **standard mode** (`--routing-mode standard`):
- **All RFC 3977 commands supported** - full bidirectional forwarding
- ✅ Compatible with all NNTP clients
- ✅ Stateful operations work normally (GROUP, NEXT, LAST, XOVER, etc.)
- ✅ Each client connection maps to one backend connection (1:1)
- ✅ Simple, predictable behavior

## Quick Start

### Prerequisites

- Rust 1.85+ (or use the included Nix flake)
- Optional: Nix with flakes for reproducible development environment

### Installation

```bash
# Clone the repository
git clone https://github.com/mjc/nntp-proxy.git
cd nntp-proxy

# Build release version
cargo build --release

# Binary will be in target/release/nntp-proxy
```

### Using Nix (Optional)

```bash
# Enter development environment
nix develop

# Or use direnv
direnv allow

# Build and run
cargo build
cargo run
```

### First Run

1. Create a configuration file (see [Configuration]#configuration section)
2. Run the proxy:

```bash
./target/release/nntp-proxy --port 8119 --config config.toml
```

3. Connect with a client:

```bash
telnet localhost 8119
```

## Configuration

The proxy uses a TOML configuration file. Create `config.toml`:

```toml
# Backend servers (at least one required)
[[servers]]
host = "news.example.com"
port = 119
name = "Primary News Server"
username = "your_username"      # Optional
password = "your_password"      # Optional
max_connections = 20            # Optional, default: 10

[[servers]]
host = "news2.example.com"
port = 119
name = "Secondary News Server"
max_connections = 10

# Health check configuration (optional)
[health_check]
interval_secs = 30         # Seconds between checks (default: 30)
timeout_secs = 5           # Timeout per check (default: 5)
unhealthy_threshold = 3    # Failures before marking unhealthy (default: 3)
```

### Configuration Reference

#### Server Configuration

| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `host` | string | Yes | - | Backend server hostname or IP |
| `port` | integer | Yes | - | Backend server port |
| `name` | string | Yes | - | Friendly name for logging |
| `username` | string | No | - | Authentication username |
| `password` | string | No | - | Authentication password |
| `max_connections` | integer | No | 10 | Max concurrent connections to this backend |
| `use_tls` | boolean | No | false | Enable TLS/SSL encryption |
| `tls_verify_cert` | boolean | No | true | Verify server certificates (uses system CA store) |
| `tls_cert_path` | string | No | - | Path to additional CA certificate (PEM format) |

### TLS/SSL Support

The proxy supports TLS/SSL encrypted connections to backend servers using **rustls** - a modern, memory-safe TLS implementation written in pure Rust.

#### Basic TLS Configuration

For servers with valid SSL certificates from recognized CAs:

```toml
[[servers]]
host = "secure.newsserver.com"
port = 563  # Standard NNTPS port
name = "Secure News Server"
use_tls = true
tls_verify_cert = true  # Uses system certificate store (default)
max_connections = 20
```

**That's it!** No additional certificate configuration needed. The proxy will:
- Use rustls with your operating system's trusted certificate store
- Verify the server's certificate against system CAs
- Establish a secure TLS 1.3 connection (with TLS 1.2 fallback)
- Support session resumption for improved performance

#### Private/Self-Signed CA

For servers using certificates from a private CA:

```toml
[[servers]]
host = "internal.newsserver.local"
port = 563
name = "Internal News Server"
use_tls = true
tls_verify_cert = true
tls_cert_path = "/etc/nntp-proxy/internal-ca.pem"  # PEM format
max_connections = 10
```

**Note**: The custom certificate is **added to** the system certificates, not replacing them.

#### System Certificate Stores

| Operating System | Certificate Store |
|-----------------|-------------------|
| Linux (Debian/Ubuntu) | `/etc/ssl/certs/ca-certificates.crt` |
| Linux (RHEL/CentOS) | `/etc/pki/tls/certs/ca-bundle.crt` |
| macOS | Security.framework (Keychain) |
| Windows | SChannel (Windows Certificate Store) |

#### Port Reference

| Port | Protocol | Description |
|------|----------|-------------|
| 119  | NNTP | Unencrypted, standard NNTP |
| 563  | NNTPS | NNTP over TLS/SSL (encrypted) |
| 8119 | Custom | Common alternative port |

#### Security Best Practices

✅ **Always verify certificates in production** (`tls_verify_cert = true`)  
✅ **Keep system certificates updated** via OS package manager  
✅ **Use TLS 1.3 when possible** (automatically negotiated by rustls)  
✅ **Use standard NNTPS port 563** for encrypted connections  
✅ **Monitor TLS handshake failures** in logs  

⚠️ **Never set `tls_verify_cert = false` in production** - this disables all certificate verification and is extremely insecure!

#### Environment Variable Overrides for Servers

Backend servers can be configured entirely via environment variables, useful for Docker/container deployments. If any `NNTP_SERVER_N_HOST` variable is found, environment variables take precedence over the config file.

**Per-server variables (N = 0, 1, 2, ...):**

| Variable | Required | Default | Description |
|----------|----------|---------|-------------|
| `NNTP_SERVER_N_HOST` | Yes | - | Backend hostname/IP (presence triggers env mode) |
| `NNTP_SERVER_N_PORT` | No | 119 | Backend port |
| `NNTP_SERVER_N_NAME` | No | "Server N" | Friendly name for logging |
| `NNTP_SERVER_N_USERNAME` | No | - | Backend authentication username |
| `NNTP_SERVER_N_PASSWORD` | No | - | Backend authentication password |
| `NNTP_SERVER_N_MAX_CONNECTIONS` | No | 10 | Max concurrent connections |

**Example Docker deployment:**
```bash
docker run -e NNTP_SERVER_0_HOST=news.example.com \
           -e NNTP_SERVER_0_PORT=119 \
           -e NNTP_SERVER_0_NAME="Primary" \
           -e NNTP_SERVER_0_USERNAME=user \
           -e NNTP_SERVER_0_PASSWORD=pass \
           -e NNTP_SERVER_1_HOST=news2.example.com \
           -e NNTP_SERVER_1_PORT=119 \
           -e NNTP_PROXY_PORT=8119 \
           nntp-proxy
```

#### Health Check Configuration

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `interval_secs` | integer | 30 | Seconds between health checks |
| `timeout_secs` | integer | 5 | Health check timeout in seconds |
| `unhealthy_threshold` | integer | 3 | Consecutive failures before marking unhealthy |

### Authentication

The proxy handles authentication transparently:

1. **Backend authentication** (when credentials are configured)
   - Configure `username` and `password` in server configuration
   - Proxy authenticates to backends during connection pool initialization
   - Connections remain pre-authenticated, eliminating per-command auth overhead
   - Credentials are used with RFC 4643 AUTHINFO USER/PASS commands

2. **Client authentication handling**
   - Client `AUTHINFO USER/PASS` commands are intercepted by the proxy
   - Proxy responds with success (281/381) without forwarding to backend
   - No actual credential validation performed (network-level access control recommended)
   - To restrict access, use firewall rules, VPN, or network segmentation

## Usage

### Command Line Options

```bash
nntp-proxy [OPTIONS]
```

| Option | Short | Environment Variable | Description | Default |
|--------|-------|---------------------|-------------|---------|
| `--port <PORT>` | `-p` | `NNTP_PROXY_PORT` | Listen port | 8119 |
| `--routing-mode <MODE>` | `-r` | `NNTP_PROXY_ROUTING_MODE` | Routing mode: hybrid, standard, per-command | hybrid |
| `--config <FILE>` | `-c` | `NNTP_PROXY_CONFIG` | Config file path | config.toml |
| `--threads <NUM>` | `-t` | `NNTP_PROXY_THREADS` | Tokio worker threads | CPU cores |
| `--help` | `-h` | - | Show help | - |
| `--version` | `-V` | - | Show version | - |

**Note**: Environment variables take precedence over default values but are overridden by command-line arguments.

### Examples

```bash
# Hybrid mode with defaults (recommended)
nntp-proxy

# Custom port and config (still uses hybrid mode)
nntp-proxy --port 8120 --config production.toml

# Standard mode (full stateful behavior)
nntp-proxy --routing-mode standard

# Per-command routing mode (pure stateless)
nntp-proxy --routing-mode per-command

# Short form for routing modes
nntp-proxy -r standard
nntp-proxy -r per-command

# Single-threaded for debugging
nntp-proxy --threads 1

# Production setup
nntp-proxy --port 119 --config /etc/nntp-proxy/config.toml

# Using environment variables for configuration
NNTP_PROXY_PORT=8119 \
NNTP_PROXY_THREADS=4 \
NNTP_SERVER_0_HOST=news.example.com \
NNTP_SERVER_0_PORT=119 \
NNTP_SERVER_0_NAME="Primary" \
nntp-proxy

# Docker deployment with environment variables
docker run -d \
  -e NNTP_PROXY_PORT=119 \
  -e NNTP_SERVER_0_HOST=news.provider.com \
  -e NNTP_SERVER_0_USERNAME=myuser \
  -e NNTP_SERVER_0_PASSWORD=mypass \
  -e NNTP_SERVER_1_HOST=news2.provider.com \
  -p 119:119 \
  nntp-proxy
```

### Operating Modes

#### Hybrid Mode (default) - `--routing-mode hybrid`

- **Intelligent switching** - Starts each client in efficient per-command routing mode
- **Auto-detection** - Switches to stateful mode when client uses GROUP, XOVER, NEXT, LAST, etc.
- **Resource efficiency** - Uses shared connection pool until stateful behavior is needed
- **Seamless transition** - Switching happens transparently without client awareness
- **Pool reservation** - Reserves stateful connections (max_connections - 1) while keeping 1 for per-command routing
- **Universal compatibility** - Works with any NNTP client, optimizing automatically based on usage patterns

#### Standard Mode - `--routing-mode standard`

- One backend connection per client
- Simple 1:1 connection forwarding
- All NNTP commands supported
- Lower overhead, easier debugging
- Predictable behavior for legacy deployments

#### Per-Command Routing Mode - `--routing-mode per-command`

- Each command routed to next backend (round-robin)
- Commands processed serially (one at a time)
- Multiple clients share backend pool
- Health-aware routing
- Better resource distribution
- Stateful commands rejected (GROUP, XOVER, etc.)

## Architecture

### Module Organization

The codebase is organized into focused modules with clear responsibilities:

| Module | Purpose |
|--------|---------|
| `auth/` | Client and backend authentication (RFC 4643 AUTHINFO) |
| `cache/` | Article caching with TTL-based expiration (cache proxy binary) |
| `command/` | NNTP command parsing and classification |
| `config/` | Configuration loading and validation (TOML + environment variables) |
| `constants/` | Buffer sizes, timeouts, and performance tuning constants |
| `health/` | Backend health monitoring with DATE command probes |
| `network/` | Socket optimization for high-throughput transfers |
| `pool/` | Connection and buffer pooling with deadpool |
| `protocol/` | RFC 3977 protocol parsing, response categorization, message-ID handling |
| `router/` | Backend selection with lock-free round-robin and health awareness |
| `session/` | Client session lifecycle and command/response streaming |
| `stream/` | Connection abstraction supporting TCP and TLS |
| `tls/` | TLS configuration and handshake management using rustls |
| `types/` | Core type definitions (ClientId, BackendId) |

### Protocol Module

The `protocol` module centralizes all NNTP protocol knowledge:

- **`commands.rs`**: Command construction helpers (QUIT, DATE, AUTHINFO, ARTICLE, etc.)
- **`responses.rs`**: Response constants and builders (AUTH_REQUIRED, BACKEND_UNAVAILABLE, etc.)
- **`response.rs`**: Response parsing with `ResponseCode` enum for type-safe categorization
  - Multiline detection per RFC 3977 (1xx, 215, 220-225, 230-231, 282)
  - Message-ID extraction and validation (RFC 5536)
  - Terminator detection for streaming responses

### How It Works

#### Hybrid Mode Flow (Default)

```
Client Connection
Send Greeting (200 NNTP Proxy Ready)
Read Command
Classify Command (is_stateful check)
┌─ Stateless Command ─────┐    ┌─ Stateful Command ────┐
│  Route to Backend       │    │  Switch to Stateful   │
│  (per-command routing)  │    │  Reserve Backend       │
│  Execute & Stream       │    │  Bidirectional Forward │
│  Return to Pool         │    │  (until disconnect)    │
└─────────────────────────┘    └────────────────────────┘
    ↓                              ↓
Return to Command Reading      Connection Cleanup
```

#### Standard Mode Flow

```
Client Connection
Select Backend (round-robin, health-aware)
Get Pooled Connection (pre-authenticated)
Bidirectional Data Forwarding
Connection Cleanup & Return to Pool
```

#### Per-Command Routing Mode Flow

```
Client Connection
Send Greeting (200 NNTP Proxy Ready)
Read Command
Classify Command (protocol/command.rs)
Route to Healthy Backend (round-robin)
Get Pooled Connection
Execute Command (waits for complete response)
Stream Response to Client
Return Connection to Pool
Repeat (serial command processing)
```

### Performance Optimizations

The proxy implements several performance optimizations:

| Optimization | Impact | Description |
|--------------|--------|-------------|
| **ResponseCode enum** | Eliminates redundant parsing | Parse response once, reuse for multiline detection and success checks |
| **Lock-free routing** | ~10-15% CPU reduction | Atomic operations for backend selection instead of RwLock |
| **Pre-authenticated pools** | Eliminates auth overhead | Connections authenticate once during pool initialization |
| **Buffer pooling** | ~200+ allocs/sec saved | Reuse pre-allocated buffers in hot paths |
| **Optimized I/O** | Fewer syscalls | 256KB buffers for article transfers, TCP socket tuning |
| **TLS 1.3 with 0-RTT** | Faster reconnections | Session resumption and early data support in rustls |
| **Direct byte parsing** | Avoids allocations | Message-ID extraction and protocol parsing work on byte slices |

### RFC Compliance

The proxy adheres to NNTP standards:

- **RFC 3977**: Network News Transfer Protocol (NNTP)
  - Correct multiline response detection (status code second digit 1/2/3)
  - Proper terminator handling (`\r\n.\r\n`)
  - Serial command processing
- **RFC 4643**: AUTHINFO USER/PASS authentication extension
- **RFC 5536**: Message-ID format validation

## Performance

### Performance Characteristics

- **CPU Usage**: Low overhead with lock-free routing and optimized protocol parsing
  - Per-command routing mode: ~15% of one core for 80 connections at 105MB/s (AMD Ryzen 9 5950X)
  - Standard mode: Similar or lower due to simpler forwarding logic
- **Memory**: Constant usage with pooled buffers; no response buffering (streaming only)
- **Latency**: Minimal overhead (~1-2ms) for command routing and parsing
- **Throughput**: Typically limited by backend servers or network, not the proxy
- **Scalability**: Efficiently handles hundreds of concurrent connections per backend

### Profiling

To generate a performance flamegraph for analysis:

```bash
# Install cargo-flamegraph (if using Nix, it's already available)
cargo install flamegraph

# Run with flamegraph profiling (per-command routing mode)
cargo flamegraph --bin nntp-proxy -- --config config.toml -r --threads 1

# Open flamegraph.svg in a browser to analyze CPU hotspots
```

## Building

### Development Build

```bash
cargo build
./target/debug/nntp-proxy
```

### Release Build

```bash
cargo build --release
./target/release/nntp-proxy
```

### Production Deployment

```bash
# Build optimized binary
cargo build --release

# Copy binary to deployment location
sudo cp target/release/nntp-proxy /usr/local/bin/

# Create config directory
sudo mkdir -p /etc/nntp-proxy

# Copy config
sudo cp config.toml /etc/nntp-proxy/

# Run as service (example systemd unit included)
sudo systemctl start nntp-proxy
```

### Static Binary (Optional)

For maximum portability, build a fully static binary:

```bash
# Install musl target
rustup target add x86_64-unknown-linux-musl

# Build static binary
cargo build --release --target x86_64-unknown-linux-musl

# Result is a static binary with no dependencies
./target/x86_64-unknown-linux-musl/release/nntp-proxy
```

## Testing

### Running Tests

```bash
# All tests
cargo test

# Unit tests only
cargo test --lib

# Integration tests only
cargo test --test integration_tests

# With output
cargo test -- --nocapture

# Quiet mode
cargo test --quiet
```

### Test Coverage

The codebase includes:
- **200+ unit tests** covering all modules
- **Integration tests** for end-to-end scenarios including:
  - Multiline response handling
  - Per-command routing mode
  - Connection pooling and health checks
  - TLS/SSL connections
- **Protocol compliance tests** for RFC 3977, RFC 4643, RFC 5536
- **Zero clippy warnings** with strict linting enabled

### Manual Testing

Test with telnet or netcat:

```bash
# Connect to proxy
telnet localhost 8119

# Should see greeting like:
# 200 news.example.com ready

# Try commands:
HELP
LIST ACTIVE
ARTICLE <message-id@example.com>
QUIT
```

### Load Testing

For performance testing, create custom scripts that:
- Open multiple concurrent NNTP connections
- Issue realistic command sequences
- Measure throughput and latency
- Monitor CPU and memory usage

## Dependencies

### Core Dependencies

| Crate | Purpose |
|-------|---------|
| `tokio` | Async runtime and networking |
| `rustls` | Modern, memory-safe TLS implementation |
| `tokio-rustls` | Tokio integration for rustls |
| `webpki-roots` | Mozilla's CA certificate bundle |
| `rustls-native-certs` | System certificate store integration |
| `tracing` / `tracing-subscriber` | Structured logging framework |
| `anyhow` | Ergonomic error handling |
| `clap` | Command-line argument parsing with derive macros |
| `serde` / `toml` | Configuration parsing and serialization |
| `deadpool` | Generic connection pooling |
| `moka` | High-performance cache (for cache proxy) |
| `memchr` | Fast byte searching (message-ID extraction) |

### Development Dependencies

- `tempfile` - Temporary files for config testing
- Test helpers in `tests/test_helpers.rs`

## Troubleshooting

### Common Issues

**"Connection refused" when starting**
- Check if port is already in use: `lsof -i :8119`
- Try a different port: `--port 8120`

**"Backend authentication failed"**
- Verify credentials in config.toml
- Test direct connection to backend
- Check backend server logs

**"Command not supported" errors**
- In per-command routing mode, stateful commands are rejected (GROUP, NEXT, etc.)
- Use message-ID based retrieval instead
- For stateful operations, use standard mode or connect directly to backend

**High CPU usage**
- Try per-command routing mode: `-r` or `--per-command-routing`
- Reduce worker threads: `--threads 1`
- Check health check interval (increase if too frequent)

**Backends marked unhealthy**
- Check backend server status
- Verify network connectivity
- Review health check configuration
- Check logs for specific errors

### Logging

Control log verbosity with `RUST_LOG`:

```bash
# Info level (default)
RUST_LOG=info nntp-proxy

# Debug level
RUST_LOG=debug nntp-proxy

# Specific module
RUST_LOG=nntp_proxy::router=debug nntp-proxy

# Multiple modules
RUST_LOG=nntp_proxy::router=debug,nntp_proxy::health=debug nntp-proxy
```

## Roadmap

### Planned Features

- [ ] Prometheus metrics endpoint for monitoring
- [ ] Configuration hot-reload without restart
- [ ] Admin HTTP API for runtime stats and control
- [ ] Response caching layer for frequently requested articles
- [ ] IPv6 support
- [ ] Connection affinity mode (sticky sessions)

### Recently Completed (v0.2.0)

- [x] **Protocol module refactoring** - Centralized NNTP protocol handling
  - ResponseCode enum for type-safe response categorization
  - Message-ID extraction and validation helpers
  - Eliminated redundant response parsing (70% traffic optimization)
- [x] **TLS/SSL support** - Secure backend connections with rustls
  - System certificate store integration
  - TLS 1.3 with session resumption
  - Per-server TLS configuration
- [x] **Multiline response fix** - Correct RFC 3977 multiline detection
  - Fixed connection pool exhaustion bug
  - Proper status code parsing
- [x] Lock-free routing with atomic operations
- [x] Health checking system with DATE command probes
- [x] Per-command routing mode
- [x] Pre-authenticated connection pools
- [x] TOML configuration with environment variable overrides

## Contributing

Contributions welcome! Please:

1. Fork the repository
2. Create a feature branch
3. Install git hooks: `./scripts/install-git-hooks.sh`
4. Make your changes with tests
5. Ensure all checks pass:
   - `cargo test` - Run all tests
   - `cargo clippy --all-targets --all-features` - Run linter
   - `cargo fmt` - Format code
6. Submit a pull request

### Development Setup

After cloning the repository, install git hooks to automatically run code quality checks:

```bash
./scripts/install-git-hooks.sh
```

The pre-commit hook will automatically run:
- `cargo fmt --check` - Verify code formatting
- `cargo clippy --all-targets --all-features` - Check for lint warnings

To bypass the hook temporarily (not recommended): `git commit --no-verify`

## License

MIT License - see LICENSE file for details.

## Acknowledgments

Built with Rust and the excellent Tokio async ecosystem.