g-tools 0.5.4

Geraldo's toolbox
Documentation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
# MSAADA - Rust Project Guidelines

> "Msaada" is Swahili for "service/servant" - A powerful HTTP server for local development

## Build & Test Commands

- Build: `cargo build`
- Run: `cargo run -- --port <PORT> --dir <DIRECTORY>`
- Run with HTTPS: `cargo run -- --port <PORT> --dir <DIRECTORY> --ssl-cert <CERT> --ssl-key <KEY>`
- Run with config: `cargo run -- --port <PORT> --dir <DIRECTORY> --config serve.json`
- Run SPA mode: `cargo run -- --port <PORT> --dir <DIRECTORY> --single`
- Release build: `cargo build --release`
- Package for distribution: `./package.sh`
- Comprehensive testing: `./tests/run_comprehensive_tests.sh`
- Test: `cargo test`
- Test single test: `cargo test <TEST_NAME>`
- Check only: `cargo check`
- Format code: `cargo fmt`
- Lint: `cargo clippy`
- Update dependencies: `cargo update`

## Code Style Guidelines

- **Imports**: Group std imports first, then external crates, then local modules
- **Formatting**: Use rustfmt (run `cargo fmt` before commits)
- **Error Handling**: Use Result/Option types with pattern matching; avoid unwrap() in production code
- **Naming Conventions**:
  - snake_case for functions, variables, modules
  - CamelCase for types, traits, enums
  - SCREAMING_CASE for constants
- **Documentation**: Document public APIs with /// comments
- **Type Safety**: Use Rust's strong type system; avoid `as` casts when possible
- **Logging**: Use the structured logger module with colored output

## Module Architecture

### Core Modules

- **`main.rs`** - Application entry point, CLI parsing, server initialization
- **`clipboard.rs`** - Clipboard integration for copying server URLs
- **`config.rs`** - Configuration file parsing (serve.json, package.json, now.json)
- **`logger.rs`** - Structured logging with colors and timestamps
- **`network.rs`** - Port availability checking and network utilities
- **`shutdown.rs`** - Graceful shutdown handling for SIGINT/SIGTERM
- **`spa.rs`** - Single Page Application support utilities
- **`tls.rs`** - SSL/TLS certificate loading and configuration

### Template Files

- **`index_template.html`** - Template for generated HTML file
- **`style_template.css`** - Template for generated CSS file
- **`main_template.js`** - Template for generated JavaScript file

## Complete Feature Set

### Core Features
- HTTP/HTTPS server with Actix-web
- Static file serving from specified directory
- POST request handling with JSON echo response
- Template file initialization (--init flag)
- Self-test endpoint for POST validation

### Advanced Features

#### SSL/TLS Support
- PEM format certificates (separate cert and key files)
- PKCS12/PFX format certificates (combined file)
- Passphrase support for encrypted certificates
- Automatic format detection based on file extension

#### Configuration System
- JSON configuration file support
- Multiple config file formats: serve.json, now.json, package.json
- Configuration precedence rules
- Schema validation with helpful error messages
- Custom config path via --config flag

#### Logging & Monitoring
- Colored terminal output with log levels (INFO, WARN, ERROR)
- Timestamp formatting for all messages
- Request/response logging with timing
- Configurable via --no-request-logging flag

#### Network Features
- CORS support with configurable headers
- Gzip compression (enabled by default)
- Port availability checking
- Automatic port switching when occupied
- Network interface detection for external IP

#### Web Development Features
- Single Page Application (SPA) mode
- ETag and Last-Modified caching headers
- Symlinks support
- Clean URLs and trailing slash handling
- URL rewrites and redirects (via config)

#### User Experience
- Clipboard integration (auto-copy server URL)
- Colored server startup messages
- Graceful shutdown on signals
- Smart error messages and hints

## CLI Arguments Reference

### Required Arguments
- `--port, -p <PORT>` - Port number to serve on
- `--dir, -d <DIRECTORY>` - Directory to serve files from

### Optional Arguments

#### Basic Options
- `--init` - Initialize basic web files (index.html, style.css, main.js)
- `--test` - Enable self-test endpoint at /self-test
- `--config <PATH>` - Specify custom path to configuration file

#### SSL/TLS Options
- `--ssl-cert <PATH>` - Path to SSL/TLS certificate (PEM or PKCS12)
- `--ssl-key <PATH>` - Path to private key (for PEM certificates)
- `--ssl-pass <PATH>` - Path to passphrase file

#### Web Features
- `--cors` - Enable CORS headers
- `--single` - SPA mode (rewrite not-found to index.html)
- `--no-compression` - Disable gzip compression
- `--symlinks` - Follow symbolic links
- `--no-etag` - Use Last-Modified instead of ETag

#### Development Options
- `--no-request-logging` - Disable request logging
- `--no-clipboard` - Don't copy URL to clipboard
- `--no-port-switching` - Don't auto-switch ports

## Dependencies

### Web Framework
- `actix-web` - Core web server framework
- `actix-files` - Static file serving
- `actix-multipart` - Multipart form handling
- `actix-cors` - CORS middleware
- `awc` - Actix Web Client for self-testing

### TLS/Security
- `rustls` - TLS implementation
- `rustls-pemfile` - PEM file parsing
- `tokio-rustls` - Async TLS
- `p12` - PKCS12 certificate support
- `actix-web-httpauth` - HTTP authentication

### Configuration & Parsing
- `clap` - Command-line argument parsing
- `serde` & `serde_json` - JSON serialization
- `jsonschema` - JSON schema validation
- `mime` - MIME type handling
- `urlencoding` - URL encoding/decoding

### Utilities
- `tokio` - Async runtime
- `futures-util` - Async utilities
- `bytes` - Byte buffer utilities
- `chrono` - Date and time handling
- `colored` - Terminal colors
- `clipboard` - Clipboard access
- `port_check` - Port availability checking
- `local-ip-address` - Network interface detection
- `flate2` - Compression support
- `signal-hook` & `signal-hook-tokio` - Signal handling
- `atty` - Terminal detection

### Development
- `env_logger` & `log` - Logging framework
- `tempfile` - Temporary file handling (dev dependency)

## Configuration File Format

### serve.json Example

```json
{
  "public": "dist",
  "cleanUrls": true,
  "trailingSlash": false,
  "rewrites": [
    { "source": "/api/(.*)", "destination": "/api/$1.html" },
    { "source": "/user/:id", "destination": "/users/:id.html" },
    { "source": "/images/*.(jpg|png|gif)", "destination": "/assets/$1.$2" }
  ],
  "headers": [
    {
      "source": "**/*.@(jpg|jpeg|gif|png)",
      "headers": [
        { "key": "Cache-Control", "value": "max-age=7200" }
      ]
    }
  ],
  "directoryListing": false,
  "etag": true,
  "symlinks": false,
  "compress": true
}
```

### URL Rewrites

msaada supports powerful URL rewriting with multiple pattern styles, similar to Vercel's serve-handler:

#### Static Rewrites
```json
{
  "rewrites": [
    { "source": "/about", "destination": "/about.html" }
  ]
}
```

#### Capture Group Substitution (Regex Style)
Use `(.*)` in source and `$1`, `$2`, etc. in destination:
```json
{
  "rewrites": [
    { "source": "/api/(.*)", "destination": "/api/$1.html" },
    { "source": "/blog/(.*)/(.*)", "destination": "/posts/$1-$2.html" }
  ]
}
```

#### Named Parameters (path-to-regexp Style)
Use `:paramName` syntax for cleaner, more readable routes:
```json
{
  "rewrites": [
    { "source": "/user/:id", "destination": "/users/:id.html" },
    { "source": "/post/:year/:slug", "destination": "/content/:year-:slug.html" }
  ]
}
```

#### Optional Parameters
Use `{/:param}` for optional segments:
```json
{
  "rewrites": [
    { "source": "/products{/:category}", "destination": "/shop/:category.html" }
  ]
}
```
Matches both `/products` and `/products/electronics`.

#### Glob Patterns
Use wildcards for flexible matching:
```json
{
  "rewrites": [
    { "source": "/img/*.jpg", "destination": "/images/$1.jpg.html" },
    { "source": "/api/**/users", "destination": "/api/$1/users.html" },
    { "source": "/files/?.txt", "destination": "/documents/$1.txt" }
  ]
}
```

- `*` - Matches any characters within a single path segment
- `**` - Matches across multiple path segments
- `?` - Matches exactly one character

#### Brace Expansion
Use `{option1,option2}` for alternative matches:
```json
{
  "rewrites": [
    { "source": "/images/*.(jpg|png|gif)", "destination": "/assets/$1.$2" }
  ]
}
```

#### Rewrite Precedence
Rewrites are evaluated in order. The first matching rule wins:
```json
{
  "rewrites": [
    { "source": "/api/special", "destination": "/special.html" },
    { "source": "/api/(.*)", "destination": "/api/$1.html" },
    { "source": "**", "destination": "/index.html" }
  ]
}
```

#### Complete Example
```json
{
  "rewrites": [
    // Named parameters for user profiles
    { "source": "/user/:username", "destination": "/users/:username.html" },

    // Capture groups for API versioning
    { "source": "/api/v(.*)/(.*)/(.*)", "destination": "/api/v$1/$2/$3.json" },

    // Glob patterns for image routing
    { "source": "/img/**/*.{jpg,png,gif}", "destination": "/assets/images/$1.$2" },

    // Optional category parameter
    { "source": "/shop{/:category}", "destination": "/products/:category.html" },

    // Static rewrites for legacy URLs
    { "source": "/old-path", "destination": "/new-path.html" }
  ]
}
```

### Configuration Precedence

1. Command-line arguments (highest priority)
2. serve.json (if exists)
3. now.json (if exists)
4. package.json "static" field (if exists)
5. Default values (lowest priority)

## POST Request API

### Endpoint
Any path accepts POST requests and returns data as JSON.

### Supported Content Types
1. **JSON**: `application/json`
2. **Form Data**: `application/x-www-form-urlencoded`
3. **Multipart Form**: `multipart/form-data` (with file uploads)
4. **Plain Text**: `text/plain`
5. **Binary**: Any other content type

### Response Format

```json
{
  "path": "/api/endpoint",
  "content_type": "application/json",
  "json_data": { ... },
  "form_data": { ... },
  "text_data": "...",
  "files": [
    {
      "field_name": "upload",
      "filename": "document.pdf"
    }
  ]
}
```

## Testing

### Unit Tests
Each module includes unit tests. Run with:
```bash
cargo test                 # All tests
cargo test config::        # Module tests
cargo test -- --nocapture  # With output
```

### Integration Tests
```bash
./tests/run_comprehensive_tests.sh           # Full test suite (600+ tests)
./tests/run_comprehensive_tests.sh --quick   # Quick test mode
./tests/run_comprehensive_tests.sh http post # Selective testing
```

### Manual Testing
```bash
# Start with test endpoint
cargo run -- --port 3000 --dir . --test

# Visit in browser
http://localhost:3000/self-test
```

## Troubleshooting

### Common Issues

1. **Port Already in Use**
   - Server auto-switches to available port
   - Use `--no-port-switching` to force specific port

2. **Certificate Errors**
   - Verify certificate format (PEM vs PKCS12)
   - Check file paths are correct
   - Ensure private key matches certificate

3. **Configuration Not Loading**
   - Check JSON syntax is valid
   - Verify file is named correctly (serve.json)
   - Use `--config` to specify custom path

4. **SPA Routing Issues**
   - Enable `--single` flag for client-side routing
   - Configure rewrites in serve.json if needed

5. **CORS Errors**
   - Add `--cors` flag to enable CORS headers
   - Configure specific origins in serve.json

## Architecture Details

### Request Flow
1. Request received by Actix-web server
2. Middleware processing (CORS, compression, headers)
3. Route matching (POST handler or static files)
4. Response generation with appropriate headers
5. Logging and metrics collection

### Module Responsibilities

- **Core Server** (`main.rs`): Initialization, routing, middleware setup
- **Configuration** (`config.rs`): File parsing, validation, precedence
- **Logging** (`logger.rs`): Structured output, formatting, colors
- **Network** (`network.rs`): Port management, IP detection
- **TLS** (`tls.rs`): Certificate loading, format detection
- **Shutdown** (`shutdown.rs`): Signal handling, cleanup
- **SPA** (`spa.rs`): Route rewriting, fallback handling
- **Clipboard** (`clipboard.rs`): Cross-platform clipboard access

### Error Handling Strategy

- Use custom error types for each module
- Propagate errors with `?` operator
- Provide helpful error messages to users
- Log errors with appropriate severity levels
- Graceful fallbacks where possible

## Future Development

### Planned Features
- Directory listing UI with customizable templates
- WebSocket support for live reload
- Request/response middleware plugins
- Authentication and authorization options
- Metrics and monitoring endpoints
- HTTP/2 and HTTP/3 support

### Performance Optimizations
- Caching layer for frequently accessed files
- Connection pooling for keep-alive
- Optimized static file serving
- Memory-mapped file support

### Security Enhancements
- Rate limiting middleware
- Security headers by default
- CSP (Content Security Policy) support
- Request validation and sanitization

## Contributing Guidelines

1. **Code Quality**
   - Run `cargo fmt` before committing
   - Run `cargo clippy` and fix warnings
   - Add tests for new functionality
   - Update documentation

2. **Pull Requests**
   - Create feature branch from main
   - Write descriptive commit messages
   - Include tests and documentation
   - Ensure CI passes

3. **Testing Requirements**
   - Unit tests for new modules
   - Integration tests for features
   - Manual testing checklist
   - Performance benchmarks for critical paths

## Continuous Integration

The project uses GitHub Actions for automated testing and deployment:

### Workflows

1. **CI Pipeline** (`.github/workflows/ci.yaml`)
   - Runs on: push to main/develop, PRs
   - Tests on: Ubuntu, macOS, Windows
   - Steps: Unit tests → Build → Integration tests → Lint/Security
   - 600+ integration tests across all features

2. **Quick Tests** (`.github/workflows/test.yaml`)
   - Runs on: PRs with code changes
   - Ubuntu-only for fast feedback
   - Includes PR comment with test results
   - Quick mode integration testing

3. **Release Pipeline** (`.github/workflows/release.yaml`)
   - Runs on: GitHub releases
   - Test gate: Full test suite must pass
   - Multi-platform binary builds
   - Automated release artifact creation

### Test Coverage

The comprehensive test suite covers:
- HTTP server functionality (static files, routing)
- HTTPS/SSL integration (PEM, PKCS12 certificates)
- Configuration system (JSON parsing, precedence)
- POST request handling (JSON, form, multipart, binary)
- Advanced features (CORS, compression, SPA mode)
- Network management (port switching, IPv6)

### CI/CD Best Practices

- **Quality Gates**: Tests must pass before merge/release
- **Multi-Platform**: Ensures compatibility across OS
- **Fast Feedback**: Quick tests for PRs, full suite for releases
- **Artifact Collection**: Test logs preserved for debugging
- **Security Scanning**: Automated dependency audits

## License

MIT License - See LICENSE file for details

## Copyright

Copyright © 2022-2025 Vincent Bruijn (vebruijn@gmail.com)