zinit 0.2.1

A process supervisor with Rhai scripting support
Documentation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
# Zinit Rhai Integration - Technical Documentation

This document explains the complete architecture of Zinit's Rhai scripting integration, including the socket protocol, real-time output streaming, and how all the pieces fit together.

## Architecture Overview

```
┌─────────────────┐         Unix Socket          ┌─────────────────────┐
│  zinit client   │◄───────────────────────────►│   zinit daemon      │
│  (CLI tool)     │    ~/hero/var/zinit.sock    │   (background)      │
└─────────────────┘                              └─────────────────────┘
        │                                                 │
        │ sends script                                    │ executes script
        │ receives streamed output                        │ via Rhai engine
        ▼                                                 ▼
┌─────────────────┐                              ┌─────────────────────┐
│  script.rhai    │                              │   ZinitEngine       │
│  or stdin       │                              │   (Rhai + zinit)    │
└─────────────────┘                              └─────────────────────┘
```

## Key Components

### 1. Fixed Paths

Zinit uses hardcoded paths - no configuration needed:

- **Config directory**: `~/hero/cfg/zinit/` - Service YAML files
- **Socket file**: `~/hero/var/zinit.sock` - Unix domain socket for RPC
- **Binary**: `~/hero/bin/zinit` - The zinit executable

Only ONE zinit instance can run at a time. The socket file acts as a lock.

### 2. Daemon Modes

```bash
zinit start       # Foreground - blocks, prints to terminal
zinit start -bg   # Background - daemonizes via double-fork
```

Background mode uses Unix double-fork to properly daemonize:
1. First fork - parent waits for socket to be ready
2. Second fork - creates session leader, detaches from terminal
3. Child runs the actual daemon with tokio runtime

**Important**: The fork must happen BEFORE creating the tokio runtime, as tokio doesn't survive fork.

### 3. Socket Protocol

The protocol is simple line-based with a delimiter:

**Client → Server:**
```
<script line 1>
<script line 2>
...
===
```

The `===` (two or more `=` characters) signals "execute now".

**Server → Client:**
```
ok
<streamed output line 1>
<streamed output line 2>
...
=====
```

Or on error:
```
ok
<any output before error>
ERROR: <error message>
=====
```

The `=====` marks end of response.

## Real-Time Output Streaming

### The Problem

Rhai scripts run synchronously in a blocking context. Without streaming, all `print()` output would be buffered and only returned after the script completes.

### The Solution

We use a callback mechanism with channels to stream output as it happens:

```
┌──────────────┐    callback     ┌──────────────┐    channel    ┌──────────────┐
│ Rhai Engine  │───────────────►│ Output Sender │─────────────►│ Socket Writer│
│  print()     │                 │  (closure)    │              │  (async task)│
└──────────────┘                 └──────────────┘              └──────────────┘
```

### Implementation Details

#### 1. Global Output Sender (`src/rhai_api/engine.rs`)

```rust
// Type alias for the output sender callback
type OutputSender = Box<dyn Fn(&str) + Send + Sync>;

lazy_static::lazy_static! {
    static ref OUTPUT_SENDER: Mutex<Option<OutputSender>> = Mutex::new(None);
    static ref OUTPUT_BUFFER: Mutex<String> = Mutex::new(String::new());
}

/// Set callback for real-time output
pub fn set_output_sender<F>(sender: F)
where
    F: Fn(&str) + Send + Sync + 'static,
{
    if let Ok(mut guard) = OUTPUT_SENDER.lock() {
        *guard = Some(Box::new(sender));
    }
}

/// Clear the output sender
pub fn clear_output_sender() {
    if let Ok(mut guard) = OUTPUT_SENDER.lock() {
        *guard = None;
    }
}

/// Called by Rhai's on_print callback
fn append_output(s: &str) {
    // Try callback first (streaming mode)
    if let Ok(guard) = OUTPUT_SENDER.lock() {
        if let Some(ref sender) = *guard {
            sender(s);
            return;
        }
    }
    // Fall back to buffer (non-streaming mode)
    if let Ok(mut buf) = OUTPUT_BUFFER.lock() {
        buf.push_str(s);
        buf.push('\n');
    }
}
```

#### 2. Rhai Engine Print Hooks (`src/rhai_api/engine.rs`)

```rust
fn register_utility_functions(engine: &mut Engine) {
    // Capture print() calls
    engine.on_print(|s| {
        append_output(s);
    });
    
    // Capture debug() calls
    engine.on_debug(|s, source, pos| {
        let location = if let Some(src) = source {
            format!("[{}:{}] ", src, pos)
        } else if !pos.is_none() {
            format!("[{}] ", pos)
        } else {
            String::new()
        };
        append_output(&format!("[DEBUG] {}{}", location, s));
    });
}
```

#### 3. Socket Handler with Streaming (`src/rhai_api/socket_server.rs`)

```rust
async fn handle_connection(stream: UnixStream, zinit: Arc<RwLock<ZInit>>) {
    // ... read script ...
    
    // Send "ok" header immediately
    writer.write_all(b"ok\n").await?;
    
    // Create channel for streaming
    let (tx, rx) = std::sync::mpsc::channel::<String>();
    
    // Spawn async task to write output as it arrives
    let writer_clone = writer.clone();
    let output_writer = tokio::spawn(async move {
        while let Ok(output) = rx.recv() {
            let mut w = writer_clone.lock().await;
            w.write_all(output.as_bytes()).await;
            w.flush().await;  // Important: flush for real-time
        }
    });
    
    // Execute script with streaming callback
    execute_script_streaming(&script, zinit, tx).await;
    
    // Wait for output writer to finish, send delimiter
    output_writer.await;
    writer.write_all(b"=====\n").await;
}

async fn execute_script_streaming(
    script: &str,
    zinit: Arc<RwLock<ZInit>>,
    output_tx: std::sync::mpsc::Sender<String>,
) {
    tokio::task::spawn_blocking(move || {
        // Set up streaming callback
        let tx = output_tx.clone();
        super::engine::set_output_sender(move |s: &str| {
            tx.send(format!("{}\n", s));
        });
        
        // Run the script
        engine.run(&script);
        
        // Clean up
        super::engine::clear_output_sender();
    }).await;
}
```

#### 4. Client Real-Time Display (`src/rhai_api/socket_server.rs`)

```rust
pub async fn send_script(socket_path: &Path, script: &str) -> Result<String> {
    // Send script with delimiter
    stream.write_all(script.as_bytes()).await?;
    stream.write_all(b"\n===\n").await?;
    
    // Read status line
    reader.read_line(&mut status_line).await?;
    
    // Stream output to stdout in real-time
    loop {
        reader.read_line(&mut line).await?;
        if is_execute_marker(&line) { break; }
        
        // Print immediately
        print!("{}", line);
        std::io::stdout().flush();
    }
}
```

## Auto-Start Feature

When running `zinit script.rhai`, zinit automatically starts if not running:

```rust
fn ensure_zinit_running() -> bool {
    // Check if already running via ping
    if ping(&socket_path) {
        return true;
    }
    
    // Start in background
    Command::new("zinit")
        .args(["start", "-bg"])
        .spawn();
    
    // Wait for socket to be ready
    for _ in 0..30 {
        sleep(100ms);
        if ping(&socket_path) {
            return true;
        }
    }
    false
}
```

## Logging Architecture

Zinit has its own logging for services (separate from Rhai `print()`):

### Service Log Modes

```rhai
zinit_service_new()
    .log("ring")    // Ring buffer (default) - keeps last N lines in memory
    .log("stdout")  // Forward to zinit's stdout
    .log("none")    // Discard all output
```

### Rhai Output vs Service Logs

| Source | Destination | When |
|--------|-------------|------|
| `print()` in Rhai | Streamed to client via socket | During script execution |
| Service stdout/stderr | Ring buffer or zinit stdout | While service runs |
| `zinit_status_md()` | Return value to client | When called |

## File Structure

```
src/
├── main.rs                 # CLI entry point, argument parsing
├── app.rs                  # Paths, initialization, daemonization
├── lib.rs                  # Library exports
├── instructions.md         # AI agent instructions (embedded)
└── rhai_api/
    ├── mod.rs              # Module exports
    ├── engine.rs           # ZinitEngine, print capture, function registration
    ├── socket_server.rs    # Unix socket server, streaming protocol
    ├── service_builder.rs  # ServiceBuilder pattern for Rhai
    └── types.rs            # Type definitions
```

## Dependencies

All key crates used in zinit:

```toml
[dependencies]
# Core
anyhow = "1.0"                    # Error handling
tokio = { version = "1.44", features = ["full", "net", "io-util", "sync"] }  # Async runtime
tokio-stream = "0.1.17"           # Async streams

# Rhai Scripting
rhai = { version = "1.19", features = ["sync", "serde"] }  # Scripting engine
lazy_static = "1.4"               # Global state for output sender

# Interactive REPL (TUI)
rustyline = { version = "14.0", features = ["derive"] }  # Readline with completion/history
colored = "2.1"                   # Terminal colors for syntax highlighting

# Unix/System
nix = "0.22.1"                    # Unix-specific (fork, signals, processes)
libc = "0.2"                      # Low-level C bindings
command-group = "1.0.8"           # Process group management
sysinfo = "0.29.10"               # System/process information (CPU, memory)

# Serialization
serde = { version = "1.0", features = ["derive"] }  # Serialization framework
serde_yaml = "0.8"                # YAML config files
serde_json = "1.0"                # JSON support

# Utilities
dirs = "5.0"                      # Home directory detection
shlex = "1.1"                     # Shell-like argument parsing
clap = { version = "4.0", features = ["derive"] }  # CLI argument parsing
fern = "0.6"                      # Logging
log = "0.4"                       # Logging facade
thiserror = "1.0"                 # Error derive macros
git-version = "0.3.5"             # Embed git version
```

### Key Library Purposes

| Library | Purpose |
|---------|---------|
| `rhai` | Embedded scripting language for service configuration |
| `rustyline` | Interactive REPL with readline-style editing, tab completion, history |
| `colored` | Terminal color output for syntax highlighting |
| `tokio` | Async runtime for socket server and concurrent operations |
| `lazy_static` | Global mutable state for output streaming callback |
| `nix` | Unix process control (fork, signals, process groups) |
| `sysinfo` | Query CPU/memory usage of services |

## Common Patterns

### Blocking Rhai in Async Context

Rhai is synchronous, but zinit uses tokio. We bridge them with `spawn_blocking`:

```rust
tokio::task::spawn_blocking(move || {
    let rt = tokio::runtime::Handle::current();
    
    // Access async zinit from sync context
    let result = rt.block_on(async {
        zinit.read().await.some_method().await
    });
    
    // Run sync Rhai code
    engine.run(&script)
}).await
```

### Macro for Async Zinit Functions

```rust
macro_rules! block_on {
    ($fut:expr) => {
        tokio::task::block_in_place(|| 
            tokio::runtime::Handle::current().block_on($fut)
        )
    };
}

// Usage in registered function
engine.register_fn("zinit_start", move |name: &str| -> bool {
    let z = zinit.clone();
    block_on!(async {
        let zinit = z.read().await;
        zinit.start(&name).await.is_ok()
    })
});
```

## Testing

### Manual Testing

```bash
# Terminal 1: Start daemon in foreground to see logs
zinit start

# Terminal 2: Run scripts
zinit rhaiexamples/01_full_test.rhai

# Or test streaming
echo 'print("line 1"); sleep(1); print("line 2");' | zinit -i
```

### Verify Streaming

Create a test script with delays:

```rhai
// rhaiexamples/02_stream_test.rhai
print("Starting...");
sleep(1);
print("1 second");
sleep(1);
print("2 seconds");
print("Done!");
```

Run it and observe output appears line-by-line with delays, not all at once.

## Troubleshooting

### Socket Issues

```bash
# Check if zinit is running
ls -la ~/hero/var/zinit.sock

# Kill stale instance
pkill -9 zinit
rm -f ~/hero/var/zinit.sock

# Start fresh
zinit start -bg
```

### Output Not Streaming

1. Check `flush()` is called after each write
2. Verify output sender callback is set before script runs
3. Check channel isn't dropping messages

### Function Not Found

Ensure the function is registered with correct argument types. Rhai is strongly typed - `zinit_stop("name")` won't match `zinit_stop()` (no args).

## Interactive REPL (TUI Shell)

The `zinit --ui` command starts an interactive shell with advanced features.

### Architecture

```
┌─────────────────────────────────────────────────────────────┐
│                    zinit --ui                               │
├─────────────────────────────────────────────────────────────┤
│  rustyline (readline)                                       │
│  ├── Tab Completion (ReplHelper)                            │
│  ├── Syntax Highlighting (Highlighter trait)                │
│  ├── History (~/.zinit_history)                             │
│  └── Hints (function signatures)                            │
├─────────────────────────────────────────────────────────────┤
│  REPL Commands (/help, /load, /functions, etc.)             │
├─────────────────────────────────────────────────────────────┤
│  Socket Client (send_script)                                │
│  └── Streams output to terminal in real-time                │
└─────────────────────────────────────────────────────────────┘
```

### Key Components (`src/repl.rs`)

#### 1. ReplHelper Struct

Implements rustyline traits for completion, highlighting, and hints:

```rust
#[derive(Helper)]
struct ReplHelper {
    hinter: HistoryHinter,
}

impl Completer for ReplHelper { ... }
impl Hinter for ReplHelper { ... }
impl Highlighter for ReplHelper { ... }
impl Validator for ReplHelper {}
```

#### 2. Function Definitions

All zinit functions are defined with signatures and descriptions for completion/help:

```rust
const ZINIT_FUNCTIONS: &[(&str, &str, &str)] = &[
    // (signature, return_type, description)
    ("zinit_ping()", "bool", "Check if zinit is responding"),
    ("zinit_start()", "bool", "Start zinit daemon if not running"),
    ("zinit_service_new()", "ServiceBuilder", "Create a new service builder"),
    // ... more functions
];

const BUILDER_METHODS: &[(&str, &str)] = &[
    (".exec(cmd)", "Command to run (required)"),
    (".test(cmd)", "Health check command"),
    // ... more methods
];

const REPL_COMMANDS: &[(&str, &str)] = &[
    ("/help", "Show this help"),
    ("/functions", "List all zinit functions"),
    ("/load <file>", "Load and execute a .rhai script"),
    // ... more commands
];
```

#### 3. Tab Completion

```rust
impl Completer for ReplHelper {
    fn complete(&self, line: &str, pos: usize, _ctx: &Context<'_>) 
        -> rustyline::Result<(usize, Vec<Pair>)> 
    {
        // Find word being typed
        let word = &line[word_start..pos];
        
        // Complete REPL commands (/help, /load, etc.)
        if word.starts_with('/') {
            for (cmd, desc) in REPL_COMMANDS { ... }
        }
        
        // Complete builder methods (.exec, .log, etc.)
        if word.starts_with('.') {
            for (method, desc) in BUILDER_METHODS { ... }
        }
        
        // Complete zinit functions
        for (func, ret, desc) in ZINIT_FUNCTIONS {
            if func_name.starts_with(word) { ... }
        }
        
        // Complete Rhai keywords
        for kw in RHAI_KEYWORDS { ... }
    }
}
```

#### 4. Syntax Highlighting

```rust
impl Highlighter for ReplHelper {
    fn highlight<'l>(&self, line: &'l str, _pos: usize) -> Cow<'l, str> {
        // Color different elements:
        // - Keywords (let, if, for) -> magenta bold
        // - zinit_* functions -> blue bold
        // - Strings ("...") -> green
        // - Numbers -> yellow
        // - Operators (=, +, -) -> yellow
        // - Brackets/parens -> cyan
        // - Comments (//) -> dimmed
    }
}

fn highlight_word(word: &str) -> String {
    if RHAI_KEYWORDS.contains(&word) {
        return format!("{}", word.magenta().bold());
    }
    if word.starts_with("zinit_") {
        return format!("{}", word.blue().bold());
    }
    // ... more rules
}
```

#### 5. Multi-line Input Detection

```rust
// Check for unclosed braces/brackets
let open_braces = buffer.matches('{').count();
let close_braces = buffer.matches('}').count();

if open_braces > close_braces {
    // Continue reading input (show "... " prompt)
    continue;
}

// Also support backslash continuation
if line.ends_with('\\') {
    continue;
}
```

#### 6. Script Loading

```rust
async fn load_script(file_path: &str, socket_path: &Path) -> Result<()> {
    // Expand ~ to home directory
    let expanded_path = if file_path.starts_with("~/") {
        dirs::home_dir().map(|h| h.join(&file_path[2..]))
    } else {
        PathBuf::from(file_path)
    };
    
    let script = std::fs::read_to_string(&expanded_path)?;
    send_script(socket_path, &script).await
}
```

### Dependencies for REPL

```toml
[dependencies]
rustyline = { version = "14.0", features = ["derive"] }
colored = "2.1"
```

### Adding a New REPL Command

1. Add to `REPL_COMMANDS` constant:
```rust
const REPL_COMMANDS: &[(&str, &str)] = &[
    // ...
    ("/mycommand", "Description of my command"),
];
```

2. Handle in the match statement in `run_repl()`:
```rust
match line {
    "/mycommand" => {
        println!("My command executed!");
        continue;
    }
    // ...
}
```

## Service Name Normalization

All service names are normalized to prevent confusion:

```rust
fn normalize_service_name(name: &str) -> String {
    name.replace('-', "_").to_lowercase()
}
```

This means:
- `Test-Service`  `test_service`
- `MY_SERVICE`  `my_service`
- `web-server`  `web_server`

Applied to all functions: `zinit_monitor`, `zinit_start`, `zinit_stop`, `zinit_status`, etc.

## Extending

### Adding a New Rhai Function

1. Add to `register_zinit_functions()` in `engine.rs`:

```rust
let z = zinit.clone();
engine.register_fn("zinit_my_func", move |arg: &str| -> bool {
    let z = z.clone();
    let name = normalize_service_name(arg);  // Don't forget normalization!
    block_on!(async {
        let zinit = z.read().await;
        zinit.my_method(&name).await.is_ok()
    })
});
```

2. Add to `ZINIT_FUNCTIONS` in `repl.rs` for completion:
```rust
("zinit_my_func(name)", "bool", "Description of my function"),
```

3. Document in `src/instructions.md`

### Adding a Builder Method

1. Add field to `ServiceBuilder` in `service_builder.rs`
2. Add method in `rhai_fns` module
3. Register in `register_types()`:

```rust
.register_fn("my_option", rhai_fns::service_my_option)
```

4. Add to `BUILDER_METHODS` in `repl.rs`:
```rust
(".my_option(value)", "Description of option"),
```