torc 0.22.2

Workflow management system
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
# Rust Developer Style Guide

This guide establishes coding standards, conventions, and workflows for Rust developers contributing
to Torc. Following these guidelines ensures consistency across the codebase and streamlines the
review process.

## Pre-commit Hooks

**Always rely on the pre-commit hooks provided.** The repository uses `cargo-husky` to install Git
hooks automatically. Before each commit, the following checks run:

```bash
cargo fmt -- --check    # Rust formatting
cargo clippy --all --all-targets --all-features -- -D warnings
dprint check            # Markdown formatting
```

If any check fails, the commit is blocked. Fix the issues before committing.

### Installing Pre-commit Hooks

Hooks are installed automatically when you run `cargo build` for the first time. If you need to
reinstall them manually:

```bash
cargo install cargo-husky
cargo build  # Triggers hook installation
```

## Code Formatting

### Rust Formatting (rustfmt)

All Rust code must pass `cargo fmt --check`. Run `cargo fmt` before committing to auto-format your
code.

**Key conventions enforced:**

- 4-space indentation
- Max line width of 100 characters
- Consistent brace placement
- Sorted imports

### Clippy Compliance

All code must compile without clippy warnings when run with `-D warnings`:

```bash
cargo clippy --all --all-targets --all-features -- -D warnings
```

**Common clippy lints to watch for:**

- `clippy::unwrap_used` - Prefer `expect()` with descriptive messages or proper error handling
- `clippy::clone_on_copy` - Avoid cloning Copy types
- `clippy::needless_return` - Omit unnecessary `return` keywords
- `clippy::redundant_closure` - Use method references where possible

### Markdown Formatting (dprint)

All Markdown files in `docs/` must comply with dprint formatting:

```bash
dprint check   # Verify formatting
dprint fmt     # Auto-format
```

**Critical requirement: Maximum line length of 100 characters for all Markdown files.**

The `dprint.json` configuration enforces:

```json
{
  "lineWidth": 100,
  "markdown": {
    "lineWidth": 100,
    "textWrap": "always"
  }
}
```

## Documentation Standards

All features must be documented in Markdown in the `docs/` directory following the
[Diataxis framework](https://diataxis.fr/):

### Diataxis Categories

| Category          | Location                | Purpose                                        |
| ----------------- | ----------------------- | ---------------------------------------------- |
| **Tutorials**     | `docs/src/tutorials/`   | Learning-oriented, step-by-step lessons        |
| **How-To Guides** | `docs/src/how-to/`      | Task-oriented, problem-solving guides          |
| **Explanation**   | `docs/src/explanation/` | Understanding-oriented, conceptual discussions |
| **Reference**     | `docs/src/reference/`   | Information-oriented, technical descriptions   |

### Design Documentation

Significant design choices must be documented in `docs/src/explanation/design/`. Each design
document should cover:

- **Problem Statement**: What problem does this solve?
- **Design Goals**: What are the requirements and constraints?
- **Solution Overview**: High-level architecture description
- **Implementation Details**: Key technical decisions and trade-offs
- **Alternatives Considered**: What other approaches were evaluated?

Existing design documents include:

- `server.md` - API handler design and request processing
- `database.md` - SQLite schema and concurrency model
- `dashboard.md` - Web dashboard architecture
- `recovery.md` - Workflow recovery mechanisms
- `workflow-graph.md` - Dependency graph implementation

### Documentation Workflow

1. Write documentation alongside code changes
2. Add new pages to `docs/src/SUMMARY.md`
3. Run `dprint fmt` to ensure formatting compliance
4. Build and preview with `mdbook serve docs/`

## Testing with rstest

All code must include tests using the `rstest` library for fixtures and parameterized testing.

### Test Organization

```
tests/
├── common.rs              # Shared test utilities and fixtures
├── test_full_workflows.rs # Integration tests
├── test_job_runner.rs     # Job runner tests
└── scripts/               # Helper scripts for tests
```

### Common Patterns

**Fixture Pattern:**

```rust
use rstest::rstest;
use serial_test::serial;

mod common;
use common::{start_server, ServerProcess};

#[rstest]
#[serial]
fn test_workflow_creation(start_server: &ServerProcess) {
    let config = &start_server.config;
    // Test code using the server fixture
}
```

**Parameterized Tests:**

```rust
#[rstest]
#[case(0, "immediate")]
#[case(60, "one_minute")]
#[case(3600, "one_hour")]
#[serial]
fn test_timeout_handling(#[case] timeout_secs: u64, #[case] description: &str) {
    // Test runs once for each case
}
```

**Shared Test Utilities (`tests/common.rs`):**

```rust
pub struct ServerProcess {
    pub config: Configuration,
    child: std::process::Child,
}

impl Drop for ServerProcess {
    fn drop(&mut self) {
        // Automatic cleanup on test completion
        let _ = self.child.kill();
    }
}

pub fn start_server() -> ServerProcess {
    let port = find_available_port();
    // Start server and wait for readiness
}
```

### Test Guidelines

1. **Use `#[serial]`** for integration tests that share resources (ports, database)
2. **Use descriptive `expect()` messages** instead of `.unwrap()`
3. **Clean up resources** using the Drop trait or explicit cleanup functions
4. **Test error conditions** not just happy paths
5. **Keep tests focused** - one behavior per test function

## HTTP API Changes

Changes to the HTTP API require updating the OpenAPI specification and regenerating client
libraries.

### Workflow

1. **Modify the OpenAPI spec:**

   ```bash
   # Update the Rust-owned API document, live handler, and models
   vim src/openapi_spec.rs
   ```

2. **Regenerate API clients:**

   ```bash
   cd api
   bash sync_openapi.sh emit
   bash sync_openapi.sh check
   bash sync_openapi.sh clients --use-rust-spec
   ```

   This regenerates:
   - Rust client: `src/client/apis/`
   - Python client: `python_client/src/torc/openapi_client/`
   - Julia client: `julia_client/Torc/src/api/`

   `clients --use-rust-spec` uses the current `api/openapi.codegen.yaml`. Run `emit` first so the
   generated clients reflect the latest Rust-owned contract, and `check` to verify the checked-in
   artifacts are still in sync.

   When the Rust-emitted spec should become the checked-in contract artifact, run:

   ```bash
   cd api
   bash sync_openapi.sh all --promote
   ```

   That updates both spec artifacts from Rust and regenerates external clients from the promoted
   contract.

3. **Update Rust generation assets as needed:**

   The Rust client and server still have legacy generated files in `src/`. Do not hand-edit those
   generated surfaces. Keep OpenAPI/client generation deterministic through `api/sync_openapi.sh`.

4. **Test all clients:**

   ```bash
   # Rust
   cargo nextest run --all-features

   # Python
   cd python_client && pytest

   # Julia
   julia --project=julia_client/Torc -e "import Pkg; Pkg.test()"
   ```

### OpenAPI Conventions

- Use descriptive `operationId` values (e.g., `create_workflow`, `list_jobs`)
- Include comprehensive request/response schemas
- Document all parameters with descriptions
- Use appropriate HTTP status codes (200, 400, 404, 500)

## Feature Implementation Across Interfaces

When implementing a user-facing feature, ensure it is exposed through the appropriate interfaces.
The following table shows where features should be implemented:

| Interface        | Location                                                            | Primary Use Case                                                         |
| ---------------- | ------------------------------------------------------------------- | ------------------------------------------------------------------------ |
| CLI              | `src/client/commands/`                                              | Command-line automation, scripting                                       |
| HTTP API         | `src/openapi_spec.rs`, `src/server/live_router.rs`, `src/models.rs` | Rust-owned API contract source                                           |
| OpenAPI artifact | `api/openapi.yaml`                                                  | Checked-in emitted contract for Python/Julia integration, external tools |
| Dashboard        | `torc-dash/src/`                                                    | Web-based monitoring and management                                      |
| TUI              | `src/tui/`                                                          | Interactive terminal monitoring                                          |
| MCP Server       | `torc-mcp-server/src/`                                              | AI assistant integration                                                 |

### CLI Implementation

Commands are implemented using `clap` with subcommand enums:

```rust
// In src/client/commands/<feature>.rs

#[derive(Subcommand, Debug, Clone)]
pub enum FeatureCommands {
    /// Create a new resource
    Create {
        /// Name of the resource
        #[arg(short, long)]
        name: String,
    },
    /// List all resources
    List {
        #[arg(long, default_value = "table")]
        format: String,
    },
}

pub fn handle_feature_commands(
    config: &Configuration,
    command: &FeatureCommands,
    format: &str,
) {
    match command {
        FeatureCommands::Create { name } => handle_create(config, name, format),
        FeatureCommands::List { format: fmt } => handle_list(config, fmt),
    }
}
```

**CLI Conventions:**

- Support both `--format table` and `--format json` output
- Use `tabled` for table formatting with `#[tabled(rename = "...")]` for column headers
- Include pagination support via `--offset` and `--limit` flags
- Provide helpful error messages with context

### HTTP API (Python/Julia)

After updating the Rust-owned API contract and promoting the emitted spec, the Python and Julia
clients are auto-generated. Ensure:

1. All new endpoints have proper request/response schemas
2. Query parameters are documented
3. Error responses are specified
4. Run `sync_openapi.sh clients` to regenerate clients

### Dashboard (torc-dash)

The dashboard is an Axum-based web server with embedded static assets:

```rust
// In torc-dash/src/main.rs

async fn handle_feature_list(
    State(state): State<AppState>,
) -> Result<Json<Vec<Feature>>, StatusCode> {
    // Proxy request to Torc API server
    let features = state.client
        .get(&format!("{}/features", state.api_url))
        .send()
        .await
        .map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?
        .json()
        .await
        .map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;

    Ok(Json(features))
}
```

**Dashboard Conventions:**

- Proxy API requests to the Torc server
- Use Axum extractors for request handling
- Return JSON for API endpoints
- Serve static files for the frontend

### TUI (Terminal User Interface)

The TUI uses `ratatui` with a component-based architecture:

```rust
// In src/tui/app.rs

pub struct App {
    pub workflows: Vec<WorkflowModel>,
    pub selected_workflow: Option<usize>,
    pub detail_view: DetailViewType,
}

impl App {
    pub fn handle_key_event(&mut self, key: KeyEvent) -> AppAction {
        match key.code {
            KeyCode::Enter => self.select_current(),
            KeyCode::Char('r') => self.refresh_data(),
            KeyCode::Char('q') => AppAction::Quit,
            _ => AppAction::None,
        }
    }
}
```

**TUI Conventions:**

- Use `anyhow::Result` for error handling
- Separate state (`app.rs`), rendering (`ui.rs`), and API calls (`api.rs`)
- Support keyboard navigation with vim-style bindings
- Display confirmation dialogs for destructive actions

### MCP Server (AI Assistant)

The MCP server exposes tools for AI assistants:

```rust
// In torc-mcp-server/src/main.rs

pub fn get_workflow_status(
    config: &Configuration,
    workflow_id: i64,
) -> Result<CallToolResult, McpError> {
    let workflow = default_api::get_workflow(config, workflow_id)
        .map_err(|e| internal_error(format!("Failed to get workflow: {}", e)))?;

    let result = serde_json::json!({
        "workflow_id": workflow.id,
        "name": workflow.name,
        "status": workflow.status,
    });

    Ok(CallToolResult::success(vec![
        rmcp::model::Content::text(serde_json::to_string_pretty(&result).unwrap_or_default())
    ]))
}
```

**MCP Conventions:**

- Return structured JSON for tool results
- Use descriptive error messages via `McpError`
- Support common workflow operations (list, status, run, cancel)
- Keep tool descriptions clear for AI consumption

## Error Handling Strategy

### Application Code (CLI, TUI, binaries)

Use `anyhow::Result` for flexible error handling:

```rust
use anyhow::{Context, Result};

pub fn run_workflow(path: &Path) -> Result<()> {
    let spec = load_spec(path)
        .context("Failed to load workflow specification")?;

    create_workflow(&spec)
        .context("Failed to create workflow")?;

    Ok(())
}
```

### Library Code

Use typed errors with `thiserror`:

```rust
use thiserror::Error;

#[derive(Error, Debug)]
pub enum WorkflowError {
    #[error("Job {job_id} not found in workflow {workflow_id}")]
    JobNotFound { job_id: i64, workflow_id: i64 },

    #[error("Invalid status transition from {from} to {to}")]
    InvalidTransition { from: String, to: String },

    #[error("API error: {0}")]
    ApiError(#[from] reqwest::Error),
}
```

### Test Code

Use `.expect()` with descriptive messages:

```rust
let workflow = create_workflow(&spec)
    .expect("Test workflow creation should succeed");

let job = get_job(config, job_id)
    .expect("Job should exist after creation");
```

## Common Patterns

### Configuration Priority

CLI arguments override environment variables, which override config files:

```rust
let api_url = cli_args.url
    .or_else(|| env::var("TORC_API_URL").ok())
    .or_else(|| config.client.as_ref()?.api_url.clone())
    .unwrap_or_else(|| "http://localhost:8080/torc-service/v1".to_string());
```

### Table Display

Use the `tabled` crate for CLI table output:

```rust
use tabled::{Table, Tabled};

#[derive(Tabled)]
struct JobRow {
    #[tabled(rename = "ID")]
    id: i64,
    #[tabled(rename = "Name")]
    name: String,
    #[tabled(rename = "Status")]
    status: String,
}

fn display_jobs(jobs: &[JobModel]) {
    let rows: Vec<JobRow> = jobs.iter().map(|j| JobRow {
        id: j.id.unwrap_or(0),
        name: j.name.clone(),
        status: format!("{:?}", j.status),
    }).collect();

    println!("{}", Table::new(rows));
}
```

### Feature Flags

Use Cargo features to conditionally compile components:

```rust
// In Cargo.toml
[features]
default = ["client"]
client = ["dep:reqwest", "dep:clap"]
server = ["dep:sqlx", "dep:axum"]
tui = ["client", "dep:ratatui"]

// In code
#[cfg(feature = "client")]
pub mod client;

#[cfg(feature = "server")]
pub mod server;
```

### Async Runtime

Create blocking clients before spawning the async runtime to avoid nested runtime issues:

```rust
fn main() -> Result<()> {
    // Create blocking client BEFORE async runtime
    let client = reqwest::blocking::Client::new();
    let server = MyServer::new(client);

    let runtime = tokio::runtime::Builder::new_multi_thread()
        .worker_threads(4)
        .enable_all()
        .build()?;

    runtime.block_on(async_main(server))
}
```

## Logging

Use `tracing` for structured logging:

```rust
use tracing::{debug, info, warn, error, instrument};

#[instrument(skip(config))]
pub fn process_job(config: &Configuration, job_id: i64) -> Result<()> {
    info!(job_id, "Processing job");

    match run_job(job_id) {
        Ok(result) => {
            debug!(job_id, ?result, "Job completed successfully");
            Ok(())
        }
        Err(e) => {
            error!(job_id, error = %e, "Job failed");
            Err(e)
        }
    }
}
```

Enable debug logging with:

```bash
RUST_LOG=debug cargo run
RUST_LOG=torc=debug,sqlx=warn cargo run  # Fine-grained control
```

## Summary Checklist

Before submitting a pull request, verify:

- [ ] `cargo fmt --check` passes
- [ ] `cargo clippy --all --all-targets --all-features -- -D warnings` passes
- [ ] `dprint check` passes (for Markdown changes)
- [ ] All tests pass with `cargo nextest run --all-features`
- [ ] New features have tests using `rstest`
- [ ] Documentation added in appropriate Diataxis category
- [ ] Design decisions documented in `docs/src/explanation/design/` if applicable
- [ ] API changes reflected in the Rust-owned OpenAPI scaffold and promoted `api/openapi.yaml`
- [ ] Client libraries regenerated with `api/sync_openapi.sh clients`
- [ ] Feature exposed through appropriate interfaces (CLI, API, TUI, etc.)