systemg

Process supervisor with dependencies, health checks, and rolling deployments. Built on systemd/cgroups.

Table of Contents

1. Getting Started
   • 1.1 How It Works
   • 1.2 Installation
   • 1.3 Running a Basic Start Command
   • 1.3 How It Works (docs)
2. Why systemg
   • 2.1 Features
   • 2.2 Comparison
3. Development
   • 3.1 Testing
   • 3.2 Build from Source
   • 3.3 Contributing

Getting Started

Installation

$ curl --proto '=https' --tlsv1.2 -fsSL https://sh.sysg.dev/ | sh

System deployments: scripts/install-systemg.sh sets up /usr/bin/sysg, /etc/systemg, /var/lib/systemg. See security guide.

Usage

$ sysg start                     # Default config (systemg.yaml)
$ sysg start --config my.yaml    # Custom config
$ sysg start --daemonize         # Background supervisor

Commands: sysg stop, sysg restart, sysg status, sysg logs

Why systemg

Compose programs into systems with explicit dependencies and health checks.

Features

• Dependencies - Topological startup order with health-aware cascading
• Rolling Deployments - Blue-green swaps with health validation
• Environment - .env file propagation
• Webhooks - Event notifications (docs)
• Cron - Scheduled tasks with overlap detection
• Spawning - Dynamic child process tracking
• OS Integration - systemd/cgroups when available
• Single Binary - No runtime dependencies

Privileged Mode (Optional)

Need to manage system daemons, bind privileged ports, or attach cgroup limits? Run the supervisor in privileged mode:

# Start with elevated privileges and system-wide state directories
$ sudo sysg --sys start --config /etc/systemg/nginx.yaml --daemonize

# Bind as root, then immediately drop to the configured service user
$ sudo sysg --sys --drop-privileges start --service web

# Check status without elevated privileges (falls back to userspace mode)
$ sysg status --service web

In privileged mode systemg relocates state to /var/lib/systemg, writes supervisor logs to /var/log/systemg/supervisor.log, and respects the new service-level fields:

services:
  web:
    command: "./server"
    user: "www-data"
    group: "www-data"
    supplementary_groups: ["www-logs"]
    limits:
      nofile: 65536
      nproc: 4096
      memlock: "unlimited"      # supports K/M/G/T suffixes
      nice: -5
      cpu_affinity: [0, 1]
      cgroup:
        memory_max: "512M"
        cpu_max: "200000 100000"
    capabilities:
      - CAP_NET_BIND_SERVICE
      - CAP_SYS_NICE
    isolation:
      network: true
      pid: true
      mount: true

All privileged operations are opt-in: services that omit these fields continue to run unprivileged, and unit tests skip elevated scenarios automatically when not running as root.

Dependency Handling

Declare service relationships with the depends_on field to coordinate startup order and health checks. Systemg will:

• start services in a topologically sorted order so each dependency is running or has exited successfully before its dependents launch;
• skip dependents whose prerequisites fail to start, surfacing a clear dependency error instead of allowing a partial boot;
• stop running dependents automatically when an upstream service crashes, preventing workloads from running against unhealthy backends.

For example:

version: "1"
services:
  database:
    command: "postgres -D /var/lib/postgres"

  web:
    command: "python app.py"
    depends_on:
      - database

If database fails to come up, web will remain stopped and log the dependency failure until the database is healthy again.

Rolling Deployments

Services can opt into rolling restarts so existing instances keep serving traffic until replacements are healthy. Add a deployment block to configure the behavior:

version: "1"
services:
  api:
    command: "./target/release/api"
    restart_policy: "always"
    deployment:
      strategy: "rolling"          # default is "immediate"
      pre_start: "cargo build --release"
      health_check:
        url: "http://localhost:8080/health"
        timeout: "60s"
        retries: 5
      grace_period: "5s"

• strategy — set to rolling to enable the zero-downtime workflow, or omit to keep the traditional stop/start cycle.
• pre_start — optional shell command executed before the new instance launches (perfect for build or migrate steps).
• health_check — optional HTTP probe the replacement must pass before traffic flips; configure timeout and retry budget per service.
• grace_period — optional delay to keep the old instance alive after the new one passes health checks, giving load balancers time to rebalance.

If any rolling step fails, systemg restores the original instance and surfaces the error so unhealthy builds never replace running services.

Cron Scheduling

Services can be configured to run on a cron schedule for short-lived, recurring tasks. Cron jobs are managed by the supervisor and run independently of regular services:

version: "1"
services:
  backup:
    command: "sh backup-script.sh"
    cron:
      expression: "0 0 * * * *"  # Run every hour at minute 0
      timezone: "America/New_York"  # Optional, defaults to system timezone

Key features:

• Standard cron syntax - Uses 6-field cron expressions (second, minute, hour, day, month, day of week).
• Overlap detection - If a cron job is scheduled to run while a previous execution is still running, the new execution is skipped and an error is logged.
• Execution history - The last 10 executions are tracked with their start time, completion time, and status.
• Service separation - A service cannot have both a command for continuous running and a cron configuration; cron is opt-in via the cron field.

Note: Cron jobs do not support restart policies, as they are designed to be short-lived tasks that complete and exit.

Dynamic Process Spawning

Services can dynamically spawn child processes at runtime with full tracking and resource limits:

version: "1"
services:
  orchestrator:
    command: "python orchestrator.py"
    spawn:
      mode: "dynamic"
      limits:
        children: 100           # Direct children limit
        depth: 3                # Maximum spawn tree depth
        descendants: 500        # Total across all levels
        total_memory: "2GB"     # Shared by entire tree
        termination_policy: "cascade"  # Clean up all children on exit

Key features:

• Process Tree Tracking - Child processes inherit parent's monitoring and logging
• Resource Limits - Configurable limits prevent fork bombs and runaway spawning
• Rate Limiting - Built-in protection (10 spawns/second) prevents abuse
• Flexible Spawning - Support for both traditional workers and LLM-powered agents
• TTL Support - Automatic cleanup of temporary processes after specified duration

From your application code, spawn children using the CLI:

# Python example
subprocess.run(["sysg", "spawn", "--name", "worker_1", "--ttl", "3600", "--", "python", "worker.py"])

Additional Commands

The sysg command-line interface provides several subcommands for managing processes:

Stop - Stop the process manager or a specific service:

# Stop the supervisor and every managed service
$ sysg stop

# Stop a specific service
$ sysg stop --service myapp

Restart - Restart the process manager:

# Restart all services managed by the supervisor
$ sysg restart

# Restart a specific service
$ sysg restart -s myapp

# Restart with a different configuration
$ sysg restart --config new-config.yaml

Status - Check the status of running services:

# Show status of all services (uses default systemg.yaml)
$ sysg status

# Show status with a specific configuration file
$ sysg status --config myapp.yaml

# Show status of a specific service
$ sysg status --service webserver

# Show all services including orphaned state
$ sysg status --all

# Refresh status every 5 seconds (also accepts 1s, 2m, 1second)
$ sysg status --stream 5

Inspect - Inspect a service or cron unit in detail:

# Inspect a specific service or cron unit by name or hash
$ sysg inspect myservice

# Show metrics in JSON format
$ sysg inspect myservice --json

# Refresh continuously using a rolling 2-minute metrics window
$ sysg inspect myservice --stream 2m

# Render output without ANSI coloring
$ sysg inspect myservice --no-color

Logs - View logs for a specific service:

# View the last 50 lines of stderr logs (default)
$ sysg logs

# View logs for a specific service
$ sysg logs --service api-service

# View a custom number of log lines
$ sysg logs --service database --lines 100

# Refresh log snapshots every 2 seconds (respects --lines)
$ sysg logs --service api-service --lines 100 --stream 2

# View specific log type (stdout, stderr, or supervisor)
$ sysg logs --service myservice --kind stderr

Spawn - Dynamically spawn child processes from parent services:

# Spawn a worker process (parent must have spawn.mode: dynamic)
$ sysg spawn --name worker_1 -- python worker.py
$ 12345  # Returns the child PID

# Spawn with time-to-live for automatic cleanup
$ sysg spawn --name temp_worker --ttl 3600 -- ./process.sh

# Spawn with custom log level for debugging
$ sysg spawn --name debug_worker --log-level debug -- python worker.py

# Spawn with environment variables (pass them directly)
$ KEY=value PORT=8080 sysg spawn --name worker -- node app.js

# Spawn an autonomous agent (pass env vars for provider/goal)
$ LLM_PROVIDER=claude AGENT_GOAL="Optimize database queries" sysg spawn --name optimizer -- python3 agent.py

Log Level - Override logging verbosity:

# Override logging verbosity for the current run (works with every subcommand; names or 0-5)
$ sysg start --log-level debug
$ sysg start --log-level 4

Comparison

Development

Testing

To run the test suite:

# Run all tests
$ cargo test

# Run specific test
$ cargo test test_service_lifecycle

Build from Source

To build systemg from source:

# Clone the repository
$ git clone https://github.com/ra0x3/systemg.git
$ cd systemg

# Build the project
$ cargo build --release

# The binary will be available at target/release/sysg

Contributing

Contributions to systemg are welcome! Please see the CONTRIBUTING.md file for guidelines.