gflow - A lightweight, single-node job scheduler

English | 简体中文

gflow is a lightweight job scheduler for a single Linux machine. It gives you a Slurm-like workflow—submit, queue, inspect, cancel, and organize jobs—without deploying a cluster. It is especially useful on shared GPU workstations, lab servers, and small research boxes.

Demo

When gflow fits well

• You have one Linux machine instead of a full cluster.
• Multiple users or experiments need to share GPUs safely.
• You want job queues, dependencies, arrays, and time limits.
• You want a lighter alternative to Slurm for local or lab infrastructure.

Core Features

• Daemon-based scheduling: gflowd keeps the queue, state, and resource allocation in one place.
• GPU-aware execution: schedule dedicated GPUs or shared GPUs with per-job VRAM limits.
• Rich submission model: submit commands or scripts with priorities, dependencies, arrays, and conda environments.
• Time limits and lifecycle control: prevent runaway jobs and manage hold, release, redo, and cancel actions.
• tmux-backed execution and logs: every job runs in its own session and streams output into persistent logs.
• Automation hooks: send webhook notifications when jobs start, finish, fail, or change state.

CLI Overview

• gflowd: initialize config and manage the scheduler daemon.
• gbatch: submit commands or scripts.
• gqueue: inspect and filter jobs.
• gjob: show details, attach, hold/release, redo, and update jobs.
• gcancel: cancel one or more jobs.
• gctl: manage GPU visibility, concurrency limits, and reservations.
• ginfo: inspect scheduler and GPU status.
• gstats: view scheduler statistics.

Installation

Prerequisites

• Linux
• tmux
• NVIDIA drivers only if you want GPU scheduling features

Install via PyPI (Recommended)

Use uv:

uv tool install runqd

Or pipx:

pipx install runqd

Or pip:

pip install runqd

Prebuilt binaries are available for Linux (x86_64, arm64).

Install Nightly Build

pip install --index-url https://test.pypi.org/simple/ runqd

Install via Cargo

cargo install gflow

Install directly from main:

cargo install --git https://github.com/AndPuQing/gflow.git --locked

Build from Source

git clone https://github.com/AndPuQing/gflow.git
cd gflow
cargo build --release

Compiled binaries are placed in target/release/.

Quick Start

1. Initialize config (optional but recommended):

   gflowd init
   

2. Start the scheduler daemon:

   gflowd up
   

3. Submit a job:

   cat > my_job.sh <<'EOF'
   #!/bin/bash
   echo "Starting job on GPU: $CUDA_VISIBLE_DEVICES"
   sleep 30
   echo "Job finished."
   EOF
   chmod +x my_job.sh
   gbatch --gpus 1 ./my_job.sh
   

4. Inspect the queue:

   gqueue
   

5. Check details or stop the daemon when done:

   ginfo
   gflowd down
   

Common Workflow

gflowd up
ginfo
gbatch --gpus 1 --time 2:00:00 --name train python train.py
gqueue -f JOBID,NAME,ST,TIME,NODES,NODELIST(REASON)
gjob show <job_id>
gcancel <job_id>

Documentation

• Website: runqd.com
• Installation: runqd.com/getting-started/installation.html
• Quick start: runqd.com/getting-started/quick-start.html
• Job submission: runqd.com/user-guide/job-submission.html
• Configuration: runqd.com/user-guide/configuration.html
• Quick reference: runqd.com/reference/quick-reference.html

Star History

Contributing

If you find a bug or want to propose an improvement, please open an Issue or submit a Pull Request.

License

gflow is licensed under the MIT License. See LICENSE for details.