{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "rust_job_queue_api_worker_system_colab.ipynb",
"provenance": []
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
}
},
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# rust-job-queue-api-worker-system Colab smoke/repro notebook\n",
"\n",
"This notebook bootstraps the crate from a clean Colab runtime, runs non-Docker checks, starts local Postgres, runs migrations, launches the API and worker, enqueues a job, polls it to a terminal state, and fetches metrics/OpenAPI.\n",
"\n",
"Academic honesty note: the full integration test suite uses Docker/testcontainers. Colab runtimes do not reliably provide Docker. Docker-backed full tests and benchmarks are optional guarded cells here; GitHub Actions and a local Docker host are the canonical full-suite environments."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%%bash\n",
"set -euxo pipefail\n",
"date -u\n",
"uname -a\n",
"lscpu | sed -n '1,25p' || true\n",
"free -h || true\n",
"command -v docker >/dev/null && docker version || echo 'Docker unavailable: full testcontainers tests and benches will be skipped unless Docker is installed.'"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Install system dependencies\n",
"\n",
"This installs Postgres for the smoke demo. It does not install Docker."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%%bash\n",
"set -euxo pipefail\n",
"sudo apt-get update\n",
"sudo apt-get install -y --no-install-recommends curl ca-certificates jq git postgresql postgresql-contrib pkg-config libssl-dev\n",
"sudo service postgresql start\n",
"sudo -u postgres psql -c \"ALTER USER postgres PASSWORD 'postgres';\"\n",
"sudo -u postgres psql -tc \"SELECT 1 FROM pg_database WHERE datname = 'jobs_colab'\" | grep -q 1 || sudo -u postgres createdb jobs_colab\n",
"psql --version"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Install Rust and clone the repo"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%%bash\n",
"set -euxo pipefail\n",
"if ! command -v cargo >/dev/null; then\n",
" curl https://sh.rustup.rs -sSf | sh -s -- -y --profile minimal\n",
"fi\n",
". \"$HOME/.cargo/env\"\n",
"rustup toolchain install 1.88.0 --profile minimal --component rustfmt --component clippy\n",
"rustc --version\n",
"cargo --version\n",
"rm -rf /content/Rust-Job-Queue-API-Worker-System\n",
"git clone https://github.com/infinityabundance/Rust-Job-Queue-API-Worker-System.git /content/Rust-Job-Queue-API-Worker-System\n",
"cd /content/Rust-Job-Queue-API-Worker-System\n",
"git rev-parse --short HEAD\n",
"cat rust-toolchain.toml"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Non-Docker checks\n",
"\n",
"These checks should work in Colab. They do not run the Docker/testcontainers integration suite."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%%bash\n",
"set -euxo pipefail\n",
". \"$HOME/.cargo/env\"\n",
"cd /content/Rust-Job-Queue-API-Worker-System\n",
"cargo fmt --all -- --check\n",
"cargo check --locked --all-features\n",
"cargo check --locked --no-default-features\n",
"cargo check --locked --no-default-features --features api\n",
"cargo check --locked --no-default-features --features worker\n",
"cargo test --lib --all-features --locked\n",
"cargo test --doc --all-features --locked"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## API and worker smoke demo"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%%bash\n",
"set -euxo pipefail\n",
". \"$HOME/.cargo/env\"\n",
"cd /content/Rust-Job-Queue-API-Worker-System\n",
"export DATABASE_URL=postgres://postgres:postgres@127.0.0.1:5432/jobs_colab\n",
"export RUST_LOG=info,sqlx=warn\n",
"cargo run --locked --bin job-queue-migrate\n",
"(cargo run --locked --bin job-queue-api > /tmp/job-queue-api.log 2>&1 & echo $! > /tmp/job-queue-api.pid)\n",
"(WORKER_CONCURRENCY=2 WORKER_METRICS_BIND_ADDR=127.0.0.1:9091 cargo run --locked --bin job-queue-worker > /tmp/job-queue-worker.log 2>&1 & echo $! > /tmp/job-queue-worker.pid)\n",
"for i in $(seq 1 60); do curl -fsS http://127.0.0.1:8080/health && break || sleep 1; done\n",
"JOB_JSON=$(curl -fsS -X POST http://127.0.0.1:8080/jobs -H 'Content-Type: application/json' -H 'Idempotency-Key: colab-demo-1' -d '{\"kind\":\"send_email\",\"payload\":{\"to\":\"a@b.c\",\"subject\":\"hi\",\"body\":\"hello\"}}')\n",
"echo \"$JOB_JSON\" | jq .\n",
"JOB_ID=$(echo \"$JOB_JSON\" | jq -r .id)\n",
"for i in $(seq 1 60); do\n",
" CUR=$(curl -fsS \"http://127.0.0.1:8080/jobs/$JOB_ID\")\n",
" echo \"$CUR\" | jq '{id,status,attempts,last_error}'\n",
" STATUS=$(echo \"$CUR\" | jq -r .status)\n",
" case \"$STATUS\" in succeeded|failed_permanent|cancelled) break ;; esac\n",
" sleep 1\n",
"done\n",
"curl -fsS http://127.0.0.1:8080/metrics | sed -n '1,20p'\n",
"curl -fsS http://127.0.0.1:9091/metrics | sed -n '1,20p'\n",
"curl -fsS http://127.0.0.1:8080/api-docs/openapi.json | jq '.paths | keys'\n",
"kill \"$(cat /tmp/job-queue-worker.pid)\" \"$(cat /tmp/job-queue-api.pid)\" || true"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Embedded worker example\n",
"\n",
"This uses a fresh database so it does not race the background worker from the previous demo."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%%bash\n",
"set -euxo pipefail\n",
". \"$HOME/.cargo/env\"\n",
"sudo -u postgres dropdb --if-exists jobs_colab_embed\n",
"sudo -u postgres createdb jobs_colab_embed\n",
"cd /content/Rust-Job-Queue-API-Worker-System\n",
"DATABASE_URL=postgres://postgres:postgres@127.0.0.1:5432/jobs_colab_embed timeout 20s cargo run --locked --example embed_worker --features worker || test $? -eq 124"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Optional Docker-backed full suite\n",
"\n",
"Run this only when Docker is available. It is skipped on default Colab runtimes that do not expose Docker."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%%bash\n",
"set -euxo pipefail\n",
". \"$HOME/.cargo/env\"\n",
"cd /content/Rust-Job-Queue-API-Worker-System\n",
"if command -v docker >/dev/null && docker version >/dev/null 2>&1; then\n",
" cargo test --all-features --locked\n",
"else\n",
" echo 'Skipping cargo test --all-features --locked: Docker unavailable.'\n",
"fi"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Optional benchmarks\n",
"\n",
"Colab benchmark numbers are Colab runtime measurements. They are not reproductions of workstation results in `bench/RESULTS.md`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%%bash\n",
"set -euxo pipefail\n",
". \"$HOME/.cargo/env\"\n",
"cd /content/Rust-Job-Queue-API-Worker-System\n",
"if command -v docker >/dev/null && docker version >/dev/null 2>&1; then\n",
" ./bench/run.sh\n",
"else\n",
" echo 'Skipping benchmarks: Docker unavailable.'\n",
"fi"
]
}
]
}