
<div align="center">
<h1>zc</h1>
<p>P2P compute broker with credit-based billing, automatic worker discovery, and real-time monitoring.</p>
</div>
<p align="center">
<a href="#overview">Overview</a> •
<a href="#installation">Installation</a> •
<a href="#macos-menu-bar-app-and-widget">macOS App</a> •
<a href="#quick-start">Quick Start</a> •
<a href="#zakuro-drive-hooks">Drive Hooks</a> •
<a href="#broker">Broker</a> •
<a href="#benchmarking">Benchmarking</a> •
<a href="#api-endpoints">API Endpoints</a> •
<a href="#development">Development</a>
</p>
## Overview
`zc` is the Zakuro AI command-line tool that acts as a P2P compute broker. It routes compute requests to workers across a distributed network, handles credit-based billing via the Zakuro dashboard API with write-ahead logging, automatically discovers workers on the WireGuard mesh, and reports live transactions, workers and metrics from the CLI and the broker's `/stats` endpoint.
**Key features:**
- **P2P compute routing** — forward requests to optimal workers based on price, latency, or availability
- **P2P broker mesh** — broker-to-broker communication with authoritative user assignment (ZAKURO_P2P mode)
- **QUIC transport** — high-throughput multiplexed UDP transport for peer task offers (QUIC-first with HTTP fallback)
- **6 routing strategies** — `best_price`, `best_latency`, `best_availability`, `round_robin`, `random`, `weighted_capacity`
- **Credit-based billing** — dashboard API-backed ledger with atomic reserve-commit transactions; no direct database access; standalone mode (`ZAKURO_MASTER_KEY`) runs a local ledger without dashboard
- **Write-ahead log** — JSONL WAL with DashMap in-memory index, batched fsync, and crash recovery
- **Zero-latency hot path** — in-memory DashMap credit operations for authoritative users (P2P mode)
- **Automatic discovery** — finds workers on the mesh subnet (parallel scan) or falls back to localhost
- **Built-in benchmarking** — measure throughput/latency and compare routing strategies
- **Cluster management** — Docker container lifecycle, image management, and diagnostics
- **Zakuro Drive hooks** — manage file-event hooks (webhook/exec) on a local [zakuro-drive](https://github.com/zakuro-ai/zakuro-drive) daemon over its QUIC API
## Installation
### Prerequisites
- [Rust](https://rustup.rs/) — recent stable (the dependency tree's MSRV is 1.88 via `time` 0.3.47; the old "1.75+" note predates that)
- [Task](https://taskfile.dev/) (replaces Make)
- Docker and Docker Compose (optional, for containerized deployments)
- A Zakuro API key (`ZAKURO_API_KEY`) — sign up at [zakuro-ai.com](https://zakuro-ai.com)
### Prebuilt binary
```bash
curl -fsSL https://get.zakuro-ai.com/zc | bash # install
zc update # update in place
```
`get.zakuro-ai.com` is the canonical binary source: a verified mirror of the
[GitHub releases](https://github.com/zakuro-ai/zc/releases). Both the installer
and `zc update` read it, so they give the same version, and neither needs a
GitHub token. `zc update` checks the download's sha256 against the manifest
before replacing the binary and refuses on a mismatch. If the mirror is down and
a GitHub token is available (`gh auth login`, `GH_PAT` or `GITHUB_TOKEN`), it
falls back to the GitHub releases. See [ZC-0015](docs/adr/0015-distribution-paths.md).
### From Source
```bash
git clone https://github.com/zakuro-ai/zc.git
cd zc
task build:bin
```
This compiles the release binary and installs it to `/usr/local/bin/zc`.
#### Install with Cargo (no clone)
```bash
cargo install --git https://github.com/zakuro-ai/zc zc2
```
`zc2` needs **no access to any private repository** — it simply does not
include `zc hooks` (Zakuro Drive file-event hooks). That command lives in a
separate `zc-hooks` helper binary which `zc hooks` execs, because it depends on
the private `zakuro-client` crate. Building it requires read access to
`zakuro-ai/zakuro-drive`:
```bash
cargo build --release -p zc-hooks
```
Put the resulting `zc-hooks` next to your `zc` binary (or anywhere on `PATH`)
and `zc hooks …` works as before.
### From crates.io
```bash
cargo install zc2
```
Add `--locked` to get the exact dependency tree CI tests; without it cargo
resolves the newest compatible versions.
Note the registry build has no `zc hooks` — that command needs the `zc-hooks`
sidecar, which depends on a private crate and so cannot be published. Use a
[binary release](https://github.com/zakuro-ai/zc/releases) if you need it.
### With Docker
```bash
task build
```
Builds the binary, then builds and starts the Docker services via `docker compose`.
### macOS menu bar app and widget
On a Mac, `Zakuro.app` puts the same box in your menu bar and on your desktop: sharing state,
credit balance, earnings, devices online, and — from the medium widget — Pause/Resume and price
controls that write straight to `zc agent`.
```bash
export ZAKURO_DEVELOPMENT_TEAM=<your Team ID>
task macos:install # build signed, install to /Applications, relaunch
zc login && zc agent install
# open the popover once, then: right-click the desktop -> Edit Widgets... -> "Zakuro"
```
Requires macOS 14+, Xcode 26 and `xcodegen`. A zc release ships the CLI binary only, so the app
is built from source. Opening the popover once is required before the widget shows anything —
it is what hands the sandboxed widget the agent's address and token.
Full instructions, including the Team ID lookup and troubleshooting: **[`macos/README.md`](macos/README.md)**.
### Verify
```bash
zc --version
```
## Quick Start
```bash
# 1. Set your API key
export ZAKURO_API_KEY="zk_myuser_abc123"
# 2. Set dashboard API URL
export ZAKURO_API_URL="https://hub.zakuro-ai.com"
# 3. Start the broker
zc broker
# The broker discovers workers, listens on 0.0.0.0:9000,
# and routes all credit operations through the dashboard API.
```
## Zakuro Drive Hooks
`zc hooks` manages file-event hooks on a running [zakuro-drive](https://github.com/zakuro-ai/zakuro-drive) daemon (`zakurod`) over its QUIC API. A hook fires on file events (`file.created` | `file.modified` | `file.deleted`) matching a glob, and either POSTs a signed JSON payload to a webhook or runs a local command.
```bash
# Webhook hook — secret read from stdin (recommended; avoids ps/history leaks)
echo "$WEBHOOK_SECRET" | zc hooks add --name slack-notify \
--on file.created --on file.modified --glob "reports/**/*.pdf" \
--webhook https://hooks.example.com/zakuro --secret-stdin
# Local command hook with a custom timeout
zc hooks add --name csv-to-parquet --on file.created --glob "**/*.csv" \
--exec "/usr/local/bin/convert-csv" --timeout 120
zc hooks list --all # include disabled hooks
zc hooks get csv-to-parquet # full config (secrets redacted by the daemon)
zc hooks update csv-to-parquet --disable
zc hooks logs --failed --from 2h
zc hooks remove csv-to-parquet
```
The daemon address defaults to `127.0.0.1:4719`; override with `--daemon HOST:PORT` or `ZAKURO_DRIVE_DAEMON`. Run `zc hooks --help` for the full flag reference (retry tuning, `--secret-from-env`, `--tls-insecure`, dead-letter views).
> **Packaging note:** `zakuro-client` (the drive daemon API client) is consumed as a git dependency on the private `zakuro-ai/zakuro-drive` repository, pinned to the `client-v0.1.0` tag. crates.io rejects git dependencies, so the crates.io publish job in `auto-release.yml` is paused until `zakuro-proto` + `zakuro-client` are published to a registry; GitHub releases with prebuilt binaries continue.
## Broker
The broker accepts compute requests, selects an optimal worker, forwards the request, and handles billing.
### Modes
| Mode | Command | Description |
|------|---------|-------------|
| Foreground | `zc broker` | Live transaction log with colored output |
| Daemon | `zc -d broker` | Background process, minimal output |
```bash
zc broker # foreground on :9000
zc -d broker # daemon on :9000
zc broker 8080 # custom port
zc broker 0.0.0.0 9000 # custom host and port
```
### Worker Discovery
| Mode | Trigger | Behavior |
|------|---------|----------|
| **Mesh** | a `zakuro0` or `wg*` interface with a `10.13.13.x` address, or `ZAKURO_WIREGUARD_IP` env | Scans peers + subnet; credits enforced for remote workers |
| **Peers** | `ZAKURO_PEERS` env set (no mesh interface) | Probes explicit peer addresses; credits enforced |
| **Local** | No mesh, no peers | Scans `localhost:8089-8091`; execution is free |
Workers must expose `/health` and `/info` endpoints to be discovered. Workers can also [register manually](#post-workers) via the API.
In a **P2P mesh**, each broker detects its own mesh IP and treats workers at `localhost`/`127.0.0.1` or at that IP as **local** (free execution). Remote workers (mesh peers) are charged via the credit ledger. See [docs/USAGE.md](docs/USAGE.md#p2p-mesh-deployment) for mesh setup instructions.
Subnet scanning is **parallelized**: phase 1 does a concurrent TCP SYN to all 253 /24 hosts (50ms timeout); phase 2 probes only reachable hosts. This reduces scan time from `253 × 50ms` to `~50ms + probe time`.
### P2P Broker-to-Broker Communication
Enable distributed broker mesh with `ZAKURO_P2P=true`. Each broker becomes authoritative for a subset of users:
| Mode | Description |
|------|-------------|
| **Authority::Local** | User owned by this broker (via FNV-1a hash) → in-memory DashMap, zero PG on hot path |
| **Authority::Peer(url)** | User owned by remote broker → HTTP `/peer/reserve` + `/peer/commit` (~1ms RTT) |
| **Authority::Standalone** | Peer unreachable → dashboard API fallback path |
**Peer endpoints** (`X-Peer-Key` auth):
- `POST /peer/reserve` — reserve credits for remote user
- `POST /peer/commit` — commit transaction after execution
- `POST /peer/cancel` — cancel reservation (refund)
- `GET /peer/balance` — get balance for remote user
- `GET /peer/health` — peer health check
**Environment variables:**
- `ZAKURO_P2P=true` — enable P2P mode (default: false)
- `ZAKURO_PEER_KEY` — shared secret for peer authentication
- `ZAKURO_OWNER_ID` — this broker's ID for authority assignment
- `ZAKURO_PEERS` — comma-separated peer broker URLs for discovery
### Routing Strategies
| Strategy | Description |
|----------|-------------|
| `best_price` | Cheapest worker **(default)** |
| `best_latency` | Fastest response time |
| `best_availability` | Most available resources |
| `round_robin` | Even distribution |
| `random` | Random selection |
| `weighted_capacity` | Weighted by available CPU/memory |
Set via the `strategy` field in `X-Zakuro-Requirements` header on `/execute` requests.
### Credit System
- **Dashboard API ledger** — all balance and transaction operations go through `ZAKURO_API_URL`; no direct database access from the broker. The dashboard returns balance in the `credits_balance` field.
- **Standalone mode** — set `ZAKURO_MASTER_KEY` (no `ZAKURO_API_URL` needed) to run a fully local credit ledger. Transactions are queued in the WAL and the flush buffer; billing is enforced even without the dashboard.
- **P2P mode** (`ZAKURO_P2P=true`) — authoritative broker owns users via FNV-1a hash, DashMap in-memory operations on the hot path. Setting `remote_only: true` in a request forces routing to peer workers (bypasses local worker selection).
- **Write-ahead log** — append-only JSONL with DashMap index (O(1) lookup), batched fsync (100ms/64 entries), crash recovery, auto-compaction every ~5 min
- **Flush thread** — batched writes every 5s (transactions + balance snapshots), synced to the dashboard API
- **Local mode** — all execution is free when no mesh network is detected
```bash
# Check balance (requires Bearer token — own user or admin)
curl http://localhost:9000/credits/my-user \
-H "Authorization: Bearer $API_KEY"
# Add credits (requires master API key)
curl -X POST http://localhost:9000/credits/my-user/add \
-H "Content-Type: application/json" \
-H "X-Api-Key: $ZAKURO_MASTER_KEY" \
-d '{"amount": 100.0, "description": "Initial deposit"}'
```
## Model Inference (`zc infer`)
```bash
# Chat model: one-shot prompt
zc infer zc://<owner>/<name> -m "Say hello" [--max-tokens N] [--system "<sys>"] [--broker <url>]
# Speech model: transcribe an audio file (served through the hub)
zc infer zc://<owner>/<name> --file clip.wav [--language sn]
```
`-m` and `--file` are mutually exclusive. `--file` accepts wav, mp3, flac, m4a,
ogg and webm; `--language` is an optional 2-3 letter code (`^[a-z]{2,3}$`). The
transcript is printed first, then a receipt (language, duration, billed
seconds, cost in zkcr). Audio longer than 10 minutes is transcribed up to that
limit and a warning says so. Hub errors (insufficient credits, model not
committed, a chat model given `--file`) are printed verbatim with ` (hub)`.
It uses the API URL and key saved by `zc login` (or `ZAKURO_API_URL` /
`ZAKURO_API_KEY`).
## Benchmarking
```bash
zc bench # default (10 concurrent, 1000 requests)
zc bench -c 100 -n 10000 # high-load
zc bench -s best_latency # specific strategy
zc bench --compare # compare all strategies
zc bench http://broker:9000 # remote broker
zc bench mesh --workers a:3960,b:3960,c:3960 | jq .
```
| Option | Short | Default | Description |
|--------|-------|---------|-------------|
| `--concurrency` | `-c` | `10` | Concurrent workers |
| `--requests` | `-n` | `1000` | Total requests |
| `--url` | `-u` | `http://127.0.0.1:9000` | Broker URL |
| `--strategy` | `-s` | `best_price` | Routing strategy |
| `--compare` | | | Compare all strategies |
| `--list-strategies` | | | List strategies |
Output includes throughput (RPS), latency percentiles (p50-p99), success/failure rates, and an ASCII histogram.
`zc bench mesh` emits a JSON mesh-warmup report with per-worker `latency_p95`,
`bandwidth_bps`, and `recommended_backpressure`, suitable for `jq` or for
feeding into operator tooling.
## Remote Monitoring
Read a running broker from another machine:
```bash
zc workers localhost:9000 # its workers, their health and prices
zc workers http://broker.mesh:9000
# transactions, request rate and latency percentiles, as JSON
curl -s -H "Authorization: Bearer $ZAKURO_API_KEY" http://broker.mesh:9000/stats
```
## System Diagnostics
```bash
zc info
```
Shows network info (hostname, mode, IPs), compute cluster availability (Ray, Dask, Spark), environment variables, and broker port status.
## API Endpoints
The broker listens on `0.0.0.0:9000` by default.
| Method | Endpoint | Auth | Description |
|--------|----------|------|-------------|
| GET | `/health` | None | Health check |
| GET | `/stats` | Bearer | Broker statistics (non-admin: own txns only) |
| GET | `/workers` | None | List registered workers |
| POST | `/workers` | `X-Worker-Key` | Register a worker |
| POST | `/workers/heartbeat` | `X-Worker-Key` | Worker heartbeat |
| DELETE | `/workers/:id` | `X-Worker-Key` | Unregister worker |
| POST | `/execute` | Bearer | Execute compute request |
| POST | `/price` | None | Estimate request price |
| POST | `/route` | None | Placement dry run: same filtering + strategy ranking as `/execute`, executes/reserves/bills/mutates nothing |
| GET | `/credits/:user` | Bearer | Get balance (own user or admin) |
| POST | `/credits/:user/add` | `X-Api-Key` (master) | Add credits |
| GET | `/me` | Bearer | Current user info |
| GET | `/ledger/status` | Bearer (admin) | Ledger status (API URL, connection state) |
| **P2P Endpoints** (ZAKURO_P2P=true) | | | |
| POST | `/peer/reserve` | `X-Peer-Key` | Reserve credits for remote user |
| POST | `/peer/commit` | `X-Peer-Key` | Commit transaction after execution |
| POST | `/peer/cancel` | `X-Peer-Key` | Cancel reservation (refund) |
| POST | `/peer/earn` | `X-Peer-Key` | Credit earnings to broker owner |
| GET | `/peer/balance` | `X-Peer-Key` | Get balance for remote user |
| GET | `/peer/workers` | `X-Peer-Key` | List this broker's local workers |
| GET | `/peer/health` | `X-Peer-Key` | Peer broker health check |
### Execute Request Options
The `/execute` endpoint accepts a JSON body (or `X-Zakuro-Requirements` header) with:
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `cpus` | float | `1.0` | Required vCPUs |
| `memory_bytes` | int | — | Required memory in bytes |
| `gpus` | int | `0` | Required GPUs |
| `estimated_duration_secs` | float | `1.0` | Estimated job duration (used for cost estimation) |
| `budget_credits` | float | — | Maximum credits to spend; request fails if cost would exceed this |
| `strategy` | string | `best_price` | Routing strategy |
| `remote_only` | bool | `false` | When `true`, skip local workers and dispatch only to peer brokers (requires `ZAKURO_P2P=true`) |
| `worker_type` | string | — | Filter workers by type (e.g. `standard`, `premium`) |
| `min_vram_gb` | int | — | Minimum advertised GPU VRAM (GiB) a candidate must have; a worker that advertises none is excluded once this is set |
| `tags` | string[] | — | Capability tags every candidate must carry (e.g. `["apple-silicon"]`). No healthy worker with all of them -> `503 NO_WORKERS_WITH_TAGS` naming the missing tag; never a fallback to an untagged node |
**Capability tags.** A broker stamps the workers it owns with `apple-silicon`
when it runs on macOS/aarch64 (detected at runtime), plus any operator tags in
`ZAKURO_NODE_TAGS=a,b`; a worker's own `/info` tags are kept. Tags travel on
`/workers` and `/peer/workers` (`tags`, omitted when empty, ignored by older
peers), so a tagged worker on another node is placeable. Example (MLX export
step, sakura-internal#66):
`-H 'X-Zakuro-Requirements: {"tags":["apple-silicon"],"cpus":2}'`.
Response headers include `X-Zakuro-Worker` (worker ID), `X-Zakuro-Cost` (credits spent), and `X-Zakuro-Credits-Remaining` (balance after execution).
### Placement Dry Run (`POST /route`)
Preview what `/execute` would do right now, for the same `ResourceRequirements`
JSON body `/execute` accepts (`cpus`, `memory_bytes`, `gpus`, `min_vram_gb`,
`strategy`, `worker_type`, `tags`, ...) -- without executing, reserving,
billing, or mutating any broker state (no quota reservation, no round-robin
cursor advance, no in-flight counters touched). Runs the exact router code
path `/execute` uses: same eligibility filtering, same strategy ranking.
```bash
curl -s -X POST http://localhost:9000/route \
-H 'Content-Type: application/json' \
-d '{"strategy": "best_price", "gpus": 1, "min_vram_gb": 16}' | jq
```
Response (200):
```json
{
"strategy": "best_price",
"chosen": {
"worker_id": "...", "node_fingerprint": "...", "address": "zc://worker-...",
"gpu_model": "RTX 4090", "gpu_vram_gb": 24, "price_per_hour": 1.8,
"estimated_cost": 0.42, "active_requests": 0,
"atelier_runner": "http://10.13.13.9:8700"
},
"candidates": [ /* same shape, ranked best first; chosen == candidates[0] */ ],
"excluded": [ { "worker_id": "...", "reason": "gpu_vram_gb 11 < min_vram_gb 16" } ]
}
```
A worker advertises an atelier runner endpoint via the tag
`atelier-runner=<http url>`; when present it is surfaced as `atelier_runner`
on `chosen` and on each candidate. When nothing is eligible, the response is
404 with `{"error": ..., "code": "NO_CAPACITY", "excluded": [...]}` listing
why every healthy worker was ruled out.
`chosen` is the policy's first eligible choice, not a dispatch guarantee:
`/execute` retries past a first choice that turns out to be at quota and
tries the rest before giving up. `/route` has no request to reserve against,
so it cannot observe or reproduce that -- `chosen` can name a worker
`/execute` ends up skipping at the moment it actually runs.
### Worker Registration
```json
{
"name": "worker-1",
"uri": "http://10.13.13.5:8089",
"worker_type": "zakuro",
"resources": {
"cpus_total": 8.0,
"cpus_available": 8.0,
"memory_total": 17179869184,
"memory_available": 17179869184,
"gpus_total": 1,
"gpus_available": 1
},
"pricing": {
"cpu_price": 0.001,
"memory_price": 0.0001,
"gpu_price": 0.01,
"min_charge": 0.001
}
}
```
## Cluster Commands
| Command | Description |
|---------|-------------|
| `zc up` | Start 1 worker + broker locally (foreground) |
| `zc up --workers N` | Start N workers + broker |
| `zc down` | Stop all workers + broker started by `zc up` |
| `zc workers` | List workers registered with the local broker |
| `zc update` | Update CLI from get.zakuro-ai.com |
| `zc pull` | Pull updated Docker images |
| `zc images` | List Zakuro images |
| `zc ps` | List running containers |
| `zc kill` | Stop running containers |
| `zc restart` | Restart containers |
## Environment Variables
| Variable | Required | Default | Description |
|----------|----------|---------|-------------|
| `ZAKURO_API_KEY` | No* | — | Authentication token for dashboard API. *Optional when using only local or discovered brokers (P2P only). |
| `ZAKURO_MASTER_KEY` | No | — | Master API key for admin operations and standalone billing. When set (without `ZAKURO_API_URL`), the broker runs a fully local credit ledger with billing enforced. |
| `ZAKURO_API_URL` | No* | — | Dashboard API URL (e.g., `https://hub.zakuro-ai.com`). *Required for dashboard-backed billing and worker sync. Omit to use standalone mode (`ZAKURO_MASTER_KEY`). |
| `ZAKURO_WORKER_KEY` | No | — | Shared secret for worker management endpoints (`POST/DELETE /workers`, heartbeat) |
| `ZAKURO_WIREGUARD_IP` | No | — | Override mesh IP detection (Docker sidecar setups). Named for the mesh we used to run; it applies to WireGuard too. |
| `ZAKURO_PEERS` | No | — | Comma-separated `ip:port` peers for explicit discovery (workers) or `http://host:port` (brokers in P2P mode) |
| `ZAKURO_DISCOVER_BROKER_PEERS` | No | `true` | When P2P is enabled and `ZAKURO_PEERS` is empty, scan localhost ports 9000–9010 for other brokers. Set to `false` to disable. When using discovered brokers, `ZAKURO_API_KEY` is optional. |
| `ZAKURO_P2P` | No | `false` | Enable P2P broker-to-broker communication |
| `ZAKURO_PEER_KEY` | No | — | Shared secret for P2P broker authentication (`X-Peer-Key` header) |
| `ZAKURO_OWNER_ID` | — | — | **Not configurable.** Always derived from `ZAKURO_API_KEY` (`zk_{user_id}_{hex}` format). Manual overrides are ignored. |
| `ZAKURO_NODE_NAME` | No | Auto-detected | Human-readable node name (falls back to the mesh hostname, then system hostname) |
| `ZAKURO_SHARE_RUNTIME` | No | `auto` | Where `zc share` / `zc up` run the worker: `auto` (the `zakuro-image-compute` container when Docker is reachable, else the host `uv run` process with a one-line warning), `host` (always the host process) or `container` (require Docker; fails clearly without it). The container publishes its port on `127.0.0.1` only. `ZC_SHARE_WORKER_IMAGE` overrides the image (use `name@sha256:...` to pin). |
| `ZAKURO_WORKER_PORT` | No | `3960` | Primary port workers listen on (discovery) |
| `ZAKURO_SCAN_RANGE` | No | — | Port range for discovery scan, e.g. `3960-3999` |
| `ZAKURO_SCAN_INTERVAL` | No | `15` | Seconds between discovery scans |
| `ZAKURO_AUTO_UPDATE` | No | off | Opt-in broker self-update from crates.io (crate `zc2`). `1`/`true`/`yes`/`on` enables it; anything else is off. Never set it on k8s fleets (their image is managed by CD). When on, the broker reads the crates.io sparse index; for a strictly newer non-yanked release it runs `cargo install zc2 --version X --locked` into a scratch dir next to the executable while it keeps serving, checks the built binary runs and reports exactly X, waits until idle (no in-flight requests, local workers or pending settlements, on 3 polls 5 s apart), swaps it in atomically and exits with code 75 so systemd (`Restart=always`) starts it. Needs `cargo` (see `ZAKURO_CARGO`) and a writable executable directory; failures are logged and retried next interval. Status (`source`, `building`, `last_error`, ...) is in `GET /health` under `auto_update`. `zc update` still uses the release mirror. |
| `ZAKURO_AUTO_UPDATE_INTERVAL` | No | `600` | Time between index checks, in seconds or with an `s`/`m`/`h` suffix (`10m`, `1h`). Minimum 300 s. +up to 10% jitter; the first check is after a random 60 s..interval delay. |
| `ZAKURO_AUTO_UPDATE_INDEX` | No | `https://index.crates.io/3/z/zc2` | Sparse-index URL for `zc2` (https; http only for `127.0.0.1`/`localhost`). For tests. |
| `ZAKURO_CARGO` | No | `cargo` on PATH, else `$HOME/.cargo/bin/cargo` | The cargo used by auto-update builds. |
| `ZAKURO_UPDATE_URL` | No | `https://get.zakuro-ai.com/zc` | Mirror base for `zc update` (the manual path; broker auto-update uses crates.io). Either a bare host base (`https://stg.get.zakuro-ai.com`) or one ending in `/zc`; trailing `/` is trimmed and `/zc` appended when missing. Must be `https://` (plain `http://` only for `127.0.0.1`/`localhost`); an invalid value is an error, never silently ignored. |
## Development
### Task Commands
This project uses [Task](https://taskfile.dev/) as its task runner. Install it with `go install github.com/go-task/task/v3/cmd/task@latest` or see [installation docs](https://taskfile.dev/installation/).
| Command | Description |
|---------|-------------|
| `task build` | Build binary and start Docker services |
| `task build:bin` | Incremental release build of `zc` and `zc-hooks`, installed to `~/.local/bin` |
| `task dev` | Incremental debug build for fixing bugs, linked as `~/.local/bin/zc-dev` |
| `task build:docker` | Build Docker image only |
| `task run` | Build and run the CLI |
| `task debug` | Build and run with `RUST_BACKTRACE=full` |
| `task release` | Run the installed release binary |
| `task broker` | Start broker in foreground |
| `task broker:daemon` | Start broker in daemon mode |
| `task bench` | Run default benchmark |
| `task bench:compare` | Compare all routing strategies |
| `task info` | Show system diagnostics |
| `task docker:up` | Start Docker services |
| `task docker:down` | Stop Docker services |
| `task docker:logs` | Tail Docker service logs |
| `task clean` | Remove build artifacts |
### Docker
```bash
# Build and run with Docker Compose
task build
# Or build the image directly
docker build -t zakuro-ai/broker -f docker/Dockerfile .
docker run -p 9000:9000 -e ZAKURO_API_KEY=demo zakuro-ai/broker
```
### Project Structure
```
src/
main.rs # CLI entry point and command routing
broker/
mod.rs # Broker state, config, and per-worker local detection
server.rs # HTTP server (tiny_http) with execute handler
worker.rs # Worker registry, resources, and health tracking
router.rs # Routing strategies and worker selection
credits.rs # Lightweight per-user balance cache and rate-limit tracker (reservation/history owned by Ledger)
ledger.rs # Central ledger: API mode (dashboard) or local in-memory reserve-commit
peer.rs # P2P broker-to-broker communication (PeerClient, PeerManager, FNV-1a authority)
flush.rs # TransactionBuffer for batched writes (5s cycle, via dashboard API)
discovery.rs # mesh/local/peer-based worker discovery
wal.rs # Write-ahead log (JSONL + DashMap index, batched fsync)
recovery.rs # WAL replay on startup (reserved→cancel, executed→commit)
bench.rs # Benchmarking tool
stats.rs # Statistics tracking
info.rs # System diagnostics
common/mod.rs # Shared utilities
exec/mod.rs # Command execution helpers
manager/mod.rs # Docker container management
network/mod.rs # Network interface detection
envs/mod.rs # Environment variable handling
```
## Troubleshooting
- **"You are missing ZAKURO_API_KEY"** — set `ZAKURO_API_KEY` for dashboard auth. For standalone billing, set `ZAKURO_MASTER_KEY` instead. P2P-only setups (auto-discovered peers) don't require either.
- **No workers discovered** — ensure workers expose `/health` and `/info` endpoints on port 3960 (or `ZAKURO_WORKER_PORT`). Check with `zc info`.
- **Credit operations failing** — ensure both `ZAKURO_API_KEY` and `ZAKURO_API_URL` are set for dashboard mode. For standalone mode, set only `ZAKURO_MASTER_KEY`. Verify the dashboard returns the `credits_balance` field in `/api/broker/balance/{id}`.
- **Zero cost on remote executions** — ensure `is_billing_enabled()` returns true: either dashboard URL+key or `ZAKURO_MASTER_KEY` must be set. Local workers (`127.0.0.1`/`localhost`) are always free; remote mesh workers are billed.
- **`remote_only: true` returns 503** — ensure `ZAKURO_P2P=true` and at least one peer broker is reachable. Without peers, `remote_only` requests cannot be served.
- **P2P mode not working** — ensure `ZAKURO_P2P=true`, `ZAKURO_PEER_KEY` matches on all brokers. Use `ZAKURO_PEERS` for explicit URLs, or leave empty to auto-discover brokers on localhost (ports 9000–9010).
- **Mesh IP not detected** — the broker recognises a `zakuro0` or `wg*` interface holding an address in `10.13.13.0/24` (see `get_mesh_ip` in `src/broker/discovery.rs`). An interface outside that subnet is ignored. Set `ZAKURO_WIREGUARD_IP` to override.
## Documentation
See [docs/USAGE.md](docs/USAGE.md) for the complete usage guide covering all API details, configuration reference, and the credit system in depth.
See [docs/QUIC_TRANSPORT.md](docs/QUIC_TRANSPORT.md) for details on the QUIC-based P2P transport layer.
## License
MIT