hatchet-rest
A client for the Hatchet REST management API — the 37 operations the official Go SDK's feature clients call: runs, logs, crons, scheduled runs, workers, workflows, metrics, rate limits, the tenant, filters, webhooks and CEL debugging.
Pre-release: the API may still change before 1.0.
Quick start
// HATCHET_CLIENT_TOKEN; the server URL and tenant come from the token's
// claims unless HATCHET_CLIENT_SERVER_URL / HATCHET_CLIENT_TENANT_ID say
// otherwise.
let rest = from_env?;
let tenant = rest.tenant.get.await?;
println!;
let recent = rest
.runs
.list
.await?;
for run in &recent.rows
crates/hatchet-rest/examples/rest_admin.rs is a runnable, longer version of
this (cargo run -p hatchet-rest --example rest_admin); see
crates/hatchet-client/examples/README.md for how to point it at an engine.
Operations
Exactly the operations the official Go SDK's feature clients call
(sdks/go/features/*.go at v0.107.0); [hatchet_rest::Operation::ALL] is
the same list in code, and cargo xtask check-rest checks it against the
vendored OpenAPI spec.
| # | operationId | method path | method |
|---|---|---|---|
| 1 | v1-workflow-run:get |
GET /api/v1/stable/workflow-runs/{v1-workflow-run} |
runs().get |
| 2 | v1-workflow-run:get-status |
GET /api/v1/stable/workflow-runs/{v1-workflow-run}/status |
runs().status |
| 3 | v1-workflow-run:list |
GET /api/v1/stable/tenants/{tenant}/workflow-runs |
runs().list |
| 4 | v1-task:cancel |
POST /api/v1/stable/tenants/{tenant}/tasks/cancel |
runs().cancel |
| 5 | v1-task:replay |
POST /api/v1/stable/tenants/{tenant}/tasks/replay |
runs().replay |
| 6 | v1-task:restore |
POST /api/v1/stable/tasks/{task}/restore |
runs().restore |
| 7 | v1-durable-task:branch |
POST /api/v1/stable/tenants/{tenant}/durable-tasks/branch |
runs().branch_durable_task |
| 8 | v1-log-line:list |
GET /api/v1/stable/tasks/{task}/logs |
logs().list |
| 9 | cron-workflow-trigger:create |
POST /api/v1/tenants/{tenant}/workflows/{workflow}/crons |
crons().create |
| 10 | cron-workflow:list |
GET /api/v1/tenants/{tenant}/workflows/crons |
crons().list |
| 11 | workflow-cron:get |
GET /api/v1/tenants/{tenant}/workflows/crons/{cron-workflow} |
crons().get |
| 12 | workflow-cron:delete |
DELETE /api/v1/tenants/{tenant}/workflows/crons/{cron-workflow} |
crons().delete |
| 13 | scheduled-workflow-run:create |
POST /api/v1/tenants/{tenant}/workflows/{workflow}/scheduled |
schedules().create |
| 14 | workflow-scheduled:list |
GET /api/v1/tenants/{tenant}/workflows/scheduled |
schedules().list |
| 15 | workflow-scheduled:get |
GET /api/v1/tenants/{tenant}/workflows/scheduled/{scheduled-workflow-run} |
schedules().get |
| 16 | workflow-scheduled:delete |
DELETE /api/v1/tenants/{tenant}/workflows/scheduled/{scheduled-workflow-run} |
schedules().delete |
| 17 | worker:list |
GET /api/v1/tenants/{tenant}/worker |
workers().list |
| 18 | worker:get |
GET /api/v1/workers/{worker} |
workers().get |
| 19 | worker:update |
PATCH /api/v1/workers/{worker} |
workers().pause, workers().unpause |
| 20 | workflow:list |
GET /api/v1/tenants/{tenant}/workflows |
workflows().list, workflows().find |
| 21 | workflow:delete |
DELETE /api/v1/workflows/{workflow} |
workflows().delete |
| 22 | workflow:get:metrics |
GET /api/v1/workflows/{workflow}/metrics |
metrics().workflow |
| 23 | tenant:get:queue-metrics |
GET /api/v1/tenants/{tenant}/queue-metrics |
metrics().queue |
| 24 | tenant:get:step-run-queue-metrics |
GET /api/v1/tenants/{tenant}/step-run-queue-metrics |
metrics().task_queue |
| 25 | rate-limit:list |
GET /api/v1/tenants/{tenant}/rate-limits |
rate_limits().list |
| 26 | tenant:get |
GET /api/v1/tenants/{tenant} |
tenant().get |
| 27 | v1-filter:list |
GET /api/v1/stable/tenants/{tenant}/filters |
filters().list |
| 28 | v1-filter:get |
GET /api/v1/stable/tenants/{tenant}/filters/{v1-filter} |
filters().get |
| 29 | v1-filter:create |
POST /api/v1/stable/tenants/{tenant}/filters |
filters().create |
| 30 | v1-filter:update |
PATCH /api/v1/stable/tenants/{tenant}/filters/{v1-filter} |
filters().update |
| 31 | v1-filter:delete |
DELETE /api/v1/stable/tenants/{tenant}/filters/{v1-filter} |
filters().delete |
| 32 | v1-webhook:list |
GET /api/v1/stable/tenants/{tenant}/webhooks |
webhooks().list |
| 33 | v1-webhook:get |
GET /api/v1/stable/tenants/{tenant}/webhooks/{v1-webhook} |
webhooks().get |
| 34 | v1-webhook:create |
POST /api/v1/stable/tenants/{tenant}/webhooks |
webhooks().create |
| 35 | v1-webhook:update |
PATCH /api/v1/stable/tenants/{tenant}/webhooks/{v1-webhook} |
webhooks().update |
| 36 | v1-webhook:delete |
DELETE /api/v1/stable/tenants/{tenant}/webhooks/{v1-webhook} |
webhooks().delete |
| 37 | v1-cel:debug |
POST /api/v1/stable/tenants/{tenant}/cel/debug |
cel().debug |
Configuration
RestClient::from_env() (or RestClient::builder()) resolves five settings,
each an explicit builder value, then a HATCHET_CLIENT_* environment
variable, then (for three of them) a claim of the API token — in that order:
| Setting | Environment variable | Token claim |
|---|---|---|
| token | HATCHET_CLIENT_TOKEN |
— |
| server URL | HATCHET_CLIENT_SERVER_URL |
server_url |
| tenant id | HATCHET_CLIENT_TENANT_ID |
sub |
| namespace | HATCHET_CLIENT_NAMESPACE |
— |
| TLS root CA file | HATCHET_CLIENT_TLS_ROOT_CA_FILE |
— |
An empty variable, or a tenant id that is the nil UUID, is treated as unset.
A missing token is Error::Config; a token whose claims give no server URL
or tenant id, with nothing set explicitly or in the environment, is
Error::InvalidToken. The server
URL's scheme decides TLS: http:// never uses it, https:// needs one of
the tls-* features below. HATCHET_CLIENT_TLS_STRATEGY and mTLS
(HATCHET_CLIENT_TLS_CERT_FILE/_KEY_FILE), which hatchet-client's gRPC
side reads, do not apply here — the official Go SDK's REST client does not
read them either.
Features
| Feature | Default | hatchet-rest enables |
Meaning (same as hatchet-client's) |
|---|---|---|---|
tls-ring |
yes | reqwest/rustls-tls-manual-roots (= …-no-provider + ring) |
ring crypto provider |
tls-aws-lc |
no | reqwest/rustls-tls-manual-roots-no-provider, dep:rustls, rustls/aws_lc_rs |
aws-lc-rs crypto provider |
tls-native-roots |
yes | reqwest/rustls-tls-native-roots-no-provider |
the platform's trust store |
tls-webpki-roots |
no | reqwest/rustls-tls-webpki-roots-no-provider |
bundled webpki roots |
With both root-store features on, reqwest trusts the union of the two.
With no crypto-provider feature, only http:// server URLs work — building
a client for an https:// URL without one is Error::Config. With only
tls-aws-lc on and no crypto provider yet installed process-wide,
RestClient::builder().build() installs aws-lc-rs as the default provider
the first time it is called.
Tracing
SchedulesApi::create opens a hatchet.schedule_workflow tracing span
(instrumentor, hatchet.workflow_name, hatchet.trigger_at), the way the
official Go SDK's schedules client does — no feature needed, and nothing is
added to the scheduled run's own metadata. This crate links no OpenTelemetry
crate on its own; see hatchet-client's README for the otel feature that
carries a trace across the engine, which lives there, not here.
Errors and retries
| Answer | Error |
|---|---|
401 or 403 |
[Error::Forbidden] |
404 with a {"message": ..} body |
[Error::NotFound] |
404 with any other body, or 501 |
[Error::NotSupportedByEngine] |
anything else with an APIErrors body |
[Error::Api] |
| anything else outside 2xx | [Error::Http] |
| no answer at all (connection failed, or ran out of retries) | [Error::Transport] |
| a request body would not encode, or a 2xx body would not decode | [Error::Json] |
None of these ever contain the API token or a webhook secret.
Only a bodiless GET is retried, on a transport error or a 429, 502,
503 or 504 — five attempts in all, full-jitter backoff from 250 ms up to
5 s (a 429's Retry-After wins if it is 5 s or less), each attempt capped
at 30 s. This is the official Go SDK's REST retry policy
(pkg/client/retry/{rest.go,backoff.go}); a POST, PATCH or DELETE is
sent once, so a cron or a cancel is never sent twice. A 500 is never
retried. This is verified against a mock server (cargo test -p hatchet-rest); a real engine was not observed to answer 429 in the
integration tests that ran against it, so the retry-then-succeed path is
verified against the mock only.
Engine versions
v0.80.0 and v0.107.0 were both observed to serve all 37 operations — there
is no per-operation version gate. What differs is smaller: since is
optional on v0.80.0's runs().list but required on v0.107.0's (this crate
always sends it, defaulting to the Unix epoch); v0.80.0 ignores the
idempotency_keys filter; a finished single-task run's output is {} on
v0.80.0 (the task's own output is still present) while v0.107.0 fills it;
pagination is absent (None) on workers().list (v0.80.0 only) and
webhooks().list (both versions); runtime_info.language was None on
both. Query parameters v0.107.0 added that v0.80.0's spec does not have
(metadata_operator and idempotency_keys on runs().list, every query
of workers().list) are
only sent when a caller gives them a value, so a call that does not use them
is the same bytes on both versions.
Relationship to hatchet-client
hatchet-client's rest feature builds a cached RestClient from the same
configuration as its gRPC Client (Client::rest(), returning a Result
since a gRPC-complete configuration can still be unusable for REST — a
non-http(s) server URL, https:// without a TLS feature, an unreadable CA
file). Cancel, replay, inspecting a run's details and status, restoring an
evicted task, and putting a rate limit's value are all available over gRPC
too (Client::runs(), Client::rate_limits()); everything else this crate
sends — logs, crons, scheduled runs, worker administration, workflow
listing, metrics, filters, webhooks, CEL debugging — has no gRPC
equivalent.
License
Licensed under either of Apache License, Version 2.0 or MIT license at your option.
Part of hatchet-rs.