# API
In general, Ocypod attempts to return meaningful HTTP status codes to indicate
whether an operation was successful or not.
As well as the status codes detailed per endpoint, Ocypod will also return
the following _5xx_ codes:
* 500 - unexpected internal error
* 503 - Redis connection unavailable
## Queue endpoints
Used for interacting with queues, i.e. creating new queues, updating queue
settings, creating jobs, and fetching jobs.
---
### `GET /queue`
Get all known queues as a JSON list.
#### Returns
* 200 - JSON list of strings
#### Example
$ curl localhost:8023/queue
["my-queue", "another_queue", "queue3"]
---
### `GET /queue/{queue_name}`
Get a queue's settings.
#### Returns
* 200 - JSON object containing queue settings
* 400 - invalid queue name passed as parameter
* 404 - no queue with given name was found
#### Example
$ curl localhost:8023/queue/example
{"timeout":"10m",
"heartbeat_timeout":"5m",
"expires_after":"5m",
"retries":5,
"retry_delays":["10s","30s","5m"]}
---
### `GET /queue/{queue_name}/size`
Get number of jobs currently queued for a given queue name.
#### Returns
* 200 - JSON integer queue size
* 400 - invalid queue name passed as parameter
* 404 - no queue with given name was found
#### Example
$ curl localhost:8023/queue/example/size
3
---
### `PUT /queue/{queue_name}`
Create a new queue, or update an existing queue with given settings.
#### Request
The request body must contain JSON of the form:
{"timeout": <duration>,
"heartbeat_timeout": <duration>,
"expires_after": <duration>,
"retries": <integer>,
"retry_delays": [<duration>[, <duration>...]]}
where `<duration>` is a human readable string of the form `"30s"`, `"5m"`, `"1w2d7h"`, etc.
Any of the fields may be omitted (e.g. `{}` is the minimum accepted input), in which case server-wide default values will be used.
Set `timeout`, `heartbeat_timeout`, or `expires_after` to "0s" to disable
timeout/expiry.
Set `retries` to `0` to disable retries.
#### Returns
* 201 - new queue created
* 204 - existing queue updated
* 400 - invalid queue name or queue settings given
#### Example
$ curl -i -H 'content-type: application/json' -XPUT -d '{"timeout": "10m"}' localhost:8023/queue/example
HTTP/1.1 201 Created
location: /queue/example
content-length: 0
date: Fri, 16 Nov 2018 17:47:55 GMT
---
### `DELETE /queue/{queue_name}`
Delete an existing queue, and any jobs still queued on it. Any running or
ended jobs won't be affected.
If any running jobs created on this queue have retries remaining, they'll
instead remain in their failed/timed out state, and be eligible for expiry
if they can't be requeued due to their original queue no longer existing.
#### Returns
* 204 - existing queue successfully deleted
* 400 - invalid queue name given
* 404 - queue with given name not found
#### Example
$ curl -i -XDELETE localhost:8023/queue/example
HTTP/1.1 204 Queue deleted
date: Tue, 20 Nov 2018 18:07:47 GMT
---
### `GET /queue/{queue_name}/job`
Get the next job to work from the given queue, if any, as a JSON job payload.
This JSON payload is of the form:
{"id": <integer>, "input": <any JSON>}
where `id` is a unique job ID that's autogenerated by Ocypod when a job is
created, and `input` is any JSON provided by the client that created the job.
When a client gets a job in this way, the job is marked as running, and is
removed from the queue.
#### Returns
* 200 - JSON payload as described above
* 204 - no jobs available in this queue
* 400 - invalid queue name given
* 404 - queue with given name not found
#### Example
$ curl -i localhost:8023/queue/example/job
HTTP/1.1 200 OK
content-length: 58
content-type: application/json
date: Tue, 20 Nov 2018 18:52:56 GMT
{"id":77,"input":{"some_key": [1, 2, 3]}}
-----
### `POST /queue/{queue_name}/job`
Create a new job on given queue with given JSON.
#### Request
The request body must contain JSON of the form:
{"input": <any JSON>,
"tags": <list of strings>,
"timeout": <duration>,
"heartbeat_timeout": <duration>,
"expires_after": <duration>,
"retries": <integer>,
"retry_delays": <list of durations>}
All fields are optional (i.e. `{}` is the minimum valid job that can be
created).
`input` represents the job's main payload, and is what clients that accept
the job will use to work on. This will typically contain input data to process,
and can be any simple or complex JSON value. Defaults to `null` if not
specified.
`tags` are an optional list of strings to attach to this job. These can be
used to look up jobs by tag using the `/tag` endpoints. Uses might include
attaching a username to jobs, or attaching a batch ID to a number of related
jobs created by the same process. Defaults to `[]` if not specified.
`timeout` is the maximum amount of time the job can run before it's marked as
timed out. Default is to use the queue's setting.
`heartbeat_timeout` is the maximum amount of time between heartbeats before the
job is marked as timed out. Default is to use the queue's setting.
`expires_after` is the amount of time the job and its metadata will remain in
Ocypod after the job completes, fails, or times out (with no retries
remaining). Default is to use the queue's setting.
`retries` is the number of times to re-queue this job on failure or time out.
Default is to use the queue's setting.
`retry_delays` is a list of durations specifying the minimum time to wait
before re-queuing a job on failure. This can be used to specify
linear/exponential/any backoff intervals when retrying jobs. Default is to use
the queue's setting.
If the number of retries exceeds the length of this list, then the last value
will always be used, e.g. with `retries: 4` and
`retry_delays: ["10s", "1m", "5m"]`, the job won't be retried for at least 10
seconds after the 1st failure, for 1 minute after the 2nd failure, for 5
minutes after the 3rd failure, and for 5 minutes on the 4th failure.
#### Returns
201 - job successfully created, response contains ID of new job, and location of job in `location` header
400 - invalid queue name or invalid job creation JSON given
404 - queue with given name not found
---
### `GET /queue/{queue_name}/job_ids`
Get a list of job IDs by status for all jobs originally created in the
given queue.
#### Returns
JSON response of the form:
{<status>: [<job ID>, ...], ...}
* 200 - JSON response as described above
* 400 - invalid queue name given
* 404 - queue with given name not found
#### Example
$ curl -i localhost:8023/queue/example/job_ids
{"completed":[1,2],"cancelled":[],"timed_out":[],"queued":[4,5],"running":[3],"failed":[]}
## Job endpoints
Endpoints for interacting with jobs in any state.
---
### `GET /job/{job_id}[?fields=<comma separated list of fields>]`
Get metadata about given job as a JSON object.
Optionally accepts a comma separated list of fields to fetch, otherwise will
get all fields by default.
Response JSON is an object with each field being a key, i.e.
{<field1>: <value>, <field2>: <value>, ... }
#### Returns
* 200 - JSON object containing requested fields
* 400 - unable to parse list of fields given, or unrecognised field given
* 404 - job with given ID does not exist
#### Example
$ curl 'localhost:8023/job/77?fields=id,status,started_at,ended_at'
{"id":77,
"status":"running",
"started_at":"2018-11-20T18:52:42.700853Z",
"ended_at":null}
---
### `PATCH /job/{job_id}`
Clients can use this to modify a job's status, and/or its `output` field.
This endpoint is generally used to mark a job as completed/failed/cancelled,
to write progress/error information to `output`, and to write final results
to `output`.
#### Request
The request must be JSON of the form:
{"status": ("completed"|"failed"|"cancelled"),
"output": <any JSON>}
All fields are optional, only fields that are present will cause any changes.
#### Response
* 204 - job successfully updated
* 400 - invalid or no JSON sent
* 404 - no job with given ID exists
* 409 - job is in state where status or output update is not allowed
#### Example
Mark a job as completed, and write its results:
$ curl -i -XPATCH -H 'content-type: application/json' -d '{"status": "completed", "output": {"result": 321, "progress": 100.0, "warnings": 3}}' localhost:8023/job/23
HTTP/1.1 204 OK
date: Wed, 21 Nov 2018 11:13:34 GMT
---
### `DELETE /job/{job_id}`
Delete a job from Ocypod, regardless of its state.
Any jobs which have ended and have a non-zero `expires_after` field will
eventually be cleaned up once they've ended, so this endpoint is only needed
if this is something you want manual control over.
While you can delete jobs that are currently running, you'll need to make sure
that clients/workers gracefully handle this case.
For more graceful job removal, you can set a job's status to `cancelled`, and
allow clients to handle this.
#### Response
* 204 - job successfully deleted
* 404 - job with given ID does not exist
#### Example
$ curl -XDELETE localhost:8023/job/123
HTTP/1.1 204 OK
date: Wed, 21 Nov 2018 11:13:34 GMT
---
### `PUT /job/{job_id}/heartbeat`
Used by clients that are working on a job to send a heartbeat for it. This is
lets the server know that the job is still actively being worked on.
This is only useful for jobs with a non-zero `heartbeat_timeout` set, and can
only be used on jobs that are running.
#### Response
* 204 - heartbeat successfully updated
* 404 - job with given ID does not exist
* 409 - job is not in a state where heartbeat can be updated (e.g. job is
queued/failed/cancelled etc.)
#### Example
$ curl -XPUT localhost:8023/job/23/heartbeat
HTTP/1.1 204 OK
date: Wed, 21 Nov 2018 11:13:34 GMT
---
### `GET /job/{job_id}/output`
Get a job's `output` JSON field.
#### Response
200 - JSON contents of the job's `output` field
404 - job with given ID does not exist
#### Example
$ curl localhost:8023/job/123/output
{"a": 1, "b": [2, 3, 4]}
---
### `PUT /job/{job_id}/output`
Set a job's `output` field to given JSON. This has the same effect as updating
using the `PATCH /job/{job_id}` endpoint and setting the `output` field in the
request JSON, but is often more convenient if only updating this field.
#### Response
* 204 - job's output field successfully set
* 400 - invalid or no JSON provided
* 404 - job with given ID does not exist
* 409 - job is not in a state where output can be update (e.g. job is already completed/cancelled)
#### Example
$ curl -i -XPUT -H 'content-type: application/json' -d '[1,2,3]' localhost:8023/job/12/output
HTTP/1.1 204 OK
date: Wed, 21 Nov 2018 11:13:34 GMT
---
## Tag endpoints
Information about jobs that were created with tags can be retreived here.
Tags are typically used as a way of grouping multiple jobs in some way.
Use cases include e.g.:
* using a "username" tag, to track which users have submitted which jobs
* using a "batch ID" tag, to track jobs related to an ETL run or other batch
process
* using a "host" tag, to track which hosts created jobs
---
### `GET /tag/{tag_name}`
Get a JSON list of job IDs with given tag. If no jobs have the requested tag,
or if the tag doesn't exist, then an empty list will be returned.
#### Response
* 200 - JSON list of job ID integers
* 400 - invalid tag name requested
#### Example
$ curl localhost:8023/tag/user3
[17, 19, 225]
---
## Information endpoints
These provide information about the Ocypod system as a whole.
### `GET /info`
TODO
### `GET /info/version`
Get the version string for the currently running Ocypod server.
#### Response
* 200 - JSON version string
#### Example
$ curl localhost:8023/info/version
"0.1.2"
## Healthcheck endpoints
Provides a way of checking the health of the Ocypod server. Returns JSON
containing the health of the system, and optionally a message about any
error detected.
### `GET /health`
Get JSON indicating the health of the Ocypod server. Currently this checks
whether it can successfully connect to and communicate with Redis.
Returns JSON of the form:
{"status": ("healthy"|"unhealthy"), "error": <error message>}
The `error` field is optionally present if they status is "unhealthy", the
"status" field is always present.
#### Response
* 200 - health check completed
#### Example
$ curl localhost:8023/health
{"status": "healthy"}