Module routes 

Every path ferrox-server serves, named once.

Only routes that actually exist belong here. A constant for a not-yet-implemented endpoint is worse than no constant at all: it reads as a promise, and a client that imports it gets a 404 with the contract crate’s blessing.

Constants

ADMIN_DOWNLOAD

Fetch a .gguf from the Hugging Face Hub into the model directory. Answers 202 with a task id.

ADMIN_MODELS

Model inventory: what is on disk, what is loaded, what failed.

ADMIN_MODELS_LOAD

Start loading a discovered model by its id. Answers 202 with a task id; the load itself runs off the request.

ADMIN_MODELS_UNLOAD

Drop the active model. Synchronous: unloading is releasing one Arc, and requests already decoding keep theirs.

ADMIN_PREPARE_STOP

Close admission, drain, and seal the final accounting snapshot. A supervisor calls this before it sends a signal, so process shutdown cannot race the last sampled token.

ADMIN_STATS

Counters, uptime, and the recent-request ring buffer.

ADMIN_TASKS

Every long-running job this server knows about, newest first.

ADMIN_TASK_CANCEL

Request cancellation of one task. A template, not a literal: the {task_id} placeholder is written in the OpenAPI style rather than any one web framework’s, because this crate is imported by clients that have never heard of the server’s router. Build a concrete path with admin_task_cancel.

ALL

Every fixed route above, for clients that want to enumerate the surface (and for the round-trip test below).

CACHE_STATS

Response- and prefix-cache counters.

COMPLETION

llama.cpp’s native completion endpoint, which is not V1_COMPLETIONS with a shorter path.

COMPLETIONS

llama.cpp mounts its native endpoint under both spellings (tools/server/server.cpp:240-241), the plural being the one its own web UI uses. Not an alias of V1_COMPLETIONS: dropping the /v1 changes the dialect, not just the path.

DETOKENIZE

llama.cpp’s spelling of V1_DETOKENIZE, mounted on the same handler (tools/server/server.cpp:260). The response carries the text under both content (llama.cpp’s key) and text (ferrox’s).

HEALTH

Liveness + readiness + capability handshake. Never behind auth, so a probe works regardless of FERROX_API_KEY.

METRICS

Prometheus text-exposition metrics.

RERANK

llama.cpp’s unprefixed spelling of V1_RERANK, mounted on the same handler (tools/server/server.cpp registers /rerank, /reranking, /v1/rerank and /v1/reranking).

SLOTS_ID

Save or restore one slot’s KV state, llama.cpp’s POST /slots/:id_slot?action=save|restore.

TOKENIZE

llama.cpp’s spelling of V1_TOKENIZE, mounted on the same handler.

V1_CACHE_REBUILD

Re-split the caches on a live engine. New generation is refused while a rebuild is in flight.

V1_CACHE_STATUS

The current cache geometry: how VRAM is split between the expert cache and the KV pools, and what a re-split could move.

V1_CANCEL

Explicit cancellation of one in-flight generation, by the request_id the server states on the first streamed chunk.

V1_CHAT_COMPLETIONS

V1_COMPLETIONS

V1_DETOKENIZE

V1_EMBEDDINGS

V1_MESSAGES

Anthropic-compatible messages endpoint.

V1_MESSAGES_COUNT_TOKENS

Anthropic’s prompt-sizing endpoint: how many input tokens a request would cost, without generating any. Behind the same key as V1_MESSAGES, because answering it requires the loaded checkpoint’s own tokenizer and chat template.

V1_MODELS

V1_REQUESTS

Incremental page over the recent-request ring: ?since=<cursor> and ?limit=<n>. The cursor is all-time, so a poller that keeps up reads each row exactly once.

V1_RERANK

Cross-encoder reranking: one query against N documents, scored by the checkpoint’s own classification head (cls / cls.output), not by the cosine similarity of two embeddings.

V1_RESPONSE

One stored response. This server is stateless, so it answers 404 – deliberately, rather than 404-ing from the router, because the two say different things: the route EXISTS and keeps nothing, which tells a client to stop polling rather than to check its base URL.

V1_RESPONSES

The OpenAI Responses surface – what codex speaks. A different request/response shaping over the same generation path, not a second engine.

V1_RESPONSE_CANCEL

Cancel one stored response. Same stateless answer; a live generation is stopped through V1_CANCEL with its request_id.

V1_STATS

Live serving telemetry: throughput over a trailing window, request latency percentile, and the cache pools’ occupancy. Distinct from ADMIN_STATS, which is this server’s own operational ring; this is the shape a desktop or dashboard polls.

V1_STREAM

Reconnect into a stream started with stream_resumable: true, resuming after the Last-Event-ID the client last saw.

V1_STREAM_POLL

The same replay buffer over plain JSON, for the case SSE cannot survive: a reverse proxy that buffers text/event-stream turns a stream into one long silence, and cannot do that to a short response that has already ended. Build a concrete path with v1_stream_poll.

V1_TOKENIZE

Functions

admin_task_cancel

The concrete cancel path for one task id.

slots_id

The concrete slot path for one slot id.

v1_stream

The concrete resume path for one request id.

v1_stream_poll

The concrete polling-fallback path for one request id.