Crate memra_server 

memra-server (BASE-4): a minimal OpenAI-ish HTTP server that serves 2-4 concurrent agents across DIFFERENT models on one endpoint via a single GPU worker thread + step-interleave scheduler.

Architecture (see worker.rs): axum runs on a tokio runtime; ONE dedicated std::thread owns the Engine + every loaded HybridModel (CUDA context is thread-affine). Handlers submit Cmds over a std mpsc channel and receive tokens back over a per-request tokio mpsc channel.

Endpoints (the full set — router() below is the authority): GET /health, GET /livez -> the SAME handler (health_live): INFERENCE liveness, not process liveness. {“status”:“ok”|“draining”|“unhealthy”, “models”:[…], “worker”:{phase, beat_age_ms, tick_max_ms, stall_threshold_ms, generation, xid_warnings}} + a top-level “detail” on a red. Draining stays 200; dead / GPU-faulted / stalled / loading is 503 (serve-hardening 2026-08-06). GET /readyz -> routability, same payload shape with “status”:“ready”|“not_ready”. Unready is NOT a restart request — draining and loading are healthy-but-unroutable. GET /models -> {“data”:[{“id”:name},…]} (OpenAI-ish); ?schema=openrouter -> Provider Monitor schema 2.4, ?schema=openmodels -> OpenModels provider feed. GET /v1/models -> existing catalog-style model list (context_length, architecture, pricing stub, top_provider; serve-tail). GET /metrics -> flat serving counters + step latency percentiles. GET /yield/metrics -> per-lane x-lane QoS counters + engine-truth step p50/p99 (lane/qos-p95 2026-08-02). POST /v1/completions -> {model,prompt|prompt_ids,max_tokens,temperature?,top_p?,top_k?, seed?,stop?,chat?,stream?,cache_salt?}. stream=true => SSE token-by-token; else a single JSON {text,tokens,stop_reason}. POST /v1/chat/completions -> OpenAI chat messages rendered by the GGUF chat template; OpenAI message/chunk response shapes. tools/tool_choice (auto|none) + role:“tool” turns render through the template’s own branch; emitted <tool_call> blocks parse into OpenAI tool_calls (+“tool_calls” finish); reasoning_effort/reasoning map onto the template’s think switch (serve-tools lane, 2026-08-02).

CONFIG: MEMRA_MODELS=“name=/path.gguf[+/draft.gguf],name2=hf:owner/repo,name3=/hf_ckpt_dir” (comma-separated; +draft.gguf attaches that model’s regime draft — docs/DRAFT-REGIME.md). A model path may be a GGUF file OR an HF safetensors checkpoint directory (config.json + model.safetensors[.index.json] — the run-safetensors load path; serve-st lane 2026-08-04). Defaults to the BASE-4 test pair (main=27B, judge=9B) if unset. MEMRA_ADDR sets the bind addr.

LIFECYCLE: SIGTERM = graceful drain (gap-scan F11) — new completion requests 503 with Retry-After, /health reports “draining”, in-flight requests (streams included) finish up to MEMRA_DRAIN_S (default 30s), then the process exits 0. Completion responses carry X-RateLimit-Limit/-Remaining/-Reset (concurrency-slot semantics; gap-scan F12).

Modules

auth

x-lane QoS (lane/dl-metering gate, QoS-only extraction 2026-08-02): lane types, SLO admission policy, engine-truth step stats live in the memra-lanes crate so out-of-process controllers (the sidecar shape) can share them.

metering

The admission/accounting seam: the server admits, denies, and reports counts; what admission MEANS — budgets, prices, tenancy policy — is a deployment concern, supplied behind metering::Metering through ServerWiring. The stock binary ships NO accounting (only the engine is open; the business tier lives in the deployment’s own binary — engine-billing-extraction-20260829, owner razor 2026-08-29: “only engine is open, business is private”). The admission/accounting seam (lane engine-billing-extraction-20260829).

Structs

HostHandoffHandle

Host-tier deploy handoff (lane/host-tier-deploy-warmth-20260901): the engine half of a deployment admin POST /admin/kv-host/export / POST /admin/kv-host/import pair. Contract notes for the deployment surface: export is called ONLY on the drained old slot (it refuses under traffic unless force, and the write stalls that slot’s ticks for its duration, expected and harmless when drained); import answers as soon as the file header validates, then re-materializes entries one per tick in the background (watch prefix_host_handoff_* in /metrics for completion). Same lifetime contract as TrimHandle: drop it on the shutdown signal.

PurgeHandle

Purge one tenant’s parked KV state (the engine half of a deployment admin /admin/tenants/{tenant}/purge; lane/kv-tenancy-compaction-20260831, tiering spec §0.5). Contract notes for the deployment surface: the path parameter is {tenant} (the keyring tenant id, the same string --gen-key <tenant> took), never {tenant_id}; fire it from key revocation AND tenant deletion; a report with device_pinned_left > 0 means in-flight sessions still lease device entries in the tenant’s namespaces, so re-fire after the drain. Cloneable, same lifetime contract as TrimHandle: drop it on the shutdown signal.

RuntimeHandles

Runtime handles handed to ServerWiring::on_ready — the narrow set of engine-runtime operations a deployment-side admin surface needs.

ServerWiring

Deployment wiring for a custom binary. serve_main is exactly serve_with(ServerWiring::reference()); a deployment-owned binary substitutes its own metering and hooks the runtime handles it needs.

TrimHandle

Ask the worker to trim its pools (the engine half of /admin/trim). Cloneable; answers with the worker’s own trim report.

Constants

BUILD_GIT_SHA

The build’s git sha when the build could read a repo, else unknown. An EXTRA provenance field: convenient, never the identity. A shipped binary outlives the commit it was cut from, and after an authorized history rewrite the sha names nothing at all.

BUILD_ID_NOTE

Why the id is degraded. Empty when it is not.

BUILD_ID_SRC

How SYSTEM_FINGERPRINT’s id was derived: source-tree (real) or degraded.

SYSTEM_FINGERPRINT

Backend-config fingerprint: memra-<crate version>-<content id>, baked by build.rs from the crate version plus a digest of the workspace’s compiled inputs. Together with seed, responses are checkable for determinism across deploys — the OpenAI system_fingerprint contract.

Functions

build_identity_line

One line of build provenance, printed at boot by EVERY binary that links this server (the stock bin and darklanes’ deployment bin both enter through serve_with).

serve_main

serve_with