Crate cortiq_server

Expand description

Cortiq Server — OpenAI-compatible API + web management dashboard.

Modules§

api: Cortiq extension API endpoints.
dashboard: Web management dashboard — served as embedded static content.
ood: The organism’s “day” side (docs/NATIVE_MODEL_TECH.ru.md §2): every request touches an idle marker, and requests the skill router finds out-of-distribution (min recon error > τ, or no skills yet) go to an append-only buffer the sleep daemon (cortiq-embryo sleep) bakes new skills from during idle time. Enabled by CMF_OOD_DIR; CMF_OOD_TAU (default 0.30) sets the OOD threshold.
openai: OpenAI-compatible API endpoints, backed by the real inference pipeline. Generation runs in spawn_blocking behind a Mutex — a panic inside the pipeline becomes a 500, never a dead process.
streaming: SSE (Server-Sent Events) streaming for OpenAI-compatible chat completions.

AppState: Shared application state: runtime (masks, metrics), a tokenizer handle that never blocks on generation, and the slot pool.
PipelinePool: Fixed pool of pipeline slots over ONE shared mmap’d model (roadmap §3 «serving полностью сериализован», этап 5.1): the weights are zero-copy shared through Arc<CmfModel>, each slot owns its KV-cache / recurrent state / sampler / workspace. A request checks a slot out for the duration of one generation, so up to slots requests decode CONCURRENTLY; excess requests queue fairly on the semaphore. This is bounded-concurrency serving, not yet continuous batching (этап 5.2+).
SlotGuard: A checked-out slot: holds both the concurrency permit and the pipeline lock until dropped.