inferencelayer 0.2.3

Kortexya's engine-native inference layer — LLM generation + embedding/encoder family on wgpu (WGSL kernels, any adapter) with a pure-Rust CPU fallback
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

//! Serving-support library for the `lfm2-serve` binary (feature `server`).
//!
//! The binary is deliberately thin — `main`, the axum handlers, the engine threads — while the
//! reusable, pure logic lives here so it is unit-testable without a GPU or a live server:
//!
//! - [`types`] — OpenAI wire structs and the [`types::CommonParams`] validation that range-checks
//!   inputs and rejects unimplemented features with a typed error.
//! - [`error`] — the [`error::ApiError`] OpenAI error envelope (`{"error":{…}}`).
//! - [`chat`] — architecture-based chat templating ([`chat::render_chat_prompt`]).
//! - [`tools`] — portable tool-call advertisement and extraction ([`tools::parse_tool_calls`]).
//! - [`stop`] — stop-sequence scanning and streaming hold-back ([`stop::first_stop`]).

pub mod b64;
pub mod chat;
pub mod error;
pub mod stop;
pub mod tools;
pub mod types;

pub use error::ApiError;