inferencelayer 0.2.9

Kortexya's engine-native inference layer — LLM generation + embedding/encoder family on wgpu (WGSL kernels, any adapter) with a pure-Rust CPU fallback
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

//! General structured-output grammar: a schema → flat-table → byte-FSM pipeline for GPU-resident
//! constrained decoding (Workstream G).
//!
//! G1 (this module) is CPU-only and the REFERENCE for everything downstream: the same compiled
//! tables the CPU FSM here interprets are the buffers the WGSL interpreter (G2) will upload and read,
//! and the serving layer (G3) drives this FSM at the per-column pick-override seam. Keeping one
//! table-interpreter — rather than compiling a shader per schema — is the design's core bet, and it
//! is de-risked HERE: any JSON-Schema construct that cannot be encoded in the tables surfaces as a
//! pure-Rust compile error before a single byte is generated.
//!
//! Three pieces:
//! - [`json_schema`] — [`JsonSchemaTables::compile`] turns a `serde_json::Value` schema into flat
//!   `u32`/`u8` tables (or refuses, naming the construct). See its docs for the exact array layout.
//! - [`json_fsm`] — [`JsonFsm`] walks those tables byte-by-byte over a fixed [`json_fsm::STATE_WORDS`]
//!   -word [`FsmState`]: `step_byte`, `allowed_bytes` (the constrained-decode mask), `is_complete`,
//!   and `step_token`.
//! - [`token_bytes`] — [`TokenByteTable`] maps a tokenizer's token ids to raw output bytes (byte-level
//!   BPE via the engine's `vocab::build_vocab_blob`, with an SPM/Unigram fallback), so the FSM can be
//!   advanced one whole token at a time.
//!
//! ## Supported subset (OpenAI strict mode)
//!
//! `object`/`array`/`string`/`number`/`integer`/`boolean`/`null`; `properties` + `required` with
//! every property required and matched in the `required` array's order (that array is the canonical
//! key order — a parsed `properties` object cannot preserve declaration order without serde_json's
//! `preserve_order`, which this workspace does not enable); mandatory `additionalProperties:false`;
//! `enum`/`const` over scalars; bounded arrays and strings; nesting depth ≤ 10; canonical output with
//! no whitespace. `pattern`/`anyOf`/`oneOf`/`allOf`/`$ref`/`format`/`minimum`/… are refused loudly,
//! with reserved node-kind discriminants ([`NodeKind::RESERVED_PATTERN`] et al.) held for the future
//! regex-DFA extension.

mod flat_ref;
pub mod gpu;
pub mod harness;
pub mod json_fsm;
pub mod json_schema;
pub mod token_bytes;

pub use gpu::{GpuGrammar, GrammarKernels, GrammarVocab};
pub use harness::{GpuSchema, GpuVocab, SchemaMeta};
pub use json_fsm::{FsmState, JsonFsm};
pub use json_schema::{JsonSchemaTables, NodeKind};
pub use token_bytes::TokenByteTable;