Crate rskit_inference 

Model-serving runtime adapter layer for rskit.

rskit-inference owns runtime-neutral prediction contracts for Triton, vLLM raw, TGI, KServe v2, BentoML, ONNX Runtime Server, TFServing, and custom REST/gRPC serving runtimes. It intentionally does not model chat completions; chat belongs to rskit-llm and provider implementations under rskit-llm-providers.

Adapters declare their executable authority through rskit_tool::Envelope on InferenceDescriptor. Consumers and orchestrators enforce envelope intersection with the five-stage permission model (declaration, registry verification, activation, per-invocation enforcement, HITL elicitation).

Re-exports

pub use echo::Echo;

pub use echo::register as register_echo;

pub use inference::Inference;

pub use inference::StreamingInference;

pub use registry::Factory;

pub use registry::Registry;

pub use registry::RegistryError;

pub use registry::default_registry;

pub use types::InferenceDescriptor;

pub use types::InferenceError;

pub use types::PredictRequest;

pub use types::PredictResponse;

pub use types::PredictStatus;

pub use types::ServingProtocol;

pub use types::Tensor;

pub use types::TensorData;

pub use types::Value;

Modules

echo

Lean default echo adapter. Deterministic echo inference adapter.

inference

Model-serving inference traits.

registry

Explicit adapter registry.

types

Runtime-neutral request, response, value, descriptor, and error types.

Structs

Model

Canonical model identity.

Usage

Token usage counters.

Traits

StreamEvent

Incremental event emitted during an AI stream.

Type Aliases

StreamEventRef

Shared stream event reference used across async boundaries.