Crate openvino_genai 

The openvino-genai crate provides high-level, ergonomic, safe Rust bindings to OpenVINO GenAI. See the repository README for more information, such as build instructions.

Most interaction with OpenVINO GenAI begins with instantiating an LlmPipeline:

let pipeline = openvino_genai::LlmPipeline::new("path/to/model", "CPU")
    .expect("to create an LLM pipeline");

Structs

ChatHistory

A chat history for multi-turn conversations.

DecodedResults

Results from LLM text generation.

GenerationConfig

Configuration for text generation.

JsonContainer

A JSON container for structured data exchange with the GenAI C API.

LlmPipeline

A pipeline for generating text using large language models.

PerfMetrics

Performance metrics from a generation operation.

Streamer

A streaming callback that receives tokens as they are generated.

ToolCall

A single tool call requested by the model.

VlmDecodedResults

Results from VLM text generation.

VlmPipeline

A pipeline for generating text from text+image inputs using Vision-Language Models.

WhisperDecodedResultChunk

A timestamped chunk from Whisper speech recognition results.

WhisperDecodedResults

Results from Whisper speech recognition.

WhisperGenerationConfig

Configuration for Whisper speech recognition.

WhisperPipeline

A pipeline for speech recognition using Whisper models.

Enums

ChatMessage

A chat message with typed variants for each role.

InferenceError

See ov_status_e; enumerates errors returned by the OpenVINO GenAI implementation.

LoadingError

Enumerate the ways that library loading can fail.

SetupError

Enumerate setup failures: in some cases, this library will call library-loading code that may fail in a different way (i.e., LoadingError) than the calls to the OpenVINO GenAI libraries (i.e., InferenceError).

StreamingStatus

The status returned from a streaming callback to control generation.

Functions

load

Load the OpenVINO GenAI shared library using automatic discovery.

load_from

Load the OpenVINO GenAI shared library from an explicit path.