Crate openvino_genai_sys 

This crate provides low-level, unsafe, Rust bindings to OpenVINO™ GenAI using its C API. If you are looking to use OpenVINO™ GenAI from Rust, you likely should look at the ergonomic, safe bindings in openvino-genai, which depends on this crate. See the repository README for more information, including build instructions.

An example interaction with raw openvino-genai-sys:

openvino_genai_sys::library::load().expect("to have an OpenVINO GenAI library available");

Modules

library

Contains extra utilities for finding and loading the OpenVINO GenAI shared libraries.

Structs

ov_genai_chat_history_opaque

ov_genai_decoded_results_opaque

ov_genai_generation_config_opaque

ov_genai_json_container_opaque

ov_genai_llm_pipeline_opaque

ov_genai_perf_metrics_opaque

ov_genai_vlm_decoded_results_opaque

ov_genai_vlm_pipeline_opaque

ov_genai_whisper_decoded_result_chunk_opaque

ov_genai_whisper_decoded_results_opaque

ov_genai_whisper_generation_config_opaque

ov_genai_whisper_pipeline_opaque

streamer_callback

@brief Structure for streamer callback functions with arguments.

Enums

StopCriteria

@brief controls the stopping condition for grouped beam search. The following values are possible: “EARLY” stops as soon as there are num_beams complete candidates. “HEURISTIC” stops when is it unlikely to find better candidates. “NEVER” stops when there cannot be better candidates.

ov_genai_chat_history_status_e

@brief Status codes for chat history operations

ov_genai_json_container_status_e

@brief Status codes for JsonContainer operations

ov_genai_streaming_status_e

ov_status_e

@enum ov_status_e @ingroup ov_base_c_api @brief This enum contains codes for all possible return values of the interface functions

Functions

load

Load all of the function definitions from a shared library.

load_from

Load all of the function definitions from a shared library at the given path.

ov_genai_chat_history_clear^⚠

@brief Clear all messages from the chat history. @param history A pointer to the ov_genai_chat_history instance. @return ov_genai_chat_history_status_e A status code, return OK(0) if successful.

ov_genai_chat_history_create^⚠

@brief Create a new empty ChatHistory instance. @param history A pointer to the newly created ov_genai_chat_history. @return ov_genai_chat_history_status_e A status code, return OK(0) if successful.

ov_genai_chat_history_create_from_json_container^⚠

@brief Create a ChatHistory instance from a JsonContainer (array). @param history A pointer to the newly created ov_genai_chat_history. @param messages A JsonContainer containing an array of message objects. @return ov_genai_chat_history_status_e A status code, return OK(0) if successful.

ov_genai_chat_history_empty^⚠

@brief Check if the chat history is empty. @param history A pointer to the ov_genai_chat_history instance. @param empty A pointer to store the boolean result (1 for empty, 0 for not empty). @return ov_genai_chat_history_status_e A status code, return OK(0) if successful.

ov_genai_chat_history_free^⚠

@brief Release the memory allocated by ov_genai_chat_history. @param history A pointer to the ov_genai_chat_history to free memory.

ov_genai_chat_history_get_extra_context^⚠

@brief Get extra context as a JsonContainer (object). @param history A pointer to the ov_genai_chat_history instance. @param extra_context A pointer to store the returned JsonContainer containing extra context. @return ov_genai_chat_history_status_e A status code, return OK(0) if successful.

ov_genai_chat_history_get_first^⚠

@brief Get the first message as a JsonContainer. @param history A pointer to the ov_genai_chat_history instance. @param message A pointer to store the returned JsonContainer containing the first message. @return ov_genai_chat_history_status_e A status code, return OK(0) if successful.

ov_genai_chat_history_get_last^⚠

@brief Get the last message as a JsonContainer. @param history A pointer to the ov_genai_chat_history instance. @param message A pointer to store the returned JsonContainer containing the last message. @return ov_genai_chat_history_status_e A status code, return OK(0) if successful.

ov_genai_chat_history_get_message^⚠

@brief Get a message at a specific index as a JsonContainer. @param history A pointer to the ov_genai_chat_history instance. @param index The index of the message to retrieve. @param message A pointer to store the returned JsonContainer containing the message. @return ov_genai_chat_history_status_e A status code, return OK(0) if successful.

ov_genai_chat_history_get_messages^⚠

@brief Get all messages as a JsonContainer (array). @param history A pointer to the ov_genai_chat_history instance. @param messages A pointer to store the returned JsonContainer containing all messages. @return ov_genai_chat_history_status_e A status code, return OK(0) if successful.

ov_genai_chat_history_get_tools^⚠

@brief Get tools definitions as a JsonContainer (array). @param history A pointer to the ov_genai_chat_history instance. @param tools A pointer to store the returned JsonContainer containing tools definitions. @return ov_genai_chat_history_status_e A status code, return OK(0) if successful.

ov_genai_chat_history_pop_back^⚠

@brief Remove the last message from the chat history. @param history A pointer to the ov_genai_chat_history instance. @return ov_genai_chat_history_status_e A status code, return OK(0) if successful.

ov_genai_chat_history_push_back^⚠

@brief Add a message to the chat history from a JsonContainer. @param history A pointer to the ov_genai_chat_history instance. @param message A JsonContainer containing a message object (e.g., {“role”: “user”, “content”: “Hello”}). @return ov_genai_chat_history_status_e A status code, return OK(0) if successful.

ov_genai_chat_history_set_extra_context^⚠

@brief Set extra context (for custom template variables) as a JsonContainer (object). @param history A pointer to the ov_genai_chat_history instance. @param extra_context A JsonContainer containing an object with extra context. @return ov_genai_chat_history_status_e A status code, return OK(0) if successful.

ov_genai_chat_history_set_tools^⚠

@brief Set tools definitions (for function calling) as a JsonContainer (array). @param history A pointer to the ov_genai_chat_history instance. @param tools A JsonContainer containing an array of tool definitions. @return ov_genai_chat_history_status_e A status code, return OK(0) if successful.

ov_genai_chat_history_size^⚠

@brief Get the number of messages in the chat history. @param history A pointer to the ov_genai_chat_history instance. @param size A pointer to store the size (number of messages). @return ov_genai_chat_history_status_e A status code, return OK(0) if successful.

ov_genai_decoded_results_create^⚠

@brief Create DecodedResults @param results A pointer to the newly created ov_genai_decoded_results. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_decoded_results_free^⚠

@brief Release the memory allocated by ov_genai_decoded_results. @param model A pointer to the ov_genai_decoded_results to free memory.

ov_genai_decoded_results_get_perf_metrics^⚠

@brief Get performance metrics from ov_genai_decoded_results. @param results A pointer to the ov_genai_decoded_results instance. @param metrics A pointer to the newly created ov_genai_perf_metrics. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_decoded_results_get_string^⚠

@brief Get string result from ov_genai_decoded_results. @param results A pointer to the ov_genai_decoded_results instance. @param output A pointer to the pre-allocated output string buffer. It can be set to NULL, in which case the *output_size will provide the needed buffer size. The user should then allocate the required buffer size and call this function again to obtain the entire output. @param output_size A Pointer to the size of the output string from the results, including the null terminator. If output is not NULL, *output_size should be greater than or equal to the result string size; otherwise, the function will return OUT_OF_BOUNDS(-6). @return ov_status_e A status code, return OK(0) if successful.

ov_genai_decoded_results_perf_metrics_free^⚠

@brief Release the memory allocated by ov_genai_perf_metrics. @param model A pointer to the ov_genai_perf_metrics to free memory.

ov_genai_generation_config_create^⚠

@brief Create ov_genai_generation_config. @param config A pointer to the newly created ov_genai_generation_config. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_create_from_json^⚠

@brief Create ov_genai_generation_config from JSON file. @param json_path Path to a .json file containing the generation configuration to load. @param config A pointer to the newly created ov_genai_generation_config. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_free^⚠

@brief Release the memory allocated by ov_genai_generation_config. @param handle A pointer to the ov_genai_generation_config to free memory. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_get_max_new_tokens^⚠

@brief Get the maximum number of tokens to generate, excluding the number of tokens in the prompt. @param handle A pointer to the ov_genai_generation_config instance. @param The maximum number of tokens to generate. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_set_assistant_confidence_threshold^⚠

@brief Set the lower token probability of candidate to be validated by main model in case of dynamic strategy candidates number update. @param handle A pointer to the ov_genai_generation_config instance. @param value The lower token probability of candidate to be validated by main model. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_set_diversity_penalty^⚠

@brief Set the diversity penalty, this value is subtracted from a beam’s score if it generates the same token as any beam from other group at a particular time. @param handle A pointer to the ov_genai_generation_config instance. @param value The parameter for diversity penalty. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_set_do_sample^⚠

@brief Set whether or not to use multinomial random sampling that add up to top_p or higher are kept. @param handle A pointer to the ov_genai_generation_config instance. @param value If set to true, multinomial random sampling will be used. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_set_echo^⚠

@brief Set whether or not to include user prompt in the output. @param handle A pointer to the ov_genai_generation_config instance. @param value If set to true, output will include user prompt. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_set_eos_token_id^⚠

@brief Set the token_id of (end of sentence) @param handle A pointer to the ov_genai_generation_config instance. @param id The eos token id. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_set_frequency_penalty^⚠

@brief Set the frequency penalty, which reduces absolute log prob as many times as the token was generated. @param handle A pointer to the ov_genai_generation_config instance. @param value The value of parameter for frequency penalty. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_set_ignore_eos^⚠

@brief Set whether or not to ignore token @param handle A pointer to the ov_genai_generation_config instance. @param value If set to true, then generation will not stop even if token is met. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_set_include_stop_str_in_output^⚠

@brief Set whether or not to include stop string that matched generation in the output. @param handle A pointer to the ov_genai_generation_config instance. @param value If set to true stop string that matched generation will be included in generation output (default: false). @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_set_length_penalty^⚠

@brief Set the length penalty, exponential penalty to the length that is used with beam-based generation. It is applied as an exponent to the sequence length, which in turn is used to divide the score of the sequence. Since the score is the log likelihood of the sequence (i.e. negative), length_penalty > 0.0 promotes longer sequences, while length_penalty < 0.0 encourages shorter sequences. @param handle A pointer to the ov_genai_generation_config instance. @param value The exponential penalty. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_set_logprobs^⚠

@brief Set the number of top logprobs computed for each position, if set to 0, logprobs are not computed and value 0.0 is returned. Currently only single top logprob can be returned, so any logprobs > 1 is treated as logprobs == 1.(default: 0). @param handle A pointer to the ov_genai_generation_config instance. @param value The number of top logprobs computed for each position. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_set_max_length^⚠

@brief Set the maximum length the generated tokens can have. Corresponds to the length of the input prompt + max_new_tokens. Its effect is overridden by max_new_tokens, if also set. @param handle A pointer to the ov_genai_generation_config instance. @param value The maximum length the generated tokens can have. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_set_max_new_tokens^⚠

@brief Set the maximum number of tokens to generate, excluding the number of tokens in the prompt. max_new_tokens has priority over max_length. @param handle A pointer to the ov_genai_generation_config instance. @param value The maximum number of tokens to generate. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_set_max_ngram_size^⚠

@brief Set the maximum ngram to use when looking for matches in the prompt. @param handle A pointer to the ov_genai_generation_config instance. @param value The maximum ngram size. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_set_min_new_tokens^⚠

@brief Set the minimum number of tokens to generate. @param handle A pointer to the ov_genai_generation_config instance. @param value The minimum number of tokens to generate. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_set_no_repeat_ngram_size^⚠

@brief Set the no_repeat_ngram_size @param handle A pointer to the ov_genai_generation_config instance. @param value If set to int > 0, all ngrams of that size can only occur once. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_set_num_assistant_tokens^⚠

@brief Set the defined candidates number to be generated by draft model/prompt lookup in case of static strategy candidates number update. @param handle A pointer to the ov_genai_generation_config instance. @param value The number of assistant tokens. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_set_num_beam_groups^⚠

@brief Set the number of groups to divide num_beams into in order to ensure diversity among different groups of beams. @param handle A pointer to the ov_genai_generation_config instance. @param value The number of beam groups. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_set_num_beams^⚠

@brief Set the number of beams for beam search. 1 disables beam search. @param handle A pointer to the ov_genai_generation_config instance. @param value The number of beams for beam search. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_set_num_return_sequences^⚠

@brief Set the number of sequences to return for grouped beam search decoding per batch element. num_return_sequences must be less or equal to num_beams. @param handle A pointer to the ov_genai_generation_config instance. @param value The number of sequences to return. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_set_presence_penalty^⚠

@brief Set the presence penalty, which reduces absolute log prob if the tokeov_genai_generation_config_set_presence_penaltyn was generated at least once. @param handle A pointer to the ov_genai_generation_config instance. @param value The value of parameter for presence penalty. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_set_repetition_penalty^⚠

@brief Set the parameter for repetition penalty. 1.0 means no penalty. @param handle A pointer to the ov_genai_generation_config instance. @param value The value of parameter for repetition penalty. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_set_rng_seed^⚠

@brief Set the seed to initialize random number generator. @param handle A pointer to the ov_genai_generation_config instance. @param value The value of seed for random number generator. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_set_stop_criteria^⚠

@brief Set the stopping condition for grouped beam search. It accepts the following values: “EARLY”, where the generation stops as soon as there are num_beams complete candidates; “HEURISTIC”, where an “HEURISTIC” is applied when it is unlikely to find better candidates; “NEVER”, where the generation stops when there cannot be better candidates. @param handle A pointer to the ov_genai_generation_config instance. @param value The stopping condition for grouped beam search. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_set_stop_strings^⚠

@brief Set the set of strings that will cause pipeline to stop generating further tokens. @param handle A pointer to the ov_genai_generation_config instance. @param strings An array of strings. @param count The number of strings in the array. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_set_stop_token_ids^⚠

@brief Set the set of tokens that will cause pipeline to stop generating further tokens. @param handle A pointer to the ov_genai_generation_config instance. @param token_ids An array of token ids. @param token_ids_num The number of token ids in the array. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_set_temperature^⚠

@brief Set the temperature value used to modulate token probabilities for random sampling. @param handle A pointer to the ov_genai_generation_config instance. @param value The value of temperature. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_set_top_k^⚠

@brief Set the top_k value. The number of highest probability vocabulary tokens to keep for top-k-filtering. @param handle A pointer to the ov_genai_generation_config instance. @param value The value of top_k. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_set_top_p^⚠

@brief Set the top_p value. If set to float < 1, only the smallest set of most probable tokens with probabilities that add up to top_p or higher are kept for generation. @param handle A pointer to the ov_genai_generation_config instance. @param value The value of top_p. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_generation_config_validate^⚠

@brief Checks that are no conflicting parameters, e.g. do_sample=true and num_beams > 1. @param handle A pointer to the ov_genai_generation_config instance. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_json_container_copy^⚠

@brief Create a copy of JsonContainer. @param source A pointer to the source ov_genai_json_container instance. @param target A pointer to store the copied ov_genai_json_container. @return ov_genai_json_container_status_e A status code, return OK(0) if successful.

ov_genai_json_container_create^⚠

@brief Create a new empty JsonContainer instance (as an object). @param container A pointer to the newly created ov_genai_json_container. @return ov_genai_json_container_status_e A status code, return OK(0) if successful.

ov_genai_json_container_create_array^⚠

@brief Create a JsonContainer instance as an empty JSON array. @param container A pointer to the newly created ov_genai_json_container. @return ov_genai_json_container_status_e A status code, return OK(0) if successful.

ov_genai_json_container_create_from_json_string^⚠

@brief Create a JsonContainer instance from a JSON string. @param container A pointer to the newly created ov_genai_json_container. @param json_str A JSON string (object, array, or primitive). @return ov_genai_json_container_status_e A status code, return OK(0) if successful.

ov_genai_json_container_create_object^⚠

@brief Create a JsonContainer instance as an empty JSON object. @param container A pointer to the newly created ov_genai_json_container. @return ov_genai_json_container_status_e A status code, return OK(0) if successful.

ov_genai_json_container_free^⚠

@brief Release the memory allocated by ov_genai_json_container. @param container A pointer to the ov_genai_json_container to free memory.

ov_genai_json_container_to_json_string^⚠

@brief Convert JsonContainer to JSON string. @param container A pointer to the ov_genai_json_container instance. @param output A pointer to the pre-allocated output string buffer. It can be set to NULL, in which case the *output_size will provide the needed buffer size. The user should then allocate the required buffer size and call this function again to obtain the entire output. @param output_size A pointer to the size of the output string, including the null terminator. If output is not NULL, *output_size should be greater than or equal to the result string size; otherwise, the function will return OUT_OF_BOUNDS(-3). @return ov_genai_json_container_status_e A status code, return OK(0) if successful.

ov_genai_llm_pipeline_create^⚠

Create an LLM pipeline (runtime-linking variant).

ov_genai_llm_pipeline_finish_chat^⚠

@brief Finish chat and clear kv cache. @param pipe A pointer to the ov_genai_llm_pipeline instance. @return Status code of the operation: OK(0) for success.

ov_genai_llm_pipeline_free^⚠

@brief Release the memory allocated by ov_genai_llm_pipeline. @param model A pointer to the ov_genai_llm_pipeline to free memory.

ov_genai_llm_pipeline_generate^⚠

@brief Generate results by ov_genai_llm_pipeline @param pipe A pointer to the ov_genai_llm_pipeline instance. @param inputs A pointer to the input string. @param config A pointer to the ov_genai_generation_config, the pointer can be NULL. @param streamer A pointer to the stream callback. Set to NULL if no callback is needed. Either this or results must be non-NULL. @param results A pointer to the ov_genai_decoded_results, which retrieves the results of the generation. Either this or streamer must be non-NULL. @return Status code of the operation: OK(0) for success.

ov_genai_llm_pipeline_generate_with_history^⚠

@brief Generate results by ov_genai_llm_pipeline using ChatHistory @param pipe A pointer to the ov_genai_llm_pipeline instance. @param history A pointer to the ov_genai_chat_history instance. @param config A pointer to the ov_genai_generation_config, the pointer can be NULL. @param streamer A pointer to the stream callback. Set to NULL if no callback is needed. Either this or results must be non-NULL. @param results A pointer to the ov_genai_decoded_results, which retrieves the results of the generation. Either this or streamer must be non-NULL. @return Status code of the operation: OK(0) for success.

ov_genai_llm_pipeline_get_generation_config^⚠

@brief Get the GenerationConfig from ov_genai_llm_pipeline. @param pipe A pointer to the ov_genai_llm_pipeline instance. @param ov_genai_generation_config A pointer to the newly created ov_genai_generation_config. @return Status code of the operation: OK(0) for success.

ov_genai_llm_pipeline_set_generation_config^⚠

@brief Set the GenerationConfig to ov_genai_llm_pipeline. @param pipe A pointer to the ov_genai_llm_pipeline instance. @param config A pointer to the ov_genai_generation_config instance. @return Status code of the operation: OK(0) for success.

ov_genai_llm_pipeline_start_chat^⚠

@brief Start chat with keeping history in kv cache. @param pipe A pointer to the ov_genai_llm_pipeline instance. @return Status code of the operation: OK(0) for success.

ov_genai_perf_metrics_add_in_place^⚠

@brief C interface for PerfMetrics& operator+=(const PerfMetrics& right)

ov_genai_perf_metrics_get_detokenization_duration^⚠

@brief Get detokenization duration (in ms) from ov_genai_perf_metrics. @param metrics A pointer to the ov_genai_perf_metrics instance. @param mean Mean of detokenization duration. @param std Standard deviation of detokenization duration. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_perf_metrics_get_generate_duration^⚠

@brief Get generate duration (in ms) from ov_genai_perf_metrics. @param metrics A pointer to the ov_genai_perf_metrics instance. .* @param mean Mean of generate duration. @param std Standard deviation of generate duration. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_perf_metrics_get_inference_duration^⚠

@brief Get inference duration (in ms) from ov_genai_perf_metrics. @param metrics A pointer to the ov_genai_perf_metrics instance. @param mean Mean of inference duration. @param std Standard deviation of inference duration. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_perf_metrics_get_ipot^⚠

@brief Get the inference time (in ms) per output token from ov_genai_perf_metrics. @param metrics A pointer to the ov_genai_perf_metrics instance. @param mean Mean of inference time per input token. @param std Standard deviation of inference time per input token. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_perf_metrics_get_load_time^⚠

@brief Get load time from ov_genai_perf_metrics. @param metrics A pointer to the ov_genai_perf_metrics instance. @param load_time Load time in ms. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_perf_metrics_get_num_generation_tokens^⚠

@brief Get the number of generated tokens from ov_genai_perf_metrics. @param metrics A pointer to the ov_genai_perf_metrics instance. @param num_generation_tokens The number of generated tokens. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_perf_metrics_get_num_input_tokens^⚠

@brief Get the number of input tokens from ov_genai_perf_metrics. @param metrics A pointer to the ov_genai_perf_metrics instance. @param num_input_tokens The number of input tokens. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_perf_metrics_get_throughput^⚠

@brief Get tokens per second from ov_genai_perf_metrics. @param metrics A pointer to the ov_genai_perf_metrics instance. @param mean Mean of throughput. @param std Standard deviation of throughput. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_perf_metrics_get_tokenization_duration^⚠

@brief Get tokenization duration (in ms) from ov_genai_perf_metrics. @param metrics A pointer to the ov_genai_perf_metrics instance. @param mean Mean of tokenization duration. @param std Standard deviation of tokenization duration. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_perf_metrics_get_tpot^⚠

@brief Get the time per output token (TPOT in ms) from ov_genai_perf_metrics. @param metrics A pointer to the ov_genai_perf_metrics instance. @param mean Mean of time per output token. @param std Standard deviation of time per output token. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_perf_metrics_get_ttft^⚠

@brief Get the time to first token (in ms) from ov_genai_perf_metrics. @param metrics A pointer to the ov_genai_perf_metrics instance. @param mean Mean of time to first token. @param std Standard deviation of time to first token. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_vlm_decoded_results_create^⚠

@brief Create VLMDecodedResults @param results A pointer to the newly created ov_genai_vlm_decoded_results. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_vlm_decoded_results_free^⚠

@brief Release the memory allocated by ov_genai_vlm_decoded_results. @param results A pointer to the ov_genai_vlm_decoded_results to free memory.

ov_genai_vlm_decoded_results_get_perf_metrics^⚠

@brief Get performance metrics from ov_genai_vlm_decoded_results. @param results A pointer to the ov_genai_vlm_decoded_results instance. @param metrics A pointer to the newly created ov_genai_perf_metrics. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_vlm_decoded_results_get_string^⚠

@brief Get string result from ov_genai_vlm_decoded_results. @param results A pointer to the ov_genai_vlm_decoded_results instance. @param output A pointer to the pre-allocated output string buffer. It can be set to NULL, in which case the *output_size will provide the needed buffer size. The user should then allocate the required buffer size and call this function again to obtain the entire output. @param output_size A Pointer to the size of the output string from the results, including the null terminator. If output is not NULL, *output_size should be greater than or equal to the result string size; otherwise, the function will return OUT_OF_BOUNDS(-6). @return ov_status_e A status code, return OK(0) if successful.

ov_genai_vlm_decoded_results_perf_metrics_free^⚠

@brief Release the memory allocated by ov_genai_perf_metrics. @param metrics A pointer to the ov_genai_perf_metrics to free memory.

ov_genai_vlm_pipeline_create^⚠

Create a VLM pipeline (runtime-linking variant).

ov_genai_vlm_pipeline_finish_chat^⚠

@brief Finish chat and clear kv cache. @param pipe A pointer to the ov_genai_vlm_pipeline instance. @return Status code of the operation: OK(0) for success.

ov_genai_vlm_pipeline_free^⚠

@brief Release the memory allocated by ov_genai_vlm_pipeline. @param pipe A pointer to the ov_genai_vlm_pipeline to free memory.

ov_genai_vlm_pipeline_generate^⚠

@brief Generate results by ov_genai_vlm_pipeline with text and image inputs @param pipe A pointer to the ov_genai_vlm_pipeline instance. @param text_inputs A pointer to the input text string. @param rgbs A pointer to the array of ov_tensor_t containing image data. @param num_images Number of images in the rgbs array. @param config A pointer to the ov_genai_generation_config, the pointer can be NULL. @param streamer A pointer to the stream callback. Set to NULL if no callback is needed. Either this or results must be non-NULL. @param results A pointer to the ov_genai_vlm_decoded_results, which retrieves the results of the generation. Either this or streamer must be non-NULL. @return Status code of the operation: OK(0) for success.

ov_genai_vlm_pipeline_get_generation_config^⚠

@brief Get the GenerationConfig from ov_genai_vlm_pipeline. @param pipe A pointer to the ov_genai_vlm_pipeline instance. @param config A pointer to the newly created ov_genai_generation_config. @return Status code of the operation: OK(0) for success.

ov_genai_vlm_pipeline_set_generation_config^⚠

@brief Set the GenerationConfig to ov_genai_vlm_pipeline. @param pipe A pointer to the ov_genai_vlm_pipeline instance. @param config A pointer to the ov_genai_generation_config instance. @return Status code of the operation: OK(0) for success.

ov_genai_vlm_pipeline_start_chat^⚠

@brief Start chat with keeping history in kv cache. @param pipe A pointer to the ov_genai_vlm_pipeline instance. @return Status code of the operation: OK(0) for success.

ov_genai_whisper_decoded_result_chunk_create^⚠

@brief Create WhisperDecodedResultChunk @param chunk A pointer to the newly created ov_genai_whisper_decoded_result_chunk. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_decoded_result_chunk_free^⚠

@brief Release the memory allocated by ov_genai_whisper_decoded_result_chunk. @param chunk A pointer to the ov_genai_whisper_decoded_result_chunk to free memory.

ov_genai_whisper_decoded_result_chunk_get_end_ts^⚠

@brief Get end timestamp from ov_genai_whisper_decoded_result_chunk. @param chunk A pointer to the ov_genai_whisper_decoded_result_chunk instance. @param end_ts A pointer to the end timestamp value. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_decoded_result_chunk_get_start_ts^⚠

@brief Get start timestamp from ov_genai_whisper_decoded_result_chunk. @param chunk A pointer to the ov_genai_whisper_decoded_result_chunk instance. @param start_ts A pointer to the start timestamp value. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_decoded_result_chunk_get_text^⚠

@brief Get text from ov_genai_whisper_decoded_result_chunk. @param chunk A pointer to the ov_genai_whisper_decoded_result_chunk instance. @param text A pointer to the pre-allocated text buffer. It can be set to NULL, in which case the *text_size will provide the needed buffer size. The user should then allocate the required buffer size and call this function again to obtain the entire text. @param text_size A Pointer to the size of the text from the chunk, including the null terminator. If text is not NULL, *text_size should be greater than or equal to the text size; otherwise, the function will return OUT_OF_BOUNDS(-6). @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_decoded_results_create^⚠

@brief Create WhisperDecodedResults @param results A pointer to the newly created ov_genai_whisper_decoded_results. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_decoded_results_free^⚠

@brief Release the memory allocated by ov_genai_whisper_decoded_results. @param results A pointer to the ov_genai_whisper_decoded_results to free memory.

ov_genai_whisper_decoded_results_get_chunk_at^⚠

@brief Get chunk at specific index from ov_genai_whisper_decoded_results. @param results A pointer to the ov_genai_whisper_decoded_results instance. @param index The index of the chunk to retrieve. @param chunk A pointer to the newly created ov_genai_whisper_decoded_result_chunk. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_decoded_results_get_chunks_count^⚠

@brief Get number of chunks from ov_genai_whisper_decoded_results. @param results A pointer to the ov_genai_whisper_decoded_results instance. @param count A pointer to the number of chunks. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_decoded_results_get_perf_metrics^⚠

@brief Get performance metrics from ov_genai_whisper_decoded_results. @param results A pointer to the ov_genai_whisper_decoded_results instance. @param metrics A pointer to the newly created ov_genai_perf_metrics. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_decoded_results_get_score_at^⚠

@brief Get score at specific index from ov_genai_whisper_decoded_results. @param results A pointer to the ov_genai_whisper_decoded_results instance. @param index The index of the score to retrieve. @param score A pointer to the score value. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_decoded_results_get_string^⚠

@brief Get string representation from ov_genai_whisper_decoded_results. @param results A pointer to the ov_genai_whisper_decoded_results instance. @param output A pointer to the pre-allocated output string buffer. It can be set to NULL, in which case the *output_size will provide the needed buffer size. The user should then allocate the required buffer size and call this function again to obtain the entire output. @param output_size A Pointer to the size of the output string from the results, including the null terminator. If output is not NULL, *output_size should be greater than or equal to the result string size; otherwise, the function will return OUT_OF_BOUNDS(-6). @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_decoded_results_get_text_at^⚠

@brief Get text result at specific index from ov_genai_whisper_decoded_results. @param results A pointer to the ov_genai_whisper_decoded_results instance. @param index The index of the text result to retrieve. @param text A pointer to the pre-allocated text buffer. It can be set to NULL, in which case the *text_size will provide the needed buffer size. The user should then allocate the required buffer size and call this function again to obtain the entire text. @param text_size A Pointer to the size of the text from the results, including the null terminator. If text is not NULL, *text_size should be greater than or equal to the text size; otherwise, the function will return OUT_OF_BOUNDS(-6). @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_decoded_results_get_texts_count^⚠

@brief Get number of text results from ov_genai_whisper_decoded_results. @param results A pointer to the ov_genai_whisper_decoded_results instance. @param count A pointer to the number of text results. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_decoded_results_has_chunks^⚠

@brief Check if chunks are available from ov_genai_whisper_decoded_results. @param results A pointer to the ov_genai_whisper_decoded_results instance. @param has_chunks A pointer to the boolean indicating if chunks are available. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_create^⚠

@brief Create ov_genai_whisper_generation_config. @param config A pointer to the newly created ov_genai_whisper_generation_config. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_create_from_json^⚠

@brief Create ov_genai_whisper_generation_config from JSON file. @param json_path Path to a .json file containing the whisper generation configuration to load. @param config A pointer to the newly created ov_genai_whisper_generation_config. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_free^⚠

@brief Release the memory allocated by ov_genai_whisper_generation_config. @param config A pointer to the ov_genai_whisper_generation_config to free memory.

ov_genai_whisper_generation_config_get_begin_suppress_tokens^⚠

@brief Get the begin suppress tokens. @param config A pointer to the ov_genai_whisper_generation_config instance. @param tokens A pointer to the pre-allocated array of token ids. The array should be allocated with the size returned by ov_genai_whisper_generation_config_get_begin_suppress_tokens_count. @param tokens_count The size of the tokens array. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_get_begin_suppress_tokens_count^⚠

@brief Get the begin suppress tokens count. @param config A pointer to the ov_genai_whisper_generation_config instance. @param tokens_count A pointer to the number of begin suppress tokens. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_get_decoder_start_token_id^⚠

@brief Get the decoder start token id. @param config A pointer to the ov_genai_whisper_generation_config instance. @param token_id A pointer to the decoder start token id. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_get_generation_config^⚠

@brief Get the underlying GenerationConfig from ov_genai_whisper_generation_config. @param config A pointer to the ov_genai_whisper_generation_config instance. @param generation_config A pointer to the newly created ov_genai_generation_config. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_get_hotwords^⚠

@brief Get the hotwords for generation. @param config A pointer to the ov_genai_whisper_generation_config instance. @param hotwords A pointer to the pre-allocated hotwords buffer. It can be set to NULL, in which case the *hotwords_size will provide the needed buffer size. The user should then allocate the required buffer size and call this function again to obtain the entire hotwords. @param hotwords_size A Pointer to the size of the hotwords buffer, including the null terminator. If hotwords is not NULL, *hotwords_size should be greater than or equal to the hotwords size; otherwise, the function will return OUT_OF_BOUNDS(-6). @return ov_status_e A status code, return OK(0) if successful. NOT_FOUND(-5) if hotwords is not set.

ov_genai_whisper_generation_config_get_initial_prompt^⚠

@brief Get the initial prompt for generation. @param config A pointer to the ov_genai_whisper_generation_config instance. @param initial_prompt A pointer to the pre-allocated initial prompt buffer. It can be set to NULL, in which case the *prompt_size will provide the needed buffer size. The user should then allocate the required buffer size and call this function again to obtain the entire initial prompt. @param prompt_size A Pointer to the size of the initial prompt buffer, including the null terminator. If initial_prompt is not NULL, *prompt_size should be greater than or equal to the prompt size; otherwise, the function will return OUT_OF_BOUNDS(-6). @return ov_status_e A status code, return OK(0) if successful. NOT_FOUND(-5) if initial prompt is not set.

ov_genai_whisper_generation_config_get_is_multilingual^⚠

@brief Get whether the model is multilingual. @param config A pointer to the ov_genai_whisper_generation_config instance. @param is_multilingual A pointer to the multilingual flag. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_get_language^⚠

@brief Get the language for generation. @param config A pointer to the ov_genai_whisper_generation_config instance. @param language A pointer to the pre-allocated language buffer. It can be set to NULL, in which case the *language_size will provide the needed buffer size. The user should then allocate the required buffer size and call this function again to obtain the entire language. @param language_size A Pointer to the size of the language buffer, including the null terminator. If language is not NULL, *language_size should be greater than or equal to the language size; otherwise, the function will return OUT_OF_BOUNDS(-6). @return ov_status_e A status code, return OK(0) if successful. NOT_FOUND(-5) if language is not set.

ov_genai_whisper_generation_config_get_max_initial_timestamp_index^⚠

@brief Get the maximum initial timestamp index. @param config A pointer to the ov_genai_whisper_generation_config instance. @param index A pointer to the maximum initial timestamp index. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_get_no_timestamps_token_id^⚠

@brief Get the no timestamps token id. @param config A pointer to the ov_genai_whisper_generation_config instance. @param token_id A pointer to the no timestamps token id. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_get_pad_token_id^⚠

@brief Get the padding token id. @param config A pointer to the ov_genai_whisper_generation_config instance. @param token_id A pointer to the padding token id. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_get_prev_sot_token_id^⚠

@brief Get the previous start of transcript token id. @param config A pointer to the ov_genai_whisper_generation_config instance. @param token_id A pointer to the previous start of transcript token id. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_get_return_timestamps^⚠

@brief Get whether to return timestamps. @param config A pointer to the ov_genai_whisper_generation_config instance. @param return_timestamps A pointer to the return timestamps flag. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_get_suppress_tokens^⚠

@brief Get the suppress tokens. @param config A pointer to the ov_genai_whisper_generation_config instance. @param tokens A pointer to the pre-allocated array of token ids. The array should be allocated with the size returned by ov_genai_whisper_generation_config_get_suppress_tokens_count. @param tokens_count The size of the tokens array. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_get_suppress_tokens_count^⚠

@brief Get the suppress tokens count. @param config A pointer to the ov_genai_whisper_generation_config instance. @param tokens_count A pointer to the number of suppress tokens. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_get_task^⚠

@brief Get the task for generation. @param config A pointer to the ov_genai_whisper_generation_config instance. @param task A pointer to the pre-allocated task buffer. It can be set to NULL, in which case the *task_size will provide the needed buffer size. The user should then allocate the required buffer size and call this function again to obtain the entire task. @param task_size A Pointer to the size of the task buffer, including the null terminator. If task is not NULL, *task_size should be greater than or equal to the task size; otherwise, the function will return OUT_OF_BOUNDS(-6). @return ov_status_e A status code, return OK(0) if successful. NOT_FOUND(-5) if task is not set.

ov_genai_whisper_generation_config_get_transcribe_token_id^⚠

@brief Get the transcribe token id. @param config A pointer to the ov_genai_whisper_generation_config instance. @param token_id A pointer to the transcribe token id. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_get_translate_token_id^⚠

@brief Get the translate token id. @param config A pointer to the ov_genai_whisper_generation_config instance. @param token_id A pointer to the translate token id. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_set_begin_suppress_tokens^⚠

@brief Set the begin suppress tokens. @param config A pointer to the ov_genai_whisper_generation_config instance. @param tokens A pointer to the array of token ids to suppress at the beginning. @param tokens_count The number of tokens in the array. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_set_decoder_start_token_id^⚠

@brief Set the decoder start token id. @param config A pointer to the ov_genai_whisper_generation_config instance. @param token_id The decoder start token id (default: 50258). @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_set_hotwords^⚠

@brief Set the hotwords for generation. @param config A pointer to the ov_genai_whisper_generation_config instance. @param hotwords The hotwords text. Can be NULL to unset. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_set_initial_prompt^⚠

@brief Set the initial prompt for generation. @param config A pointer to the ov_genai_whisper_generation_config instance. @param initial_prompt The initial prompt text. Can be NULL to unset. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_set_is_multilingual^⚠

@brief Set whether the model is multilingual. @param config A pointer to the ov_genai_whisper_generation_config instance. @param is_multilingual True if the model is multilingual (default: true). @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_set_language^⚠

@brief Set the language for generation. @param config A pointer to the ov_genai_whisper_generation_config instance. @param language The language token (e.g., “en”, “fr”, “de”). Can be NULL to unset. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_set_max_initial_timestamp_index^⚠

@brief Set the maximum initial timestamp index. @param config A pointer to the ov_genai_whisper_generation_config instance. @param index The maximum initial timestamp index (default: 50). @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_set_no_timestamps_token_id^⚠

@brief Set the no timestamps token id. @param config A pointer to the ov_genai_whisper_generation_config instance. @param token_id The no timestamps token id (default: 50363). @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_set_pad_token_id^⚠

@brief Set the padding token id. @param config A pointer to the ov_genai_whisper_generation_config instance. @param token_id The padding token id (default: 50257). @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_set_prev_sot_token_id^⚠

@brief Set the previous start of transcript token id. @param config A pointer to the ov_genai_whisper_generation_config instance. @param token_id The previous start of transcript token id (default: 50361). @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_set_return_timestamps^⚠

@brief Set whether to return timestamps. @param config A pointer to the ov_genai_whisper_generation_config instance. @param return_timestamps True to return timestamps for segments (default: false). @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_set_suppress_tokens^⚠

@brief Set the suppress tokens. @param config A pointer to the ov_genai_whisper_generation_config instance. @param tokens A pointer to the array of token ids to suppress during generation. @param tokens_count The number of tokens in the array. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_set_task^⚠

@brief Set the task for generation. @param config A pointer to the ov_genai_whisper_generation_config instance. @param task The task (“translate” or “transcribe”). Can be NULL to unset. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_set_transcribe_token_id^⚠

@brief Set the transcribe token id. @param config A pointer to the ov_genai_whisper_generation_config instance. @param token_id The transcribe token id (default: 50359). @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_set_translate_token_id^⚠

@brief Set the translate token id. @param config A pointer to the ov_genai_whisper_generation_config instance. @param token_id The translate token id (default: 50358). @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_generation_config_validate^⚠

@brief Validate the whisper generation configuration. @param config A pointer to the ov_genai_whisper_generation_config instance. @return ov_status_e A status code, return OK(0) if successful.

ov_genai_whisper_pipeline_create^⚠

Create a Whisper pipeline (runtime-linking variant).

ov_genai_whisper_pipeline_free^⚠

@brief Release the memory allocated by ov_genai_whisper_pipeline. @param pipeline A pointer to the ov_genai_whisper_pipeline to free memory.

ov_genai_whisper_pipeline_generate^⚠

@brief Generate results by ov_genai_whisper_pipeline from raw speech input. @param pipeline A pointer to the ov_genai_whisper_pipeline instance. @param raw_speech A pointer to the raw speech input array (float values). @param raw_speech_size The size of the raw speech input array. @param config A pointer to the ov_genai_whisper_generation_config, the pointer can be NULL. @param results A pointer to the ov_genai_whisper_decoded_results, which retrieves the results of the generation. @return Status code of the operation: OK(0) for success.

ov_genai_whisper_pipeline_get_generation_config^⚠

@brief Get the WhisperGenerationConfig from ov_genai_whisper_pipeline. @param pipeline A pointer to the ov_genai_whisper_pipeline instance. @param config A pointer to the newly created ov_genai_whisper_generation_config. @return Status code of the operation: OK(0) for success.

ov_genai_whisper_pipeline_set_generation_config^⚠

@brief Set the WhisperGenerationConfig to ov_genai_whisper_pipeline. @param pipeline A pointer to the ov_genai_whisper_pipeline instance. @param config A pointer to the ov_genai_whisper_generation_config instance. @return Status code of the operation: OK(0) for success.

Type Aliases

ov_genai_chat_history

@struct ov_genai_chat_history @brief Opaque type for ChatHistory

ov_genai_decoded_results

@struct ov_genai_decoded_results @brief type define ov_genai_decoded_results from ov_genai_decoded_results_opaque

ov_genai_generation_config

@struct ov_genai_generation_config @brief type define ov_genai_generation_config from ov_genai_generation_config_opaque

ov_genai_json_container

@struct ov_genai_json_container @brief Opaque type for JsonContainer

ov_genai_llm_pipeline

@struct ov_genai_llm_pipeline @brief type define ov_genai_llm_pipeline from ov_genai_llm_pipeline_opaque @return ov_status_e A status code, return OK(0) if successful.

ov_genai_perf_metrics

@struct ov_genai_perf_metrics @brief type define ov_genai_perf_metrics from ov_genai_perf_metrics_opaque.

ov_genai_vlm_decoded_results

@struct ov_genai_vlm_decoded_results @brief type define ov_genai_vlm_decoded_results from ov_genai_vlm_decoded_results_opaque

ov_genai_vlm_pipeline

@struct ov_genai_vlm_pipeline @brief type define ov_genai_vlm_pipeline from ov_genai_vlm_pipeline_opaque

ov_genai_whisper_decoded_result_chunk

@struct ov_genai_whisper_decoded_result_chunk @brief type define ov_genai_whisper_decoded_result_chunk from ov_genai_whisper_decoded_result_chunk_opaque

ov_genai_whisper_decoded_results

@struct ov_genai_whisper_decoded_results @brief type define ov_genai_whisper_decoded_results from ov_genai_whisper_decoded_results_opaque

ov_genai_whisper_generation_config

@struct ov_genai_whisper_generation_config @brief type define ov_genai_whisper_generation_config from ov_genai_whisper_generation_config_opaque

ov_genai_whisper_pipeline

@struct ov_genai_whisper_pipeline @brief type define ov_genai_whisper_pipeline from ov_genai_whisper_pipeline_opaque

ov_tensor_t

@struct ov_tensor_t @ingroup ov_tensor_c_api @brief type define ov_tensor_t from ov_tensor