Crate rkllm_rs Copy item path Source prelude RKLLMEmbedInput @struct RKLLMEmbedInput
@brief Represents an embedding input to the LLM. RKLLMExtendParam @struct RKLLMExtendParam
@brief The extend parameters for configuring an LLM instance. RKLLMInferParam @struct RKLLMInferParam
@brief Structure for defining parameters during inference. RKLLMInput @struct RKLLMInput
@brief Represents different types of input to the LLM via a union. RKLLMLoraAdapter @struct RKLLMLoraAdapter
@brief Defines parameters for a Lora adapter used in model fine-tuning. RKLLMLoraParam @struct RKLLMLoraParam
@brief Structure defining parameters for Lora adapters. RKLLMMultiModelInput @struct RKLLMMultiModelInput
@brief Represents multimodal input (e.g., text and image). RKLLMParam @struct RKLLMParam
@brief Defines the parameters for configuring an LLM instance. RKLLMPromptCacheParam @struct RKLLMPromptCacheParam
@brief Structure to define parameters for caching prompts. RKLLMResult @struct RKLLMResult
@brief Structure to represent the result of LLM inference. RKLLMResultLastHiddenLayer @struct RKLLMResultLastHiddenLayer
@brief Structure to hold the hidden states from the last layer. RKLLMResultLogits @struct RKLLMResultLogits
@brief Structure to hold the logits. RKLLMTokenInput @struct RKLLMTokenInput
@brief Represents token input to the LLM. __fsid_t max_align_t CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7 INT8_MAX INT8_MIN INT16_MAX INT16_MIN INT32_MAX INT32_MIN INTPTR_MAX INTPTR_MIN INT_FAST8_MAX INT_FAST8_MIN INT_FAST16_MAX INT_FAST16_MIN INT_FAST32_MAX INT_FAST32_MIN INT_LEAST8_MAX INT_LEAST8_MIN INT_LEAST16_MAX INT_LEAST16_MIN INT_LEAST32_MAX INT_LEAST32_MIN LLMCallState_RKLLM_RUN_ERROR < An error occurred during the LLM call. LLMCallState_RKLLM_RUN_FINISH < The LLM call has finished execution. LLMCallState_RKLLM_RUN_NORMAL < The LLM call is in a normal running state. LLMCallState_RKLLM_RUN_WAITING < The LLM call is waiting for complete UTF-8 encoded character. PTRDIFF_MAX PTRDIFF_MIN RKLLMInferMode_RKLLM_INFER_GENERATE < The LLM generates text based on input. RKLLMInferMode_RKLLM_INFER_GET_LAST_HIDDEN_LAYER < The LLM retrieves the last hidden layer for further processing. RKLLMInferMode_RKLLM_INFER_GET_LOGITS < The LLM retrieves logits for further processing. RKLLMInputType_RKLLM_INPUT_EMBED < Input is an embedding vector. RKLLMInputType_RKLLM_INPUT_MULTIMODAL < Input is multimodal (e.g., text and image). RKLLMInputType_RKLLM_INPUT_PROMPT < Input is a text prompt. RKLLMInputType_RKLLM_INPUT_TOKEN < Input is a sequence of tokens. SIG_ATOMIC_MAX SIG_ATOMIC_MIN SIZE_MAX UINT8_MAX UINT16_MAX UINT32_MAX UINTPTR_MAX UINT_FAST8_MAX UINT_FAST16_MAX UINT_FAST32_MAX UINT_LEAST8_MAX UINT_LEAST16_MAX UINT_LEAST32_MAX WINT_MAX WINT_MIN _ATFILE_SOURCE _BITS_STDINT_INTN_H _BITS_STDINT_LEAST_H _BITS_STDINT_UINTN_H _BITS_TIME64_H _BITS_TYPESIZES_H _BITS_TYPES_H _BITS_WCHAR_H _DEFAULT_SOURCE _FEATURES_H _POSIX_C_SOURCE _POSIX_SOURCE _STDC_PREDEF_H _STDINT_H _SYS_CDEFS_H __FD_SETSIZE __GLIBC_MINOR__ __GLIBC_USE_C2X_STRTOL __GLIBC_USE_DEPRECATED_GETS __GLIBC_USE_DEPRECATED_SCANF __GLIBC_USE_IEC_60559_BFP_EXT __GLIBC_USE_IEC_60559_BFP_EXT_C2X __GLIBC_USE_IEC_60559_EXT __GLIBC_USE_IEC_60559_FUNCS_EXT __GLIBC_USE_IEC_60559_FUNCS_EXT_C2X __GLIBC_USE_IEC_60559_TYPES_EXT __GLIBC_USE_ISOC2X __GLIBC_USE_LIB_EXT2 __GLIBC__ __GNU_LIBRARY__ __HAVE_GENERIC_SELECTION __KERNEL_OLD_TIMEVAL_MATCHES_TIMEVAL64 __LDOUBLE_REDIRECTS_TO_FLOAT128_ABI __RLIM_T_MATCHES_RLIM64_T __STATFS_MATCHES_STATFS64 __STDC_IEC_559_COMPLEX__ __STDC_IEC_559__ __STDC_IEC_60559_BFP__ __STDC_IEC_60559_COMPLEX__ __STDC_ISO_10646__ __TIMESIZE __USE_ATFILE __USE_FORTIFY_LEVEL __USE_ISOC11 __USE_ISOC95 __USE_ISOC99 __USE_MISC __USE_POSIX __USE_POSIX2 __USE_POSIX199309 __USE_POSIX199506 __USE_POSIX_IMPLICITLY __USE_XOPEN2K __USE_XOPEN2K8 __WORDSIZE __WORDSIZE32_PTRDIFF_LONG __WORDSIZE32_SIZE_ULONG __WORDSIZE_TIME64_COMPAT32 __bool_true_false_are_defined __glibc_c99_flexarr_available false_ true_ rkllm_abort ⚠ @brief Aborts an ongoing LLM task.
@param handle LLM handle.
@return Status code (0 for success, non-zero for failure). rkllm_clear_kv_cache ⚠ @brief Clear the key-value cache for a given LLM handle.
@param handle LLM handle.
@param keep_system_prompt Flag indicating whether to retain the system prompt in the cache (1 to retain, 0 to clear).
@return Status code (0 if cache was cleared successfully, non-zero for otherwise). rkllm_createDefaultParam ⚠ @brief Creates a default RKLLMParam structure with preset values.
@return A default RKLLMParam structure. rkllm_destroy ⚠ @brief Destroys the LLM instance and releases resources.
@param handle LLM handle.
@return Status code (0 for success, non-zero for failure). rkllm_init ⚠ @brief Initializes the LLM with the given parameters.
@param handle Pointer to the LLM handle.
@param param Configuration parameters for the LLM.
@param callback Callback function to handle LLM results.
@return Status code (0 for success, non-zero for failure). rkllm_is_running ⚠ @brief Checks if an LLM task is currently running.
@param handle LLM handle.
@return Status code (0 if a task is running, non-zero for otherwise). rkllm_load_lora ⚠ @brief Loads a Lora adapter into the LLM.
@param handle LLM handle.
@param lora_adapter Pointer to the Lora adapter structure.
@return Status code (0 for success, non-zero for failure). rkllm_load_prompt_cache ⚠ @brief Loads a prompt cache from a file.
@param handle LLM handle.
@param prompt_cache_path Path to the prompt cache file.
@return Status code (0 for success, non-zero for failure). rkllm_release_prompt_cache ⚠ @brief Releases the prompt cache from memory.
@param handle LLM handle.
@return Status code (0 for success, non-zero for failure). rkllm_run ⚠ @brief Runs an LLM inference task synchronously.
@param handle LLM handle.
@param rkllm_input Input data for the LLM.
@param rkllm_infer_params Parameters for the inference task.
@param userdata Pointer to user data for the callback.
@return Status code (0 for success, non-zero for failure). rkllm_run_async ⚠ @brief Runs an LLM inference task asynchronously.
@param handle LLM handle.
@param rkllm_input Input data for the LLM.
@param rkllm_infer_params Parameters for the inference task.
@param userdata Pointer to user data for the callback.
@return Status code (0 for success, non-zero for failure). rkllm_set_chat_template ⚠ @brief Sets the chat template for the LLM, including system prompt, prefix, and postfix. LLMCallState @enum LLMCallState
@brief Describes the possible states of an LLM call. LLMHandle @typedef LLMHandle
@brief A handle used to manage and interact with the large language model. LLMResultCallback @typedef LLMResultCallback
@brief Callback function to handle LLM results.
@param result Pointer to the LLM result.
@param userdata Pointer to user data for the callback.
@param state State of the LLM call (e.g., finished, error). RKLLMInferMode @enum RKLLMInferMode
@brief Specifies the inference modes of the LLM. RKLLMInputType @enum RKLLMInputType
@brief Defines the types of inputs that can be fed into the LLM. __blkcnt64_t __blkcnt_t __blksize_t __caddr_t __clock_t __clockid_t __daddr_t __dev_t __fsblkcnt64_t __fsblkcnt_t __fsfilcnt64_t __fsfilcnt_t __fsword_t __gid_t __id_t __ino64_t __ino_t __int8_t __int16_t __int32_t __int64_t __int_least8_t __int_least16_t __int_least32_t __int_least64_t __intmax_t __intptr_t __key_t __loff_t __mode_t __nlink_t __off64_t __off_t __pid_t __quad_t __rlim64_t __rlim_t __sig_atomic_t __socklen_t __ssize_t __suseconds64_t __suseconds_t __syscall_slong_t __syscall_ulong_t __time64_t __time_t __timer_t __u_char __u_int __u_long __u_quad_t __u_short __uid_t __uint8_t __uint16_t __uint32_t __uint64_t __uint_least8_t __uint_least16_t __uint_least32_t __uint_least64_t __uintmax_t __useconds_t int_fast8_t int_fast16_t int_fast32_t int_fast64_t int_least8_t int_least16_t int_least32_t int_least64_t intmax_t uint_fast8_t uint_fast16_t uint_fast32_t uint_fast64_t uint_least8_t uint_least16_t uint_least32_t uint_least64_t uintmax_t wchar_t RKLLMInput__bindgen_ty_1