Skip to main content

add_rpc_device

llama_cpp_4::rpc::server

Function add_rpc_device 

Source 
pub fn add_rpc_device(
    endpoint: &str,
) -> Result<NonNull<ggml_backend_device>, RpcError>
Expand description

Add a new RPC device

This function registers a new RPC device that can be used for inference.

§Arguments

  • endpoint - The RPC server endpoint to connect to

§Returns

The device handle if successful