Function serve 

pub fn serve(
    endpoint: &str,
    cache_dir: Option<&Path>,
    n_threads: usize,
    devices: &[NonNull<ggml_backend_device>],
) -> Result<(), RpcError>

Serve one or more local devices over RPC.

This call blocks. ggml_backend_rpc_start_server runs the accept loop on the calling thread and does not return while the server is live, so run it on a dedicated thread if the caller needs to stay responsive.

Arguments

• endpoint - Address to listen on (e.g. "0.0.0.0:50052")
• cache_dir - Directory for the server-side tensor cache, or None to disable caching
• n_threads - Worker threads used to service requests
• devices - Local devices to expose; clients address them by index, in the order given here

Errors

Returns RpcError::StringConversion if endpoint or cache_dir contains an interior NUL, RpcError::InvalidEndpoint if cache_dir is not valid UTF-8, and RpcError::ServerError if devices is empty or exceeds llama.cpp’s server limit.

Example

use llama_cpp_4::rpc::serve;

// `devices` comes from the ggml backend registry.
serve("0.0.0.0:50052", None, 4, &devices)?;