Skip to main content

set_param_sharing_scope

Function set_param_sharing_scope 

Source
pub fn set_param_sharing_scope(label: &str)
Expand description

Declare which compiles belong to the same model, for backends that can share parameter storage between executables.

A model that compiles several graphs (a prefill graph plus decode buckets) otherwise re-uploads every weight per graph. Sharing that storage means skipping the re-upload, which is only sound when the same parameter name and size really is the same tensor — and nothing in a graph establishes that, since parameter content arrives later via set_param. So the caller declares it: pass a stable label per model (a weights path works well).

No-op unless a backend that supports sharing is compiled in. Today that is CUDA, additionally gated behind RLX_CUDA_SHARED_PARAMS=1.