Skip to main content

upload_decode_token_data

fuel_tensor::persistent_decode

Function upload_decode_token_data 

Source 
pub fn upload_decode_token_data(
    device: &Device,
    host: &DecodeTokenHost,
    cache_dtype: DType,
    device_offset: Option<usize>,
) -> Result<DecodeTokenData>
Expand description

Upload one step’s host data to device-resident Arcs — the same upload path KvCache::with_capacity uses. On CPU the Storage wraps the host bytes; on GPU it performs the (tiny) H2D copy.