Module kv_cache_copy 

KV cache GPU copy dispatch.

Copies new K or V data directly from a source GPU buffer into a pre-allocated KV cache buffer at the correct write position, with optional modulo wrapping for sliding window (ring buffer) caches.

This eliminates the CPU round-trip that append_bf16 requires: instead of GPU -> CPU (as_slice) -> CPU (copy loop) -> shared buffer, the GPU copies directly between two shared Metal buffers.

Statics

KV_CACHE_COPY_SHADER_SOURCE

MSL source for the KV cache copy kernel (embedded at compile time).

Functions

dispatch_kv_cache_copy

Dispatch a GPU copy from a source bf16 buffer into a KV cache buffer.

dispatch_kv_cache_copy_batch_f32

Dispatch a batched GPU copy from a source f32 buffer into a f32 KV cache.

dispatch_kv_cache_copy_batch_f32_kv_dual

Fused single-position K + V cache copy (F32 source → F32 cache) — DECODE shape.

dispatch_kv_cache_copy_batch_f32_to_f16

Dispatch a batched F32→F16 copy from a source f32 buffer into an f16 KV cache.

dispatch_kv_cache_copy_batch_f32_to_f16_batched

ADR-040 M4 — BATCHED multi-sequence F16-K copy: writes all n_queries decode queries’ K into their own physical-slot regions of the shared multi_seq F16 cache in ONE dispatch (grid.z = N), replacing the per-slot host-side loop. src is [N, n_heads*head_dim] F32; cache is the FULL multi_seq buffer [n_seqs, n_heads, capacity, head_dim] F16; slot_id/ seq_pos are [N] u32. Byte-identical to N single-slot calls.

dispatch_kv_cache_copy_batch_f32_to_f16_kv_dual

Fused single-position K + V cache copy (F32 source → F16 cache) — DECODE shape.

dispatch_kv_cache_copy_f32

Dispatch a GPU copy from a source f32 buffer into a f32 KV cache buffer.

dispatch_kv_cache_copy_seq_bf16

Multi-position, all-heads KV cache copy (BF16 source → F32 cache, batched prefill).

dispatch_kv_cache_copy_seq_bf16_to_bf16_head_major

ADR-030 iter-95: bit-exact BF16→BF16 strided cache copy from pf_k_perm (head-major BF16) to bf16_xlen_cache (head-major BF16).

dispatch_kv_cache_copy_seq_f32

Multi-position, all-heads KV cache copy (F32 → F32 cache, batched prefill).

dispatch_kv_cache_copy_seq_f32_dual

Fused K + V cache copy (F32 source → F32 cache). Wave P4.11.

dispatch_kv_cache_copy_seq_f32_to_f16

Multi-position, all-heads KV cache copy (F32 source → F16 cache, batched prefill).

dispatch_kv_cache_copy_seq_f32_to_f16_dual

Fused K + V cache copy (F32 source → F16 cache). Wave P4.11 f16-cache variant of dispatch_kv_cache_copy_seq_f32_dual.

register

Register KV cache copy shader source with the given kernel registry.