Functionsยง
- compute_
kv_ transfer_ delay - Compute the KV transfer delay duration for a given number of input tokens.
- compute_
prefill_ handoff_ delay_ ms - Compute the modeled handoff delay after a prefill worker emits its terminal token.
- prefill_
handoff_ transfer_ timing - sleep_
precise - Sleep for the specified duration using timerfd on Linux for precision.
- sleep_
until_ precise - Sleep until the specified deadline using timerfd on Linux for precision.