pub fn gpu_active() -> boolExpand description
Whether a real GPU execution provider will actually run inference — i.e. the policy wants a GPU and the provider’s runtime libraries load. Used to scale batch size: small mini-batches under-utilize a GPU and pay kernel-launch/host↔device-copy overhead per call that isn’t amortized (notably under WSL2 GPU passthrough), but oversizing batches for a GPU that silently fell back to CPU makes the CPU path dramatically slower — so this must reflect the EP that ORT will really register, not just the policy.