Expand description
Mock LLM scheduler and KV manager for testing.
This crate provides a mock implementation of an LLM scheduler that simulates KV cache management, request scheduling, and token generation timing without requiring actual GPU resources or a full distributed runtime.
Modules§
- cache
- Cache data structures for KV block management.
- common
- Shared components used across all engine implementations.
- engine
- Engine factory — creates the appropriate scheduler based on
EngineType. - kv_
manager - Pluggable KV cache block managers.
- live
- Reusable live-request boundary for the Mocker schedulers.
- loadgen
- replay
- scheduler
- Engine-specific scheduling implementations.
- services
- Runtime services used by the mocker.