Expand description
§TPT Runtime (tpt-gpu-runtime)
Core library for the TPT GPU compute runtime. Provides GPU memory management, command queue / scheduler, kernel launch interface, and device abstraction.
Re-exports§
pub use error::ErrorCode;pub use error::ErrorContext;pub use error::TptrError;pub use error::TptrResult;pub use device::Device;pub use device::DeviceHandle;pub use device::DeviceInfo;pub use device::DeviceProperties;pub use memory::MemAccess;pub use memory::MemType;pub use memory::MemoryAllocation;pub use memory::MemoryRegion;pub use command::Command;pub use command::CommandQueue;pub use command::CommandScheduler;pub use command::QueueHandle;pub use command::QueuePriority;pub use kernel::Dim3;pub use kernel::Kernel;pub use kernel::KernelConfig;pub use kernel::KernelHandle;pub use kernel::KernelLaunchMode;pub use inference::parse_gguf_header;pub use inference::GpuInferenceEngine;pub use inference::LlmInference;pub use inference::ModelInfo;pub use arch::template_for_arch;pub use arch::ArchTemplate;pub use arch::ForwardOp;pub use kv_cache::KvCache;pub use rope::apply_rope;pub use rope::RopeConfig;
Modules§
- arch
- Arch-template dispatch layer.
- command
- device
- error
- Error Handling Framework - Structured error types for the TPT runtime.
- inference
- LLM inference API for the TPT runtime.
- kernel
- kv_
cache - Sliding-window KV cache for autoregressive decoding.
- memory
- rope
- Rotary Position Embedding (RoPE).