Skip to main content

Crate tpt_gpu_runtime

Crate tpt_gpu_runtime 

Source
Expand description

§TPT Runtime (tpt-gpu-runtime)

Core library for the TPT GPU compute runtime. Provides GPU memory management, command queue / scheduler, kernel launch interface, and device abstraction.

Re-exports§

pub use error::ErrorCode;
pub use error::ErrorContext;
pub use error::TptrError;
pub use error::TptrResult;
pub use device::Device;
pub use device::DeviceHandle;
pub use device::DeviceInfo;
pub use device::DeviceProperties;
pub use memory::MemAccess;
pub use memory::MemType;
pub use memory::MemoryAllocation;
pub use memory::MemoryRegion;
pub use command::Command;
pub use command::CommandQueue;
pub use command::CommandScheduler;
pub use command::QueueHandle;
pub use command::QueuePriority;
pub use kernel::Dim3;
pub use kernel::Kernel;
pub use kernel::KernelConfig;
pub use kernel::KernelHandle;
pub use kernel::KernelLaunchMode;
pub use inference::parse_gguf_header;
pub use inference::GpuInferenceEngine;
pub use inference::LlmInference;
pub use inference::ModelInfo;
pub use arch::template_for_arch;
pub use arch::ArchTemplate;
pub use arch::ForwardOp;
pub use kv_cache::KvCache;
pub use rope::apply_rope;
pub use rope::RopeConfig;

Modules§

arch
Arch-template dispatch layer.
command
device
error
Error Handling Framework - Structured error types for the TPT runtime.
inference
LLM inference API for the TPT runtime.
kernel
kv_cache
Sliding-window KV cache for autoregressive decoding.
memory
rope
Rotary Position Embedding (RoPE).

Macros§

tptr_err