Skip to main content

Crate vkfft_rs

Crate vkfft_rs 

Source
Expand description

Pure-Rust reimplementation of VkFFT.

vkfft-rs ports VkFFT’s Application -> Plan -> Code architecture into typed Rust state: configuration and device-aware planning, Stockham/Rader/Bluestein and real/R2R composition, backend-neutral program/resource IR, generated GPU kernels, and correctness-first execution runtimes. The fixed porting baseline is VkFFT 1.3.4 at upstream commit 066a17c17068c0f11c9298d848c2976c71fad1c1.

§Default build

The default feature set has no GPU runtime loader. It provides the planner, CPU reference paths, typed IR, scheduler policies, LUT generation, and backend source/SPIR-V generation. This makes planning and code generation usable without a CUDA, HIP, OpenCL, Level Zero, Metal, or Vulkan runtime installed on the host.

§Runtime features

  • vulkan-runtime: Vulkan 1.1 execution through ash, including explicit device selection, async tickets, pooled resources, resident program chaining, and versioned pipeline caches.
  • cuda-runtime: CUDA Driver + NVRTC execution loaded dynamically at runtime.
  • hip-runtime: HIP + HIPRTC/AMD compiler execution loaded dynamically at runtime.
  • opencl-runtime: OpenCL execution loaded dynamically at runtime.
  • level-zero-runtime: Intel Level Zero execution loaded dynamically at runtime. It can be combined with opencl-runtime to use the validated Intel native-binary compiler fallback when ocloc is unavailable.
  • metal-runtime: correctness-first Metal F32/F16 execution on macOS; the runtime surface is implemented, while real Apple-GPU validation remains an explicit hardware gate.

§Planning example

use vkfft_rs::{AlgorithmKind, FftConfig, FftPlan};

let plan = FftPlan::build(FftConfig::new(vec![1024, 17, 65537]))?;
assert_eq!(plan.axes[0].algorithm.kind(), AlgorithmKind::Stockham);
assert_eq!(plan.axes[1].algorithm.kind(), AlgorithmKind::Rader);
assert_eq!(plan.axes[2].algorithm.kind(), AlgorithmKind::Bluestein);

Re-exports§

pub use application::TransformIr;
pub use binary16::Binary16;
pub use binary16::Complex16;
pub use binary16::decode_complex16_native;
pub use binary16::encode_complex16_native;
pub use bluestein_ir::BluesteinPassIr;
pub use bluestein_ir::BluesteinPassOperation;
pub use bluestein_ir::BluesteinPipelineIr;
pub use bluestein_ir::execute_bluestein_ir;
pub use complex::Complex;
pub use complex::Complex32;
pub use complex::Complex64;
pub use config::Backend;
pub use config::ConvolutionConjugation;
pub use config::DctType;
pub use config::DeviceProfile;
pub use config::Direction;
pub use config::DstType;
pub use config::FftConfig;
pub use config::GpuVendor;
pub use config::PLANNER_TUNING_PROFILE_VERSION;
pub use config::PlannerTuning;
pub use config::Precision;
pub use config::PrecisionCompute;
pub use config::PrecisionLayout;
pub use config::PrecisionStorage;
pub use config::SubgroupProfile;
pub use config::TransformKind;
pub use config::ZeroPaddingDomain;
pub use config::ZeroPaddingRange;
pub use convolution_ir::ConvolutionDirectRaderMultiKernelStepIr;
pub use convolution_ir::ConvolutionFftRaderMultiKernelStepIr;
pub use convolution_ir::ConvolutionFftRaderStepIr;
pub use convolution_ir::ConvolutionIr;
pub use convolution_ir::ConvolutionMatrixLayout;
pub use convolution_ir::ConvolutionMatrixStockhamStepIr;
pub use convolution_ir::ConvolutionMultiKernelStockhamStepIr;
pub use convolution_ir::ConvolutionMultiplyIr;
pub use convolution_ir::ConvolutionMultiplyPolicy;
pub use convolution_ir::ConvolutionThreeUploadMultiKernelStockhamIr;
pub use convolution_ir::ConvolutionTwoUploadMultiKernelStockhamIr;
pub use convolution_ir::NdConvolutionIr;
pub use convolution_ir::NdRealConvolutionIr;
pub use convolution_ir::execute_convolution_ir;
pub use convolution_ir::execute_nd_convolution_ir;
pub use convolution_ir::execute_nd_real_convolution_ir;
pub use double_double::ComplexDoubleDouble;
pub use double_double::DoubleDouble;
pub use double_double::decode_complex_double_double;
pub use double_double::dft as double_double_dft;
pub use double_double::encode_complex_double_double;
pub use double_double::quick_two_sum;
pub use double_double::two_prod;
pub use double_double::two_sum;
pub use double_double::unit_root as double_double_unit_root;
pub use double_double_ir::DoubleDoubleBluesteinConvolutionIr;
pub use double_double_ir::DoubleDoubleBluesteinIr;
pub use double_double_ir::DoubleDoubleDirectRaderIr;
pub use double_double_ir::DoubleDoubleFftRaderIr;
pub use double_double_ir::DoubleDoubleNdAxisIr;
pub use double_double_ir::DoubleDoubleNdFftIr;
pub use double_double_ir::DoubleDoubleNdR2rAxisIr;
pub use double_double_ir::DoubleDoubleNdR2rIr;
pub use double_double_ir::DoubleDoubleNdRealFftIr;
pub use double_double_ir::DoubleDoubleOneDimIr;
pub use double_double_ir::DoubleDoubleR2rAlgorithm;
pub use double_double_ir::DoubleDoubleR2rIr;
pub use double_double_ir::DoubleDoubleRealFftIr;
pub use double_double_ir::DoubleDoubleStockhamIr;
pub use double_double_ir::execute_double_double_bluestein_ir;
pub use double_double_ir::execute_double_double_bluestein_ir_f64_storage;
pub use double_double_ir::execute_double_double_c2r_ir;
pub use double_double_ir::execute_double_double_c2r_ir_f64_storage;
pub use double_double_ir::execute_double_double_direct_rader_ir;
pub use double_double_ir::execute_double_double_direct_rader_ir_f64_storage;
pub use double_double_ir::execute_double_double_fft_rader_ir;
pub use double_double_ir::execute_double_double_fft_rader_ir_f64_storage;
pub use double_double_ir::execute_double_double_nd_c2r_ir;
pub use double_double_ir::execute_double_double_nd_c2r_ir_f64_storage;
pub use double_double_ir::execute_double_double_nd_ir;
pub use double_double_ir::execute_double_double_nd_ir_f64_storage;
pub use double_double_ir::execute_double_double_nd_r2c_ir;
pub use double_double_ir::execute_double_double_nd_r2c_ir_f64_storage;
pub use double_double_ir::execute_double_double_nd_r2r_ir;
pub use double_double_ir::execute_double_double_nd_r2r_ir_f64_storage;
pub use double_double_ir::execute_double_double_one_dim_ir;
pub use double_double_ir::execute_double_double_one_dim_ir_f64_storage;
pub use double_double_ir::execute_double_double_r2c_ir;
pub use double_double_ir::execute_double_double_r2c_ir_f64_storage;
pub use double_double_ir::execute_double_double_r2r_ir;
pub use double_double_ir::execute_double_double_r2r_ir_f64_storage;
pub use double_double_ir::execute_double_double_stockham_ir;
pub use double_double_ir::execute_double_double_stockham_ir_f64_storage;
pub use double_double_recursive_ir::DoubleDoubleCooleyTukeyPassIr;
pub use double_double_recursive_ir::DoubleDoubleCooleyTukeyPassOperation;
pub use double_double_recursive_ir::DoubleDoubleRecursiveCooleyTukeyIr;
pub use double_double_recursive_ir::DoubleDoubleRecursiveFftIr;
pub use double_double_recursive_ir::DoubleDoubleRecursiveFftNodeIr;
pub use double_double_recursive_ir::DoubleDoubleThreeUploadFourStepPlanIr;
pub use double_double_recursive_ir::DoubleDoubleTwoUploadFourStepPlanIr;
pub use double_double_recursive_ir::execute_double_double_recursive_ir;
pub use double_double_recursive_ir::execute_double_double_recursive_ir_f64_storage;
pub use error::Result;
pub use error::VkFftError;
pub use kernel_ir::BufferAccess;
pub use kernel_ir::BufferBinding;
pub use kernel_ir::BufferRole;
pub use kernel_ir::DispatchGeometry;
pub use kernel_ir::FourStepMapping;
pub use kernel_ir::KernelIr;
pub use kernel_ir::KernelOperation;
pub use kernel_ir::RaderGeneratorMapping;
pub use kernel_ir::RaderScatterMapping;
pub use kernel_ir::RealEvenInversePreprocessMapping;
pub use kernel_ir::RealEvenPackMapping;
pub use kernel_ir::RealEvenUnpackMapping;
pub use kernel_ir::RegisterStageBoundary;
pub use kernel_ir::RegisterStageBoundaryResidency;
pub use kernel_ir::RegisterStockhamStage;
pub use kernel_ir::RegisterSubgroupBoundary;
pub use kernel_ir::RegisterSubgroupLaneModel;
pub use kernel_ir::RegisterSubgroupSource;
pub use kernel_ir::ScalarType;
pub use kernel_ir::SharedBuffer;
pub use kernel_ir::SharedMemoryPlan;
pub use kernel_ir::StockhamExecutionLayout;
pub use kernel_ir::StockhamInputModifier;
pub use kernel_ir::StockhamIoMapping;
pub use kernel_ir::StockhamOutputModifier;
pub use kernel_ir::StockhamSharedMemoryLayout;
pub use kernel_ir::StockhamStage;
pub use kernel_ir::StockhamWorkgroupGrouping;
pub use kernel_ir::ThreeUploadFourStepMapping;
pub use kernel_ir::WorkgroupSize;
pub use kernel_ir::execute_stockham_ir;
pub use lut::BluesteinTable;
pub use lut::DoubleDoubleBluesteinTable;
pub use lut::DoubleDoubleRaderTable;
pub use lut::DoubleDoubleStockhamTwiddleStage;
pub use lut::DoubleDoubleStockhamTwiddleTable;
pub use lut::RaderTable;
pub use lut::StockhamTwiddleStage;
pub use lut::StockhamTwiddleTable;
pub use lut::stockham_root_table_double_double;
pub use mixed_ir::MixedPassIr;
pub use mixed_ir::MixedPassOperation;
pub use mixed_ir::MixedPrimeRaderIr;
pub use mixed_ir::MixedRaderStockhamIr;
pub use mixed_ir::execute_mixed_rader_stockham_ir;
pub use nd_ir::NdAxisIr;
pub use nd_ir::NdExternalTensorLayout;
pub use nd_ir::NdFftIr;
pub use nd_ir::NdFormattedCopyOperation;
pub use nd_ir::NdFormattedCopyPassIr;
pub use nd_ir::NdPassIr;
pub use nd_ir::NdPassOperation;
pub use nd_ir::execute_nd_fft_ir;
pub use nd_real_ir::NdRealFftIr;
pub use nd_real_ir::execute_nd_c2r_ir;
pub use nd_real_ir::execute_nd_r2c_ir;
pub use one_dim_ir::OneDimFftIr;
pub use one_dim_ir::execute_one_dim_fft_ir;
pub use planner::AlgorithmKind;
pub use planner::AxisAlgorithm;
pub use planner::AxisPlan;
pub use planner::FftPlan;
pub use planner::RaderMode;
pub use planner::RaderPrimePlan;
pub use planner::RadixPlan;
pub use precision::PRECISION_REPORT_SCHEMA_VERSION;
pub use precision::PrecisionCaseReport;
pub use precision::PrecisionMetrics;
pub use precision::PrecisionTransformFamily;
pub use precision::complex_precision_metrics;
pub use precision::real_precision_metrics;
pub use program_ir::ExternalBufferLayout;
pub use program_ir::ProgramAllocation;
pub use program_ir::ProgramAllocationId;
pub use program_ir::ProgramAllocationKind;
pub use program_ir::ProgramIr;
pub use program_ir::ProgramMemoryPlan;
pub use program_ir::ProgramPass;
pub use program_ir::ProgramPassBinding;
pub use program_ir::ProgramResource;
pub use program_ir::ProgramResourceId;
pub use program_ir::ProgramResourceInitialization;
pub use program_ir::ProgramResourceKind;
pub use r2r_ir::NdR2rAxisIr;
pub use r2r_ir::NdR2rIr;
pub use r2r_ir::R2rFftPassIr;
pub use r2r_ir::R2rFftPassOperation;
pub use r2r_ir::R2rFftReductionIr;
pub use r2r_ir::R2rIr;
pub use r2r_ir::R2rNdPassIr;
pub use r2r_ir::R2rNdPassOperation;
pub use r2r_ir::R2rTransform;
pub use r2r_ir::execute_nd_r2r_ir;
pub use r2r_ir::execute_r2r_ir;
pub use rader_ir::RaderDirectIr;
pub use rader_ir::RaderFftInputStrategy;
pub use rader_ir::RaderFftPassIr;
pub use rader_ir::RaderFftPassOperation;
pub use rader_ir::RaderFftPipelineIr;
pub use rader_ir::execute_rader_direct_ir;
pub use rader_ir::execute_rader_fft_ir;
pub use real_ir::RealFftAlgorithm;
pub use real_ir::RealFftIr;
pub use real_ir::RealFftKind;
pub use real_ir::RealPassIr;
pub use real_ir::RealPassOperation;
pub use real_ir::execute_c2r_ir;
pub use real_ir::execute_r2c_ir;
pub use recursive_ir::CooleyTukeyPassIr;
pub use recursive_ir::CooleyTukeyPassOperation;
pub use recursive_ir::FourStepInputLayout;
pub use recursive_ir::FourStepOutputLayout;
pub use recursive_ir::FourStepPlanIr;
pub use recursive_ir::FourStepTwiddlePlacement;
pub use recursive_ir::FourStepUploadIr;
pub use recursive_ir::FusedFftRaderStaticResourceReport;
pub use recursive_ir::RecursiveCooleyTukeyIr;
pub use recursive_ir::RecursiveFftIr;
pub use recursive_ir::RecursiveFftNodeIr;
pub use recursive_ir::execute_recursive_fft_ir;
pub use scheduler::DoubleDoubleStockhamUploadSchedule;
pub use scheduler::GpuSchedulerPolicy;
pub use scheduler::NvidiaVulkanSchedulerTuning;
pub use scheduler::RaderFftRegisterSchedule;
pub use scheduler::RaderFftStageLaneLayout;
pub use scheduler::RaderFftStageLaneSchedule;
pub use scheduler::RaderFftTransposeSchedule;
pub use scheduler::RadixRegisterSchedule;
pub use scheduler::StockhamUploadSchedule;
pub use scheduler::VKFFT_RADIX_TABLE_LEN;
pub use scheduler::has_specialized_gpu_scheduler_policy;
pub use scheduler::plan_gpu_double_double_stockham_uploads_for_batches;
pub use scheduler::plan_gpu_force_rader_two_upload;
pub use scheduler::plan_gpu_power_of_two_radix_registers;
pub use scheduler::plan_gpu_power_of_two_stockham_uploads;
pub use scheduler::plan_gpu_power_of_two_stockham_uploads_for_batches;
pub use scheduler::plan_gpu_rader_fft_registers;
pub use scheduler::plan_gpu_rader_fft_registers_for_containers;
pub use scheduler::plan_gpu_scheduler_policy;
pub use scheduler::plan_gpu_small_mixed_radix_registers;
pub use scheduler::plan_gpu_smooth_stockham_uploads;
pub use scheduler::plan_gpu_smooth_stockham_uploads_for_batches;
pub use scheduler::plan_gpu_stockham_twiddle_source;
pub use scheduler::plan_nvidia_vulkan_power_of_two_radix_registers;
pub use scheduler::plan_nvidia_vulkan_power_of_two_stockham_uploads;
pub use scheduler::plan_nvidia_vulkan_power_of_two_stockham_uploads_for_batches;
pub use scheduler::plan_nvidia_vulkan_rader_fft_registers;
pub use scheduler::plan_nvidia_vulkan_rader_fft_registers_for_containers;
pub use scheduler::plan_nvidia_vulkan_small_mixed_radix_registers;
pub use scheduler::plan_nvidia_vulkan_smooth_stockham_uploads;
pub use scheduler::plan_nvidia_vulkan_smooth_stockham_uploads_for_batches;
pub use scheduler::plan_nvidia_vulkan_two_three_radix_registers;
pub use scheduler_snapshot::SCHEDULER_SNAPSHOT_SCHEMA_VERSION;
pub use scheduler_snapshot::SchedulerAxisClassification;
pub use scheduler_snapshot::SchedulerPhysicalAxisProbeContext;
pub use scheduler_snapshot::SchedulerRaderUploadProbe;
pub use scheduler_snapshot::SchedulerSnapshotMismatch;
pub use scheduler_snapshot::SchedulerSnapshotPayload;
pub use scheduler_snapshot::SchedulerSnapshotRecord;
pub use scheduler_snapshot::compare_upstream_scheduler_snapshot_jsonl;
pub use scheduler_snapshot::fixed_upstream_scheduler_snapshot_corpus;
pub use scheduler_snapshot::scheduler_axis_classification_probe;
pub use scheduler_snapshot::scheduler_rader_upload_axis_block_probe_records;
pub use scheduler_snapshot::scheduler_rader_upload_axis_block_probe_records_with_context;
pub use scheduler_snapshot::scheduler_rader_upload_probe_record;
pub use zero_pad_ir::NdZeroPadPassIr;
pub use zero_pad_ir::ZeroPadPassIr;
pub use zero_pad_ir::ZeroPadPassOperation;
pub use zero_pad_ir::execute_nd_zero_pad_pass;
pub use zero_pad_ir::execute_zero_pad_pass;

Modules§

application
High-level typed transform planning facade.
backend
Backend lowering interfaces.
binary16
Pure-Rust IEEE-754 binary16 storage values.
bluestein_ir
Backend-neutral multi-pass Bluestein execution IR.
complex
config
convolution_ir
Application-level convolution IR matching VkFFT’s performConvolution semantics.
double_double
Error-free-transform based double-double arithmetic.
double_double_ir
Executable Stockham IR for VkFFT’s double-double compute modes.
double_double_recursive_ir
Recursive mixed-radix composition for double-double one-dimensional FFTs.
error
kernel_ir
Backend-neutral compute-kernel IR.
lut
Backend-independent lookup-table generation.
mixed_ir
Backend-neutral mixed Stockham × Rader axis decomposition.
nd_ir
Backend-neutral multidimensional C2C FFT composition.
nd_real_ir
Multidimensional real/complex FFT composition.
one_dim_ir
Generic one-dimensional complex FFT composition.
planner
precision
Precision-reporting helpers shared by correctness sweeps and benchmark tooling.
program_ir
Backend-neutral multi-pass program/resource IR.
r2r_ir
Correctness-first real-to-real DCT/DST execution IR.
rader_ir
Backend-neutral correctness-first direct-multiplication Rader IR.
real_ir
Backend-neutral 1D real/complex transform composition.
recursive_ir
Recursive backend-neutral Cooley-Tukey decomposition for composite Rader axes.
reference
Correctness-oriented CPU reference transforms.
scheduler
Typed host-side scheduling metadata ported from VkFFT’s GPU scheduler.
scheduler_snapshot
Stable, machine-readable candidate snapshots of the Rust scheduler decisions.
zero_pad_ir
Typed upstream-style zero-padding boundary passes.

Constants§

UPSTREAM_VKFFT_COMMIT
Upstream VkFFT commit used as the initial porting baseline.
UPSTREAM_VKFFT_VERSION
VkFFT release encoded by the upstream header used as the initial porting baseline.