Crate kn_cuda_eval
source ·Expand description
A Cuda CPU executor for neural network graphs from the kn_graph
crate. The core type is CudaExecutor.
This crate is part of the Kyanite project, see its readme for more information. See system-requirements for how to set up the cuda libraries.
§Quick demo
// load and optimize the graph
let graph = load_graph_from_onnx_path("test.onnx", false)?;
let graph = optimize_graph(&graph, Default::default());
// select a device
let device = CudaDevice::new(0).unwrap();
// build an executor
let batch_size = 8;
let mut executor = CudaExecutor::new(device, &graph, batch_size);
// evaluate the graph with some inputs, get the outputs back
let inputs = [DTensor::F32(Tensor::zeros(vec![batch_size, 16]))];
let outputs: &[DTensor] = executor.evaluate(&inputs);
Modules§
- The autokernel infrastructure and specific kernels.
- On-device tensor data structure.
- The main executor type and the compiler for it.
- Tensor utility.
- Shape utilities.
- Testing and debugging infrastructure.
- Miscellaneous utilities.
Structs§
- Export the CudaDevice type for convenience: often an explicit dependency on the
kn_cuda_sys
crate is not needed. A cuda device index.