Crate kn_cuda_eval

source ·
Expand description

A Cuda CPU executor for neural network graphs from the kn_graph crate. The core type is CudaExecutor.

This crate is part of the Kyanite project, see its readme for more information. See system-requirements for how to set up the cuda libraries.

§Quick demo

// load and optimize the graph
let graph = load_graph_from_onnx_path("test.onnx", false)?;
let graph = optimize_graph(&graph, Default::default());

// select a device
let device = CudaDevice::new(0).unwrap();

// build an executor
let batch_size = 8;
let mut executor = CudaExecutor::new(device, &graph, batch_size);

// evaluate the graph with some inputs, get the outputs back
let inputs = [DTensor::F32(Tensor::zeros(vec![batch_size, 16]))];
let outputs: &[DTensor] = executor.evaluate(&inputs);

Modules§

Structs§

  • Export the CudaDevice type for convenience: often an explicit dependency on the kn_cuda_sys crate is not needed. A cuda device index.