virtio-accel

An experimental virtual-accelerator protocol plus production-oriented Rust implementations.

virtio-accel defines a protocol and ships executable no_std guest, device, transport, queue, and TOSA layers for exposing an accelerator to a guest: contexts, buffers, programs, execution queues, submissions, and events. The workspace also contains a real macOS Core ML backend that lowers device-neutral TOSA into Core ML and submits ANE-capable predictions with direct buffer bindings. The first target is NPU execution, while the object model deliberately leaves room for GPUs, DSPs, and other program-driven accelerators.

This is no longer a specification-only repository: the frozen protocol review input is developed alongside runnable guest/device machinery, conformance infrastructure, TOSA ingestion and analysis, and host backends. Host integrations remain isolated in adapter crates and never become dependencies of the portable facade. The project claims no Virtio device ID (yet).

This project is pre-standardization and experimental. Protocol 1.0 is frozen as a versioned review input for independent implementation — it is stable enough to build against and to disagree with in writing, not an approved Virtio specification.

Backend support

“Supported” below means that the backend admits the declared program and dtype and exercises it end-to-end; support in the TOSA parser or shared numerical corpus alone does not imply hardware execution. “Not implemented” describes this repository, not necessarily the underlying hardware.

The Core ML row describes model-boundary support; restricted INT32 outputs are also available for operators such as ARGMAX. Core ML chooses ANE or CPU placement per operation. Its INT4 facilities are compressed-weight storage rather than TOSA INT4 tensor execution, and this backend does not silently dequantize unsupported FP8, INT8, or INT4 graphs. See the virtio-accel-coreml support boundary.

Independently of backend execution, virtio-accel-tosa validates the TOSA 1.0 profiles and extensions for all five dtype columns, and virtio-accel-conformance ships shared fixtures and oracles for them. The byte-oriented virtio-accel-mock backend remains test infrastructure rather than a typed hardware implementation.

Workspace

Dependencies point downward only:

virtio-accel-split-queue ---> virtio-accel-transport
                                      ^
                                      |
virtio-accel-device ----------+-------+------> virtio-accel-core
          |
          +-----> virtio-accel-proto

virtio-accel-guest -----------> virtio-accel-transport
          |
          +--------------------> virtio-accel-proto

virtio-accel-conformance --------------------> virtio-accel-core
virtio-accel-tosa ---------------------------> virtio-accel-core
virtio-accel-coreml ----------+--------------> virtio-accel-core
                              |
                              +--------------> virtio-accel-tosa
other provider adapters --------------------> virtio-accel-core

The transport crate exposes reset-scoped chain identities, flattened direction/length metadata, and owned publication/completion tokens. Neither it nor the device-state layer leaks guest addresses, ring pointers, or concrete descriptor types into the command engine or provider backend.

Install

[dependencies]
virtio-accel = "0.1"

The facade is no_std. Add the reference backend as a dev-dependency to run the example below:

[dev-dependencies]
virtio-accel-mock = "0.1"

On an ANE-capable Mac, add virtio-accel-coreml = "0.1" separately for the host-native backend. Its production program format is TOSA 1.0; validation, analysis, and Core ML model generation all happen inside that adapter. It is intentionally not re-exported by the portable facade.

Add virtio-accel-tosa = "0.1" separately to validate TOSA 1.0 artifacts, inspect safe borrowed graph and typed-attribute views, enforce complete stable-op semantics for a declared target, and construct the device-neutral TOSA artifact envelope. Model::analyze_for also produces bounded dense IDs, topological order, liveness, runtime obligations, and specialization keys for Core ML, OpenVINO, or another provider. It is intentionally not re-exported by the facade.

Production TOSA-to-Core ML example

On macOS 14+ with an accessible Apple Neural Engine, the backend-local example sends a TOSA 1.0 IDENTITY graph through the real lowering, compilation, direct-binding, asynchronous prediction, and teardown path:

cargo run -p virtio-accel-coreml --example tosa_coreml

TOSA -> Core ML -> ANE-capable result: 3.25

The portable facade, device engine, transport, and guest layers see only the TOSA artifact format, target identity, and opaque bytes. Core ML protobufs, temporary compilation assets, Foundation, and the Objective-C bridge remain owned by virtio-accel-coreml.

Portable lifecycle example

A full submission against the in-memory reference backend — allocate a buffer, load an artifact, bind it to a slot, submit, and observe the event:

use virtio_accel::core::{
    Accelerator, AccessMode, ArtifactRef, BindingRef, BufferDesc, BufferRange, BufferUsage,
    ContextDesc, EventState, MemoryDomain, QueueDesc, SubmitFailure, Timeout,
};
use virtio_accel_mock::{MockAccelerator, reference};

let backend = MockAccelerator::default();
let context = backend.create_context(ContextDesc::default())?;

// An 8-byte shared buffer the program may read and write.
let desc = BufferDesc::new(
    8,
    8,
    MemoryDomain::Shared,
    BufferUsage::TRANSFER_SOURCE
        | BufferUsage::TRANSFER_DESTINATION
        | BufferUsage::PROGRAM_INPUT
        | BufferUsage::PROGRAM_OUTPUT
        | BufferUsage::MUTABLE_STATE,
)?;
let (mut buffer, _) = backend.allocate_buffer(&context, desc)?.into_parts();
backend.write_buffer(&mut buffer, 0, &[0x00, 0x11, 0x7f, 0x80, 0xa5, 0xff, 0x3c, 0xc3])?;

// A deterministic test-only artifact: XOR every byte bound to slot 7 with 0x5a.
let artifact = reference::ReferenceArtifact::xor(7, 0x5a);
let program = backend.load_program(
    &context,
    ArtifactRef {
        format: reference::ARTIFACT_FORMAT,
        target: reference::TARGET_IDENTITY,
        payload: artifact.as_bytes(),
        resident_bytes: reference::RESIDENT_BYTES,
    },
)?;
let queue = backend.create_queue(&context, QueueDesc::default())?;

let bindings = [BindingRef {
    slot: 7,
    buffer: &buffer,
    range: BufferRange::new(0, 8)?,
    access: AccessMode::ReadWrite,
}];

// Submission is asynchronous at the ownership boundary, so it always yields an event.
let event = backend
    .submit(&queue, &program, &bindings, Timeout::Infinite)
    .map_err(|failure| match failure {
        SubmitFailure::Rejected(error) | SubmitFailure::Indeterminate { error, .. } => error,
    })?;
assert_eq!(backend.poll_event(&event)?, EventState::Pending);

// The mock backend runs under harness control, so the caller drives completion.
backend.complete(&event)?;
assert_eq!(backend.poll_event(&event)?, EventState::Complete);

let mut output = [0_u8; 8];
backend.read_buffer(&buffer, 0, &mut output)?;
assert_eq!(output, [0x5a, 0x4b, 0x25, 0xda, 0xff, 0xa5, 0x66, 0x99]);

Every object is released explicitly, and a release can itself fail; see examples/reference_execution.rs for the teardown path.

cargo run --example reference_execution

Protocol 1.0

The protocol defines fixed headers and payloads for device discovery, contexts, buffers, programs, execution queues, submissions, and events. Two properties shape most of the API:

• Unknown values stay raw. Unrecognized opcodes, statuses, and event states remain integers until validated, so decoding untrusted bytes never constructs an invalid Rust enum.
• Failure still returns an event. A successful submit returns an event; an indeterminate failure must also return one, because the operation's resources are still owned by the device. Guest-visible object IDs are opaque, kind-tagged, generational, and never reused after generation exhaustion.

The primary zerocopy ABI and the manual clean-room codec both decode and re-encode every canonical frame. Their bridge test exchanges bytes only, providing an independent implementation check without making the conformance codec a production dependency.

Writing a backend

Implement the Accelerator contract from virtio-accel-core, then run the standard semantic suite against it. The suite is transport-free: no wire format, virtqueue, OS, or vendor dependency.

cargo run --example backend_conformance

memory.shared: Passed
buffer.transfer-permissions: Passed
submission.context-isolation: Passed
event.cancellation-races: Passed
accounting.resource-lifecycle: Passed
...

The backend implementer guide walks through the hooks, the optional resource-accounting and progress adapters, and the fault-injection harness.

Documentation

Portability

Project-authored portable and reference code forbids or denies unsafe code. The audited Core ML adapter keeps its unsafe FFI isolated to macOS; the TOSA crate confines official generated FlatBuffers accessors to a private module behind bounded verification. CI enforces each portability tier, including compile-only checks of the adapter's unsupported-platform surface.

Concrete VMM, kernel, OS, and vendor adapters do not change the portable v1 protocol and must not become default dependencies of a portable crate. Cargo features must be additive: disabling default features may remove convenience behavior, but must never select a different protocol interpretation.

Development

Minimum supported Rust version is 1.85 (edition 2024), checked in CI.

cargo fmt --all -- --check
python3 ci/check-release-policy.py
cargo clippy --workspace --all-targets --all-features -- -D warnings
cargo test --workspace --all-targets --all-features
cargo run --example backend_conformance
cargo run --example reference_execution
cargo run -p virtio-accel-coreml --example tosa_coreml # macOS 14+ with ANE
python3 ci/publish-dry-run.py

Target checks need the corresponding standard libraries:

rustup target add aarch64-unknown-none riscv64gc-unknown-none-elf wasm32-unknown-unknown

Status

Included in protocol 1.0:

• one command virtqueue at index zero
• device discovery and exact protocol compatibility checks
• contexts, buffers, opaque programs, execution queues, submissions, and events
• bounded explicit buffer transfers
• event polling, optional cancellation, release, reset, and backend-discard recovery
• direct-binding requirements for program-visible buffers
• checked finite limits for untrusted byte counts, descriptor counts, object counts, and retained backend storage
• an independent clean-room codec and a transport-free semantic conformance suite

Reserved and unadvertised — an implementation that advertises one of these is not 1.0 conformant until a future version assigns its negotiation, ownership, synchronization, and conformance rules:

• multi-queue and event queues
• external memory import/export
• timeline fences
• secure contexts
• packed virtqueues
• protocol-level negotiation for additional VMM, kernel, OS, and vendor integrations
• a standardized graph IR, compiler, or executable format

Protocol 1.0 numeric opcodes, statuses, and payload layouts are frozen for the portable v1.0 baseline by the final freeze audit. Future changes must follow the coordinated change procedure in wire-abi.md and the release and evolution policy; incompatible changes require a new protocol major version.

Contributing

Contributions are welcome, including disagreement with frozen decisions — a reasoned objection is worth more than a workaround built on top of one. See CONTRIBUTING.md for the local gates, the scope boundaries, and how wire changes are classified before code is merged.

• Questions and backend porting help → Discussions → Q&A
• Early design ideas → Discussions → Ideas
• Suspected vulnerabilities → not a public issue; follow SECURITY.md

License

Licensed under either of Apache License, Version 2.0 or MIT license at your option.

Contributions are dual-licensed on the same terms, with no separate CLA.