# Architecture
## Scope
The repository contains both the portable protocol implementation and concrete host backend work.
The portable layers define the semantics that must agree across every guest transport, virtual
machine monitor, host operating system, and hardware provider. Platform integrations are adapters;
they implement those semantics without owning or redefining them.
Protocol 1.0 does not assign a Virtio device ID. It also does not standardize vendor executable
formats: artifact format identifiers and target words remain opaque to the transport.
The normative terminology, object model, and compatibility rules live in
[specification.md](specification.md), with exact layouts in [wire-abi.md](wire-abi.md) and command
queue rules in [virtqueue.md](virtqueue.md). This document explains the implementation boundaries
that preserve those rules. The portable trust assumptions and denial-of-service bounds are mapped
in [threat-model.md](threat-model.md).
## Load-bearing invariants
### Wire safety
Every wire structure is fixed-width, little-endian, pointer-free, and valid at byte alignment.
`zerocopy` derives reject layouts containing implicit padding. Raw numeric opcodes and status values
are validated before conversion to semantic types, preserving forward compatibility without invalid
Rust enum discriminants.
Request and response buffers are untrusted. The command-frame preflight validates descriptor
direction, total byte counts, reserved-zero fields, array multiplication, configured limits, and
command-specific response capacity before a decoded request can reach semantic dispatch.
### Object identity and ownership
Guest-visible handles are opaque `u64` values. The device table combines a slot number with a
device-instance namespace, resource-kind tag, and generation. Removing an object increments its
generation; a slot is permanently retired before generation overflow. Therefore stale, wrong-kind,
cross-device, and pre-reset handles never alias a live object when each reset epoch receives a fresh
namespace.
`DeviceState` composes typed context, buffer, program, execution-queue, and event tables. Context
records retain live-child counts, while event records retain queue, program, and buffer references
until event destruction. Destroying a context with children or a referenced buffer, program, or
queue returns `BUSY`.
The state model has no internal locks or interior mutability. Every transition requires exclusive
access, so a future concurrent command engine has one outer synchronization boundary and no nested
resource-lock order. Creation checks quotas and reserves fallible table capacity before invoking a
provider. Release moves a handle to an explicit `Releasing` state and either commits removal or
restores the same live ID after a rejected provider release.
`CommandProcessor` preserves that ownership model rather than placing an atomic or lock in each
record. One mutable processor owns one backend and one object graph. A transport may move that owner
between workers or serialize admission at its queue boundary; provider-native asynchronous work
continues behind borrowed handles and event objects. Atomics belong in provider completion tokens
or the concrete Virtio status publication path where state is genuinely shared, not in portable
object lookup or reference counting.
Provider releases have an explicit failure boundary too. A rejected release returns the still-live
handle for retry. An indeterminate release invalidates the guest ID and requires device recovery;
the adapter must never guess that the resource is either safe to reuse or safe to free.
### Reset and quarantine
The transport stops fetching chains and publishing completions before handing exclusive ownership
to `CommandProcessor::reset`. The processor then makes one bounded pass: events first, followed by
execution queues, programs, buffers, and contexts. Pending events are cancelled only when the
backend advertises cancellation; no reset path spins, waits, or creates a background executor.
A completely drained graph receives a fresh `ObjectNamespace` and may continue with the same
backend. Any unresolved pending event, rejected reset release, indeterminate release, device loss,
or accounting contradiction produces `BackendDiscardRequired`. That result reports both resources
released during the pass and resources still represented or previously orphaned in quarantine.
The result is sticky: later reset calls make no provider calls, and the complete processor/backend
instance must be discarded rather than reattached to newly initialized queues.
This keeps synchronization at the existing owner boundary. Reset needs no per-record atomics or
locks; provider completion tokens remain responsible for the cancellation/completion race.
### Submission acceptance
A rejected submission guarantees that the backend accepted no execution and retained no resources.
When acceptance cannot be determined, `SubmitFailure::Indeterminate` carries an event. That event is
the ownership token for all referenced resources until it becomes terminal and is destroyed.
This distinction must survive every provider and transport adapter. Collapsing the two cases into a
single error would permit use-after-free during device-reset and timeout races.
### Time
Wire timeouts are relative nanosecond durations measured from device admission. Guest and host
monotonic clocks do not share an epoch, so absolute guest timestamps are never compared with a host
clock. A zero timeout means infinite.
### Memory
The baseline contract uses device-owned buffers plus bounded read/write transfers. External memory,
shared mappings, and fences are optional features because their ownership, cache visibility, and
synchronization rules differ by transport and host OS. Adding them requires a separate invariant
and threat-model pass.
Provider-owned shared memory is distinct from external memory. `MemoryDomain::Shared` requests one
allocation that the provider can access through a host mapping and bind directly for accelerator
execution. It does not expose a guest address or platform handle and does not imply cross-process
sharing or implicit cache coherence.
The allocation result reports verified backing properties, actual retained bytes, and actual
alignment separately from the provider-native handle. Logical buffer bytes may be smaller than a
page-, section-, or device-aligned backing allocation. The command engine retains those facts in its
buffer record for compatibility checks and resource accounting, while submission passes only
borrowed native handles. This lets the device reject a dishonest or degraded allocation before it
becomes guest-visible without adding metadata lookup or boxing to the execution hot path.
`ResourcePolicy` supplies mandatory host-private aggregate limits without duplicating the wire or
provider capability limits. The command engine prechecks logical buffer bytes, reconciles the
provider's actual backing charge before publishing an ID, and charges program resident storage
before provider invocation. Charges remain represented across rejected releases; indeterminate
ownership transfers them to quarantine and makes the entire backend discard-only. `ResetReport`
therefore reports both object counts and exact released or quarantined retained bytes.
Bulk byte payloads cross the semantic boundary through `ByteSource` and `ByteSink`. Both abstractions
support checked random access over segmented storage and an optional contiguous view. A command
engine can therefore expose validated descriptor-backed regions directly: a provider streams them
into final storage, while an already contiguous payload remains one borrowed slice.
## Queue model
Command virtqueue zero is the baseline bidirectional transport queue. One descriptor chain contains
device-readable request bytes and device-writable response bytes. Completion may be out of
submission order, keyed by the request ID.
`virtio-accel-transport` defines the queue boundary without choosing a ring implementation or guest
memory library. Driver publication transfers ownership of a complete chain until used-ring
consumption or reset returns it. Device pop returns a non-`Copy` chain consumed by completion. Queue
identities include a monotonic reset epoch, so stale completion is rejected before guest bytes, a
used element, or a notification can be published.
Publication and completion are release boundaries for request and response bytes; the corresponding
pop operations are acquire boundaries. Notification enablement includes the required atomic recheck,
represented explicitly as `Idle` or `WorkPending`. Concrete adapters may use atomics and atomic
pointers for shared indices and ownership transfer, but the portable traits require no lock, thread,
executor, or global runtime.
Queue configuration may reserve storage bounded by the validated queue size. Every steady-state
operation is nonblocking and heap-allocation-free: publish, pop, complete, notification suppression,
notification recheck, and reset. Reset may move already-owned storage into a reclamation result but
does not allocate or wait for a peer.
`virtio-accel-guest` owns one portable driver queue without internal synchronization. It
preallocates a caller-selected number of tracking slots, writes fixed prefixes directly into
caller-owned chains, and retains bulk read responses in reclaimed chain storage. Prepared transfer
and artifact tails are published without another payload copy. Non-`Copy` typed handles carry the
queue reset epoch; release operations consume them and report whether a failure is retryable,
invalidated, indeterminate, or an opaque unknown status.
`virtio-accel-split-queue` is the deterministic executable implementation of that boundary. It
preallocates descriptor ownership, chain records, available entries, and used entries at
configuration. Its split-ring counters use wrapping `u16` arithmetic, direct chains retain their
scatter/gather buffers, and profile-invalid flags or indirect descriptors are classified before
byte access. Driver and device operations take `&mut SplitQueue`, so ordinary ring state needs no
atomics, atomic pointers, locks, or compare-and-swap loops. Non-atomic `Rc` ownership keeps a reset
reclamation token and a consumed device token tied to the same buffers; one `AtomicU64` reset epoch
is the only synchronization primitive, because it must invalidate byte ports already issued to a
device token before driver ownership is reclaimed.
The baseline `SUBMIT` command returns an event object; `POLL_EVENT` provides portable progress without
requiring unsolicited device writes. Optional multi-queue and event-queue features are reserved for
later validation. Split and packed virtqueue mechanics belong to transport adapters, not the command
engine.
An accelerator execution queue is a separate context-scoped backend object. It never denotes a
Virtio queue index.
## Performance posture
The semantic hot path uses associated handle types and borrowed binding slices, avoiding trait-object
dispatch and per-binding boxing. Wire decoding will operate directly over validated descriptor-backed
regions. Object lookup is constant time and bounded by advertised limits. The queue ports add no
allocation or copy to the steady-state path; mapping implementations can present borrowed segmented
byte ports directly to the command processor.
`Accelerator` deliberately places no `Send` or `Sync` bound on the backend or its associated handle
types. The reference command engine specializes over one concrete backend and owns it behind one
mutable admission boundary. A provider can therefore preserve thread-affine native handles without
boxing, atomics, or locks. Providers that opt into cross-thread auto traits own the synchronization
needed by their actual shared state; the portable object graph does not speculate by adding it to
every handle.
The source-level trait may be erased only after an adapter fixes all associated handle types. Stable
binary plugin loading, cross-module allocation ownership, and an erased handle ABI are deliberately
outside v1. A future plugin adapter can add those policies without changing static providers or
weakening the submit and release contracts.
Backend metadata is fetched and validated once before object tables are constructed. Assigned
reserved capabilities, a missing usable memory domain, and zero advertised limits fail construction.
Unknown capability bits remain available for diagnostics but do not select operations. The command
engine then uses the cached capabilities and limits to reject unsupported work before provider
invocation.
`WRITE_BUFFER` and `READ_BUFFER` are the baseline's explicit content-copy boundaries. Device-local
memory may require bounded staging during those operations. Allocation, submission, polling, and
release do not receive permission to copy a bound buffer merely because a native import or binding
path is inconvenient.
Every buffer declared for program input, output, or mutable state reports
`BufferProperties::DIRECT_BINDING`. This means a compatible submission binds that exact allocation
without copying the bound range to or from a different allocation. A backend that cannot honor the
requested placement and direct-binding contract rejects allocation; a program-specific alignment or
format mismatch rejects submission as `INCOMPATIBLE`. Neither path may silently degrade to a bounce
buffer.
The mutable side of an explicit write receives `&mut Buffer`, allowing implementations to use
ordinary provider handles and mappings rather than forcing interior mutability or a lock into every
buffer. Submission remains borrowed and allocation-free in the semantic API.
Program artifacts use the same byte-source abstraction. Program loading is a slow lifecycle path,
so an object-safe source is an acceptable dispatch cost; forcing a frame-sized allocation and copy
for every segmented artifact is not. Providers can parse a contiguous borrowed artifact in place or
read segmented bytes directly into final resident storage.
Zero-copy guest-memory imports are deliberately deferred rather than pretending that DMA-BUF,
Windows shared handles, and other mechanisms have identical lifetime or coherency semantics. When
external memory is specified, fallback staging will require explicit negotiation and copy
accounting; it will not weaken the provider-owned direct-binding rule. The non-normative
[external-memory handoff design](plans/issue-113-external-memory-handoff.md) fixes the proposed
ownership, visibility, reset, isolation, fallback, and conformance boundaries without assigning or
advertising the reserved protocol feature.
### Backend fast-path checklist
A provider implementation should make the native buffer handle own or reference everything needed
to reuse the allocation efficiently: the final backing object, device address or import, host
mapping when present, alignment facts, and synchronization state. Native mapping or import setup
belongs at allocation or another amortized lifecycle boundary, not in every submission.
The intended steady-state submission path is a bounded walk over the borrowed binding slice,
validation of program-specific compatibility, native handle/address binding, and queue admission. It
does not allocate per binding, assemble a second binding array with owned payloads, or copy tensor
contents. Small command and metadata writes are not buffer staging and remain provider-specific.
The command engine uses three bounded metadata allocations for `SUBMIT`: decoded bindings, retained
buffer IDs, and borrowed native binding references. Duplicate-slot detection sorts the decoded
allocation in place; event state takes ownership of the retained-ID allocation. No allocation owns
buffer contents, boxes individual bindings, or survives event reclamation except the retained ID
list required for exactly-once reference release.
[performance.md](performance.md) owns quantitative evidence: explicit-transfer bytes, staged bytes
and allocations, submission allocations, retained memory, and host preparation versus device
execution time. A backend should be diagnosable when it misses the intended path rather than
requiring a profiler to discover an undocumented copy.
## Deterministic reference execution
`virtio-accel-mock::reference` defines a test-only artifact envelope for executable backend tests.
Its fixed 24-byte payload carries an artifact version, a binding-ABI version, an operation, binding
slots, one byte operand, and reserved-zero bytes. The mock additionally requires its provider-owned
format ID, target identity, and exact resident charge before accepting a program. These values and
payload bytes are implementation fixtures, not additions to the normative accelerator ABI;
production command and transport layers continue to pass artifact formats, targets, and payloads
through opaquely.
The reference operations are a lifecycle barrier, equal-length copy, fill, and in-place XOR. Each
operation validates its exact slot and access contract before event admission. Buffers use shared
atomic-byte backing so an accepted event retains only fixed operation metadata, ranges, and atomic
reference-counted backing pointers. Submission does not lock, stage buffer contents, or allocate an
owned binding mirror. Explicit segmented transfers use a fixed-size stack window rather than
coalescing the complete transfer.
Events remain pending until the harness calls `complete`. A single compare-exchange chooses among
execution, cancellation, and injected device loss; after execution starts, cancellation and device
loss report `Busy`. Completion publishes buffer mutations before the terminal event state, while
the harness controls latency and completion order by deciding when and in which order to complete
accepted events.
## Deterministic fault injection
`virtio-accel-mock::fault::FaultAccelerator<A>` wraps any backend with a validated explicit fault
script. Each step selects one `Accelerator` method, its one-based call occurrence, and a compatible
action: error before invocation, error after successful invocation, rejected ownership transfer,
indeterminate ownership transfer, or a persistent terminal event completion. Post-create errors
synchronously release the newly created provider resource before returning the injected error.
Post-admission submission errors always return an event as indeterminate rather than misreporting
accepted work as rejected.
The wrapper assigns harness-local IDs to contexts, buffers, programs, queues, and events. Its audit
snapshot records every method call, injected action, release attempt, rollback, remaining script
step, and last known provider ownership state. It also tracks context children and the resources
retained by each event, rejecting double release, use after release, parent release with live
children, and release of an event-retained resource before the wrapped provider sees the invalid
call. Live or indeterminate resources become clean only after successful release or an explicit
backend-discard acknowledgement.
Fault scripting is single-threaded test control built on `Rc<RefCell<_>>`. It may allocate a mapped
binding vector to interpose on submission and is intentionally outside production performance
claims; the wrapped backend and ordinary command path retain their synchronization and copy
contracts.
## Reusable backend conformance
`virtio-accel-conformance` depends only on `virtio-accel-core` and runs each semantic case against a
fresh backend instance. A provider supplies one executable target fixture plus test-only progress
and optional resource-accounting hooks. Stable case IDs cover metadata, reserved intent, every
advertised memory domain, segmented transfers and artifacts, bounds, permissions, bindings,
context isolation, admission, pending-event release, terminal stability, finite timeout, and both
cancellation race outcomes.
Mandatory cases cannot skip. Memory-domain and cancellation cases skip only when the corresponding
semantic capability is absent, and reports preserve the explicit reason. Accounting, when exposed,
is sampled before and after every case and counts both live and indeterminate native resources.
The reference backend passes the suite directly and through `FaultAccelerator`; intentionally
broken adapters prove that each major contract area produces a named failure.
The suite is portable `std` test tooling rather than a production dependency. It preserves static
backend and handle dispatch and introduces no wire types, virtqueues, host APIs, threads, or global
synchronization. The [backend implementer guide](backend-implementer-guide.md) maps each case to
trait obligations and separates semantic evidence from the quantitative allocation and copy budgets
in [performance.md](performance.md).
Its `numerics` module complements lifecycle conformance with checked-in TOSA graphs and shared
FP32, FP16, BF16, FP8E4M3, FP8E5M2, INT8, INT32-accumulator, and packed INT4 oracles. Identity edge
values, non-square batched matrix multiplication, NHWC max pooling, explicit FP8-to-BF16 CAST, and
signed INT32-to-INT8 RESCALE are shared cases. A backend runs every case it advertises and rejects
unsupported profiles, extensions, and dtypes at program admission; provider-specific graphs cannot
stand in for the shared bytes.
The exact integer oracle is implemented in portable Rust from the TOSA scaling and accumulation
rules. The first production integer tiers use direct INT8 boundaries for identity and widen INT8
MATMUL operands to INT32 before explicit zero-point subtraction and INT32 accumulation. XDNA also
implements the released per-tensor scale32/single-round RESCALE back to signed INT8. Core ML
encodes MATMUL as an ML Program on macOS 26+; OpenVINO encodes the same MATMUL semantics as IR
Convert, Subtract, and MatMul nodes; XDNA specializes the exact expressions into AIE kernels.
None of these paths converts through floating point.
## Production lowering boundaries
The first production artifact path is now TOSA-to-Core ML. The facade, guest, transport, and device
engine pass the TOSA format ID, target words, and opaque bytes without importing Core ML. The Core ML
adapter depends inward on `virtio-accel-tosa`, verifies and analyzes one static TOSA graph, derives
ordinal input/output slots, emits a Core ML model, and confines temporary model compilation plus
Foundation/Objective-C state to the host-native boundary. Program-visible buffers remain exact
provider allocations from load through asynchronous prediction; lowering never authorizes staging
at submission.
The provider-neutral analysis is intentionally the shared seam rather than a second repository-wide
graph IR. A host backend may lower its supported subset directly, reject unsupported semantics at
program load, and add capability coverage without making its native SDK or model representation a
dependency of the portable stack.
Before constructing partitions, a host scheduler may query the optional
`virtio_accel_tosa::TosaCapabilityProvider` interface. Descriptors remain outside `DeviceInfo` and
the wire contract: they distinguish exact targets, dtype roles, operator/attribute constraints,
static-shape limits, and runtime-condition policy. They are conservative preflight data, while the
provider retains final authority at `load_program` for concrete graphs and current resources.
The Intel OpenVINO provider consumes the same verified model and analysis. The adapter lowers one
static TOSA graph to an in-memory OpenVINO IR document plus weights blob, compiles it for one
enumerated NPU, GPU, or CPU device with the accuracy-preserving execution hint, and binds exact
provider allocations as host-pointer tensors from load through asynchronous inference; completion
is accepted only when the runtime reports the caller's own allocation as its output storage. Its
validation runs the backend conformance suite and the shared numerical TOSA corpus on every
enumerated device and reports the direct-binding and explicit-transfer counters through the
conformance diagnostics hook. This keeps the Core ML and Intel paths on one bounded TOSA contract
without requiring either provider to adopt the other's native graph representation.
The AMD XDNA provider follows that OpenVINO boundary with one ecosystem-forced compiler
divergence. Safe Rust verifies and analyzes one static TOSA graph, matches only the advertised BF16,
explicit FP8-to-BF16 storage-conversion, or exact integer templates, and invokes the pinned
MLIR-AIE/IRON compiler as a bounded subprocess during program load or offline catalog population.
Guest TOSA bytes never enter Python: the subprocess receives a small validated specialization, and
its measured toolchain identity participates in the content-addressed cache key. A serving host may
load the resulting crate-local XDNP artifact without Python or a compiler. Each artifact carries an
exact per-slot byte/access plan so fixed DMA extents are checked again at submission.
Native execution uses the amdxdna HRX C ABI rather than XRT. One process-wide device owner creates
per-backend streams; each backend serializes accepted work through a bounded ring and worker that
bridges blocking HRX synchronization to latched, nonblocking event polling. Persistent HRX mappings
are the exact allocations bound to dispatch, and diagnostics prove that submission introduces no
bounce allocation or transfer. HRX exposes no bounded cancellation, so finite deadlines are
rejected before acceptance and a two-tier poison/watchdog model handles device loss. These choices
preserve OpenVINO's static admission, direct-binding, capability, and conformance seams; the helper
process, serialized stream, and XDNA-specific local-memory envelopes are documented hardware/runtime
constraints rather than portable API changes or silent fallback paths.
The Vulkan provider is the first GPU-class consumer of the seam and keeps the data plane
graph-shaped (ADR 0001): one admitted TOSA graph becomes one sequence of compute pipelines created
at load from crate-authored SPIR-V kernels specialized by validated shape constants, so guest bytes
never reach a driver's shader compiler (ADR 0003, ADR 0007). Every kernel addresses tensors through
one descriptor — an array of storage buffers holding the submission's bound slots and a
per-program arena for constants and intermediates — so one module per kernel serves every binding
layout, and a whole graph is recorded into one command buffer with compute barriers between
dependent dispatches. Buffers are dedicated `VkDeviceMemory` allocations bound directly as storage
buffers; host-visible domains stay persistently mapped, and device-local memory is reached only
through bounded staging inside the explicit transfer calls. Each context owns a bounded ring of
command buffers, fences, and descriptor sets; `vkQueueSubmit2` success is the admission boundary
and `vkGetFenceStatus` is the whole completion path, so no worker thread bridges the runtime.
Device loss poisons the instance. The backend runs the conformance suite and the shared FP32
operator corpus on every device it enumerates; the FP32 operator tier is verified on the Mesa
lavapipe CI lane, on Intel ANV (Arc 140V), and on Apple M4 via MoltenVK. The FP16 tier (ADR
0008) — the same operators over packed binary16 storage with crate-owned conversions and
binary32 evaluation, advertised on every device — has executed its corpus on Apple M4 via
MoltenVK, Intel Arc LNL (Mesa ANV), and AMD Radeon 860M (RADV), and the lavapipe CI lane
exercises it on every change.
The Qualcomm adapter uses the same seam. Its safe planner admits 41 of the 42 floating-point
operators shared by Core ML and OpenVINO, including owned constants/data movement, FP16 unary and
binary computation, BOOL comparison/selection/logical tensors, and INT32 indexing results. `ERF` is
the explicit exception because QAIRT 2.49 exposes no public QNN ERF operation. A separate exact
integer target supports INT8 identity and zero-point-aware INT8 MATMUL with INT32 output. Typed
tensor plans carry scalar size, owned constant bytes, and QNN scale-offset metadata through an owned
ABI, so submission range checks use exact one-, two-, or four-byte element storage. QNN static
parameters, descriptor arity, axis/permutation vectors, constant byte lengths, and generated-tensor
counts are checked before provider calls. Unsupported native pool, reverse, and product-reduction
forms decompose into HTP Gather and elementwise nodes rather than falling back to a host runtime.
FP32 remains rejected because a pinned v73 probe returned FP16-rounded results, and ambiguous generic
FLOAT8 cannot be advertised as either TOSA format. With a complete QAIRT/QNN C development package
on Windows ARM64, the audited boundary creates and finalizes QNN HTP graphs, binds exact caller
buffers, and publishes completion from a bounded worker. Driver-only and AppBuilder/Genie
installations are intentionally insufficient to enable that boundary.
The portable command engine depends on `virtio-accel-proto`, `virtio-accel-core`, and the
transport-neutral region metadata re-exported by `virtio-accel-device`. Its baseline processor:
1. Decodes one bounded request from abstract readable/writable byte regions.
2. Maintains typed object records and context dependency counts.
3. Translates wire types into validated semantic values.
4. Passes transfer and artifact regions directly to backend byte ports.
5. Produces a response without knowing about rust-vmm or a host operating system.
Submission/event retention, deterministic reset, the bounded split-virtqueue model, the no-std
reference guest, deterministic reference execution, scripted ownership-boundary faults, and the
reusable backend conformance suite now complete both portable queue endpoints and the provider
contract evidence. The threat model and enforceable aggregate resource policy close the security
model. Coverage-guided fuzzing exercises protocol decoding, descriptor segmentation, and bounded
stateful command sequences. The deterministic state-model replay suite extends that coverage with
random context/resource graphs, stale-ID probes, submission/cancel/completion/reset race schedules,
and minimized replay output for failures. A thin rust-vmm adapter supplying `virtio-device`,
`virtio-queue`, and `vm-memory` integration remains a later platform layer, as do Linux
vhost-user and an in-kernel guest driver.