# Plugin Testing
Plugin tests protect compatibility surfaces and security invariants. They
should prove externally visible behavior, not implementation trivia.
## Required Coverage
Every runtime slice needs focused tests before real guest execution.
- config and manifest default-deny behavior
- effective grants are subsets of config and manifest request
- config narrowing of broader manifest workspace requests
- explicit whole-workspace grant spelling
- zero and oversized runtime-limit rejection
- validated runtime limits expose host queue caps as host-sized non-zero proofs
while preserving serialized config round trips
- runtime, workspace I/O budget, and workspace observation task queue
zero-limit diagnostics use closed field vocabularies with stable text
- workspace request-budget, pending-effect, intent queue, proposal lane,
observe-task pending-capacity, workspace worker, and operational queue
saturation diagnostics retain non-zero cap proofs while keeping stable
redacted text and operational event shapes
- queue, handle-store, pending-batch, and proposal-lane constructors that
receive validated limit or budget proofs are infallible, with zero-limit
coverage kept at the proof constructors
- paired successful-return, owner-work, post-update schedule, and scheduled
owner-publication constructors enforce same-plugin identity outside debug
builds
- deterministic WIT vocabulary drift
- deterministic WIT host-import metadata drift across import names,
authorizing capabilities, and operational diagnostics
- deterministic WIT drift coverage proving task-based `workspace.observe`
start/poll/take imports are exposed as a complete group, and that the public
ABI still exposes no blocking read import
- deterministic task-import metadata coverage tying each
`workspace.observe` start/poll/take name to the `workspace.observe`
capability atom and one stable redacted operational import class
- deterministic guest-decode field coverage, so generated-to-Alma validation
errors use closed stable field labels instead of adapter-supplied strings
- guest-decode limit coverage, so adapter decode and return-size checks derive
from non-zero validated runtime caps instead of raw caller convention
- deterministic capability atom drift across config/manifest names, WIT
capability adapters, authorization-ref path arity, and redacted shapes
- deterministic config and manifest schema drift
- deterministic proposal review policy schema drift
- deterministic plugin golden coverage for `tests/goldens/plugin`
- generated WIT resources do not expose Bevy entities, host paths, filesystem
authority, permissions, or raw file handles
- generated WIT does not expose a broad workspace resource handle
- denied imports produce no effects
- failed guest updates discard pending effects
- pending effect batch limit and saturation errors expose closed stable field
and rejection-class vocabularies
- empty sealed effect, workspace I/O, and workspace observe-task batches do not
consume bounded queue capacity or turn otherwise full queues into saturation
failures
- failed guest updates discard pending workspace I/O
- sealed batches drain into typed ECS requests or proposals only
- drained effect request splits return named owner request batches and retain
the drained plugin identity instead of exposing positional tuples
- plugin buffer edit proposals can be held pending, applied through the buffer
owner, or rejected without mutation
- plugin intent queue, proposal lane, proposal decision, and proposal
receipt-log errors expose closed stable rejection classes
- sealed effect batch admission to `PluginIntentQueue` returns the still-sealed
batch on saturation for retry or explicit redacted discard
- proposal lane rendering exposes only proposer identity, capability, target
shape, base revision, redacted edit shape, and redacted diff byte counts
- proposal receipts correlate proposal ids with proposer identity, capability,
target, base revision, proposal-lane rejection, buffer-owner acceptance, or
buffer-owner rejection
- proposal provenance and receipt capability spellings derive from
`CapabilityAtom`
- proposal status rendering exposes only pending state or closed receipt
outcome shape
- per-plugin proposal review policy defaults to `auto_apply` and configured
`manual_review` keeps buffer edit proposals pending without owner mutation
- workspace worker queue is bounded, cancelable, and FIFO
- workspace worker queue limit and saturation errors expose closed stable field
and rejection-class vocabularies
- owner rejection after drain is observable
- debug and operational diagnostics redact payloads and paths while exposing
only closed failure shapes
- successful-return aggregate batch debug output reports identity and counts
rather than nested sealed effect, workspace I/O, or observe-task internals
- failed-update, owner-work, post-update schedule, and scheduled owner discard
report debug output reports identity and counts rather than nested discard
reports, unpublished work, task handles, paths, or payload bytes
- workspace I/O passes through `fs_utils`
- workspace I/O execution reports are operation-typed, so read completions
expose only read outcomes and write completions expose only write outcomes
- successful workspace I/O report debug output redacts host paths, workspace
paths, and file bytes while explicit success accessors still return bounded
owner-facing data
- workspace observe delivery projection returns only bounded bytes or a closed
rejection, rejects write completions, and redacts paths and bytes from debug
output
- workspace observe delivery shape is the shared redacted projection for task
result diagnostics and runtime trace adapters
- workspace observe task ids are reserved before guest-visible task resources
return, are not reused after failed updates, and are paired with
pending-capacity reservations consumed by sealed batches after successful
guest return
- workspace observe task resources dropped before successful guest return
remove the matching pending request, release the reservation, and do not
publish sealed filesystem work
- workspace observe task requests retain the producing update's read byte cap,
and queue execution uses the retained cap rather than a caller-supplied limit
- workspace read and write execution paths require non-zero byte cap proofs,
projected from validated workspace I/O budgets for deferred reads, direct
observations, task-style observations, and deferred artifact writes
- pending and sealed workspace artifact write batches retain only bounded
payload proofs and redact paths and payload bytes from debug output
- workspace observe task queues, sealed task batches, and host import batches
retain `Send + Sync` auto-traits for future runtime scheduling
- workspace observe task reservation accounting is owned by explicit RAII queue
slots, not raw shared-owner counts, and internal underflow cannot silently
pass in release builds
- workspace observe task result vocabulary stays separate from queue mutation,
so adapter-facing identity-scoped task handles, owned queue keys, internal
handle-borrowed lookup matching, shared closed task state, payload-free queue
state queries, payload-free poll results, consuming take results, shared
redacted poll/take result shapes, and identity-scoped retained completion
shapes can be encoded without granting queue scheduling or retention
authority
- sealed workspace observe task batches enqueue only after successful guest
return, consume previously held reservations, and reject cross-queue
reservations without partial enqueue while returning the still-sealed batch
for retry or explicit discard with redacted cleanup reporting
- sealed workspace observe task admission keeps the validated pending queue
storage paired with the admission proof through enqueue, preserving the
all-or-nothing reservation boundary without caller-side queue conventions,
and retryable enqueue-error splits expose named queue-error and batch fields
- sealed workspace I/O worker queue saturation returns the still-sealed batch
for retry or explicit discard; retryable enqueue-error splits expose named
queue-error and batch fields and redact paths and payload bytes
- workspace I/O worker and observation task queue all-identity cancellation
clears queue-owned work without filesystem execution or guest delivery,
returns deterministic per-identity count reports, and redacts paths and bytes
- Wasmtime workspace observation task revocation cleanup requires a matching
`PluginRevocationReport`, cancels queue-owned pending and retained task
state, removes uncommitted active-update task reservations, tombstones
persistent task resources, and remains idempotent
- Wasmtime workspace observation task shutdown cleanup cancels runtime-owned
queued work, removes uncommitted active-update task reservations, tombstones
persistent task resources, and leaves caller-held sealed batches caller-owned
- Wasmtime state revocation through the runtime-owned helper rejects
runtime/state identity mismatches before mutating host state, and returns the
lifecycle revocation proof together with observe-task cleanup through named
split fields
- Wasmtime workspace observation task enqueue rejects a sealed task batch whose
identity does not match the runtime instance before queue admission, and
enqueue, single-task execution, capped batch execution, and drain-all
execution methods reject a revoked `PluginInstanceState`, including the case
where a successful update already returned a sealed task batch before
revocation
- Wasmtime update-and-schedule helper coverage proves a real component can
start a workspace observation task, have it scheduled on the producing
runtime queue, and return a `WasmtimePluginScheduledUpdate` proof that keeps
sealed editor and workspace I/O batches paired with already-scheduled task
evidence until owner publication
- Wasmtime workspace observation task execution coverage proves the unbounded
drain-all helper is named explicitly, while single-task and non-zero capped
batch execution remain the owner-loop-friendly APIs
- owner-publication coverage proves paired editor effect and workspace I/O
batches validate both owner queues into a private paired admission proof that
retains the target queue borrows before mutation, individual owner queue
admissions own the sealed batches until publication, saturated queues reject
without partial publication, retryable owner work is retained, successful
publication returns named count-shaped evidence, and debug output exposes
only counts instead of unpublished status text, paths, payload bytes, or
retained batch internals. Retryable owner-queue enqueue splits expose named
queue-error and owner-work fields.
- scheduled-update owner-publication coverage proves the ECS helper accepts
`WasmtimePluginScheduledUpdate`, returns a named owner-publication success
report carrying owner-publication evidence and the observe-task scheduling
report, and on owner queue rejection preserves explicitly named
already-scheduled task evidence with retryable owner work or a named discard
report without leaking paths, payloads, status text, retained batch
internals, or task-handle lists in debug output
- scheduled-update owner-publication retry coverage proves a rejected owner
publication can retry without splitting the already-scheduled task report,
returns the same paired failure while capacity is still exhausted, and
publishes owner work after capacity frees
- Wasmtime post-update observe-task schedule failures retain the unpublished
owner-work proof with the retryable task enqueue error, expose count-shaped
debug output instead of retained batch internals, redact task paths from
display and debug output, rebuild scheduled-update batches after successful
retry, and return one redacted report when callers explicitly discard the
unpublished owner work and retained task batch
- scheduled-update and scheduled-owner-publication public split APIs return
named part structs rather than positional tuples for owner work, retryable
queue errors, discard evidence, and already-scheduled task evidence
- active-update, task-resource authority, Wasmtime revocation cleanup, and
adapter-local update resource splits return named part structs rather than
positional tuples
- Wasmtime scheduled-update and observe-task enqueue errors expose redacted
operational events for reportable failures without leaking unpublished owner
work, task paths, payloads, or filesystem data
- Wasmtime observe-task enqueue errors expose a closed public kind plus
source-specific accessors for lifecycle, state, batch-identity, and queue
failures while retaining the retryable sealed batch behind a private payload,
return named borrowed identity pairs for state and batch mismatches, and
their debug output stays count-shaped instead of formatting the retained batch
- Wasmtime workspace observation task poll/take revalidate the task resource's
original workspace path against the current update snapshot before
resource-table lookup or queue access; the resource authority path is paired
with the reserved task handle by the host-session proof and named when split
for resource-table storage, denied revalidation tombstones the resource
without freeing the table rep, the persistent
resource cap is derived as a non-zero proof from the queue limits, and denied
access yields the typed authorization rejection
- Wasmtime update setup tombstones task resources still marked as started by an
abandoned prior update, so stale uncommitted task resources fail closed
before the next guest call can poll or take them
- Wasmtime workspace observation task resource tracker/table divergence is
covered for start, resolve, and drop paths, including reused table reps and
table lookup/delete errors that tombstone the tracked resource, and fails
closed through the stable resource-handle rejection vocabulary in release
builds
- pending workspace observe task batches reject cross-plugin access proofs and
paired workspace I/O ledgers before charging request budget or retaining task
work
- workspace I/O commit errors and observe-task queue errors expose closed
stable error classes, and their identity or sealed-reservation mismatches
expose closed stable source labels, not presentation-only strings
- direct host-session workspace observation authorizes through captured grants,
requires explicit filesystem policy context, returns only bounded bytes or a
closed rejection, spends the shared per-update workspace request budget before
filesystem access, rejects cross-plugin access proofs before charging budget,
and does not enqueue deferred workspace I/O
- deferred workspace I/O, direct host-session observations, and task-style
observation reservations share one per-update request cap, and rejection
leaves the relevant pending batch unchanged
- deferred workspace I/O request counts derive from retained pending operations
while non-deferred direct and task-style reads share the same cap
- workspace read/write access proofs retain the issuing plugin identity, and
pending workspace I/O rejects proofs from another plugin identity before any
filesystem policy execution
- workspace observation task queues allocate non-zero task ids, reserve
non-reused ids and pending capacity for pending update batches, enforce
pending and retained-result caps, return state, poll, and take results only
through identity-scoped lookups, keep state and poll results payload-free,
consume completed outcomes at most once through take, execute reads through
`fs_utils` with the retained read byte cap, support non-zero capped batch
execution without draining unbudgeted pending tasks, evict old retained
results, keep queue and pending/sealed batch debug output count-shaped, emit
redacted saturation events, and remove both pending and retained state on
revocation
- workspace observation task requests remain pending until successful guest
return, failed updates discard them, sealed task batches enqueue
all-or-nothing, returned resources from failed updates are tombstoned, and
pre-commit task-resource drops remove the pending request so queue saturation
cannot partially publish filesystem work
- production workspace observation task enqueueing goes through sealed
successful-return batches only; direct queue insertion from authorized read
proofs remains test-only for focused queue coverage, and runtime scheduling
tests cover the active-state gate
- runtime trace events adapt effects, workspace I/O, workspace I/O
cancellation reports, workspace observation task completions with
identity-scoped completion shapes, observation task cancellation and discard
counts, operational events, and proposal receipts without exposing raw paths,
raw ECS ids, status text, replacement text, file bytes, OS errors, or guest
payloads
- workspace imports authorize each operation by kind and normalized path before
queuing deferred filesystem work
- Wasmtime host-import rejection retention keeps the first redacted import
class paired with its typed source error and does not let later failures
overwrite it
- host-import source errors project closed operational rejection classes before
runtime adapters build operational events
- guest-visible `workspace.observe` remains task-only. Tests cover result
delivery, byte bounds, redaction, poll/take state mapping, discard semantics,
revocation cleanup, generated adapter mapping, and a real component that
starts, schedules, polls, takes, drops before commit, and drops after
completion. Do not add a blocking synchronous read import.
Property tests are preferred for identity grammar, component path shape,
workspace path validation, grant prefix coverage, validated limits, handle
generations, bounded pending batches, permission checks, and serialized policy
round trips.
Use corpus or fuzz-style tests when untrusted parsing, deserialization, path
handling, or protocol boundaries become broad enough that examples no longer
cover the input space.
## Current Plugin Goldens
`tests/goldens/plugin` is the checked-in golden surface for static plugin
configuration and manifest contracts. `tests/plugin_schema.rs` is part of the
normal Cargo integration suite and now verifies:
- every JSON fixture in the directory has an explicit coverage label
- every required plugin golden coverage label is present
- config and manifest schemas accept valid fixtures and reject invalid fixtures
- both schemas pin `CURRENT_WIT_WORLD`
- both schemas expose only reviewed capability names
- config schema runtime-limit caps match the Rust host caps
- config schema proposal review modes match the reviewed Rust vocabulary
These goldens cover schema and policy drift.
`tests/goldens/plugin/runtime-trace` is the checked-in replay surface for
redacted runtime trace output. The fixture format is one
`PluginRuntimeTraceEvent` display line per event. The focused trace test builds
events from existing pending batches, workspace I/O reports, workspace
observation task completions, update discard reports including task-request
discard counts, operational events, and proposal receipts, then compares the
rendered lines without serializing `Debug`.
## Current Runtime Fixtures
Real Wasm component fixture coverage exists for:
- accepted and denied `status.publish`
- accepted and denied `buffer.observe`
- accepted and denied `buffer.propose_edit`
- accepted `workspace.artifact_write`
- task-based `workspace.observe` start/schedule/poll/take/drop through the
runtime and host-owned task queue
- task-based `workspace.observe` revocation cleanup through the runtime-owned
cancellation hook
- task-based `workspace.observe` shutdown cleanup through the runtime-owned
cancellation hook
- task-based `workspace.observe` late enqueue/execute rejection after
revocation
- task-based `workspace.observe` tombstoning for denied revalidation and
failed updates, including stale-handle alias regression coverage
- observe-then-propose execution through `PluginIntentQueue` and the buffer
owner
- runtime-produced `buffer.propose_edit` publication through
`PluginIntentQueue` and the buffer owner
- deterministic FIFO ordering across runtime-produced batches from multiple
plugins
- fuel exhaustion after a queued effect
- wall-clock timeout after a queued effect
- invalid canonical ABI UTF-8 after a queued effect
- oversized host-call payloads after a queued effect
- invalid resource handles after a queued effect
- stale handle generations after a queued effect
- invalid workspace artifact paths after queued workspace I/O
Remaining fixture gap:
- extend runtime trace goldens when new event classes, proposal owners, or
workspace outcomes are added
## Abuse Test Rule
Every failed guest update must discard pending effects and pending workspace
I/O before ECS publication or filesystem policy execution. Tests should assert
both the failure shape and the discarded-work counts.
Redaction assertions should check `Debug`, `Display`, and operational events
where those surfaces exist. The tests should use recognizable secret text so a
leak fails clearly.
## Capability Extension Checklist
Add one capability at a time. The same slice must update:
- WIT source and generated binding drift checks
- WIT resource, result, variant, enum, option, and flags shapes for new public
API contracts
- Rust capability vocabulary
- capability atom spelling and redacted shape projection
- capability atom authorization-ref path arity
- JSON schema and goldens
- manifest projection
- host authorization
- generated binding adaptation, Alma-owned validation, and redaction
- host import implementation
- tests for accepted, denied, malformed, oversized, and stale inputs as
relevant
- `docs/plugins/runtime.md`, `docs/plugins/security.md`, and this file
Do not expose raw ECS access, raw filesystem paths, process execution,
environment access, clipboard access, lifecycle shutdown, synthetic user input,
or arbitrary configuration mutation without a named capability and a reviewed
owner boundary.