tensor-wasm-tenant 0.3.8

Multi-tenant CUDA context management (MPS-backed or per-context fallback).
Documentation

tensor-wasm-tenant

Multi-tenant CUDA context management for Craton TensorWasm. Provides TenantContext, which bundles a per-tenant CUDA context, stream, and memory pool, and TenantRegistry, which maps TenantId values to live contexts and handles their lifecycle. Designed to ride on NVIDIA MPS when available, falling back to per-context isolation otherwise so workloads from different tenants never observe one another's GPU state.

Feature flags

Flag Default Description
cuda no Use real CUDA contexts (vs in-process stub for unit tests).
loom no Swap std::sync::atomic::AtomicU64 for loom::sync::atomic::AtomicU64 on the consume_bytes / release_bytes CAS hot path so tests/loom_consume_release.rs can exhaustively explore the two-thread interleavings. Pure model-checking — do not enable in production builds.
strict-cap-binding yes Gates the typed *_strict admin error variants (e.g. RegistryError::CapabilityFromForeignRegistry) and the cap_binding_strict integration test. It no longer toggles any security behaviour — capability-to-registry binding is now unconditional (see Cap binding). Kept in default so the *_strict surface is available out of the box.

NVIDIA MPS detection is unconditional — there is no feature flag for it. At runtime TenantRegistry::mps_or_fallback probes for the MPS control daemon (honouring $CUDA_MPS_PIPE_DIRECTORY, else /tmp/nvidia-mps) and returns MpsDecision::Mps when present, falling back to per-tenant contexts otherwise.

See docs/BUILD.md for the project-wide flag taxonomy.

Cap binding

TenantRegistry::new() mints a RegistryAdminCapability; every successful register_with_capability call mints a per-tenant TenantCapability. Both gate the hot-path mutation methods (admin enumeration / eviction for the former, per-tenant consume_bytes_with_capability / release_bytes_with_capability for the latter).

Binding is unconditional — capabilities are always registry-scoped. Every cap carries an Arc<()> token cloned from the minting registry's per-instance allocation, and every admin / quota check compares with Arc::ptr_eq. A cap minted by registry A is therefore rejected by registry B even when both name the same TenantId, and forged/foreign admin caps are refused — whether or not the strict-cap-binding feature is enabled, including in a release build with default-features = false. Two independent registries running in the same process are cleanly separated by construction; embedders no longer have to keep registry handles per-trust-domain to get isolation. Foreign-cap quota calls surface as [TensorWasmError::TenantIsolationViolation].

The strict-cap-binding feature (on by default) no longer toggles this security behaviour. It only gates the typed *_strict admin variants, which surface a foreign cap as [RegistryError::CapabilityFromForeignRegistry] rather than the generic refusal, plus the cap_binding_strict integration test.

Per-tenant quotas

TenantContext carries two independent quotas, each set through a chained TenantContextBuilder method and each enforced by a lock-free CAS-loop counter on the context. CPU and GPU caps are separate so a host-RAM exhaustion cannot share a budget with a GPU exhaustion.

Setter Counter Refusal error Default Enforcement
with_memory_quota_bytes(bytes) bytes_in_use (CPU / host-side) TensorWasmError::MemoryExhausted 8 GiB (DEFAULT_QUOTA_BYTES) In-process.
with_gpu_memory_bytes_cap(bytes) gpu_bytes_in_use (GPU-side) TensorWasmError::GpuMemoryExhausted None (operator trust) In-process today (v0.3.7); v0.4 also pins a driver-level cap via cuMemPoolSetAttribute(CU_MEMPOOL_ATTR_RELEASE_THRESHOLD, ...). See docs/GPU-QUOTAS.md.
with_recorded_cuda_mem_pool_quota(bytes) (none) (none — informational) None Recorded only; forward-compat hook for the v0.4 cust-successor migration.

The GPU side of the quota is consumed and released through the allocator path in tensor-wasm-mem — TensorWasmMemoryCreator::with_tenant_context / with_pool_and_tenant_context route every fresh UnifiedBuffer through consume_gpu_bytes (on allocation) and release_gpu_bytes (on Drop). Pool-carved memories share one slab allocation and are intentionally unmetered; see docs/GPU-QUOTAS.md for the rationale.

Dependencies

Internal workspace crates:

  • tensor-wasm-core — shared TenantId / InstanceId newtypes and the TensorWasmError enum returned by quota enforcement.

External crates (pinned at workspace root):

  • tokio — async runtime for context lifecycle tasks.
  • thiserror — derive macro for tenant-level errors.
  • tracing — structured spans/events for context create/destroy and the underflow / pop-failure warnings.
  • dashmap — concurrent map of TenantId to Arc<TenantContext>; combined with the AtomicU64 counters on TenantContext, this is the entirety of the registry's concurrency story — there are no mutexes on the hot path.
  • cust (optional, behind the cuda feature) — CUDA driver-API bindings; provides the primary-context API used by ContextIsolated tenants.