pub struct TensorWasmMemoryCreator { /* private fields */ }Expand description
A MemoryCreator that hands out TensorWasmLinearMemory instances.
Wrap in Arc and pass to wasmtime::Config::with_host_memory.
Implementations§
Source§impl TensorWasmMemoryCreator
impl TensorWasmMemoryCreator
Sourcepub fn new(device_id: DeviceId) -> Self
pub fn new(device_id: DeviceId) -> Self
Construct without an underlying pool. New memories allocate fresh
UnifiedBuffers on every new_memory call. Uses the default
MemoryCreatorConfig (per-instance cap =
HARD_MAX_LINEAR_MEMORY_BYTES).
Sourcepub fn with_config(device_id: DeviceId, config: MemoryCreatorConfig) -> Self
pub fn with_config(device_id: DeviceId, config: MemoryCreatorConfig) -> Self
Construct without an underlying pool, with a custom
MemoryCreatorConfig (e.g. a tighter per-instance cap for
multi-tenant deployments). The configured cap is clamped to
HARD_MAX_LINEAR_MEMORY_BYTES — see
MemoryCreatorConfig::effective_max_linear_memory_bytes.
Sourcepub fn with_tenant_context(
device_id: DeviceId,
tenant_ctx: Arc<TenantContext>,
) -> Self
pub fn with_tenant_context( device_id: DeviceId, tenant_ctx: Arc<TenantContext>, ) -> Self
Construct a tenant-aware creator that accounts every fresh
UnifiedBuffer against the tenant’s GPU memory cap.
Roadmap feature #8 builder: holds an Arc<TenantContext> and
routes the no-pool / pool-fallback allocation paths through
UnifiedBuffer::new_on_with_tenant_context. On a cap
violation the underlying error is
tensor_wasm_core::error::TensorWasmError::GpuMemoryExhausted;
it surfaces here as a String because Wasmtime’s
MemoryCreator::new_memory returns Result<_, String>.
Pool-carved memories are intentionally unmetered — see the
note on MemoryCreatorState::tenant_ctx for the
rationale and the v0.4 follow-up.
Sourcepub fn with_pool_and_tenant_context(
device_id: DeviceId,
pool: Arc<UnifiedMemoryPool>,
tenant_ctx: Arc<TenantContext>,
) -> Self
pub fn with_pool_and_tenant_context( device_id: DeviceId, pool: Arc<UnifiedMemoryPool>, tenant_ctx: Arc<TenantContext>, ) -> Self
Tenant-aware variant of Self::with_pool.
Behaves like Self::with_pool for the pool-carve hot path
(no extra counter traffic) and like
Self::with_tenant_context for the fallback fresh-allocation
path.
Sourcepub fn with_pool(device_id: DeviceId, pool: Arc<UnifiedMemoryPool>) -> Self
pub fn with_pool(device_id: DeviceId, pool: Arc<UnifiedMemoryPool>) -> Self
Construct with a pre-allocated pool. new_memory carves out of this
slab; on exhaustion it falls back to a fresh UnifiedBuffer so a
short slab does not turn into a fatal error.
Lifetime contract. Pool-backed linear memories form an
all-or-nothing batch: the caller must drop every Wasm instance handed
out by this creator before dropping the last Arc<UnifiedMemoryPool>
reference and calling UnifiedMemoryPool::reset. See new_memory
for the unsafe rationale.
Sourcepub fn with_pool_and_config(
device_id: DeviceId,
pool: Arc<UnifiedMemoryPool>,
config: MemoryCreatorConfig,
) -> Self
pub fn with_pool_and_config( device_id: DeviceId, pool: Arc<UnifiedMemoryPool>, config: MemoryCreatorConfig, ) -> Self
Construct with both a pre-allocated pool and a custom
MemoryCreatorConfig.
Sourcepub fn pool(&self) -> Option<&Arc<UnifiedMemoryPool>>
pub fn pool(&self) -> Option<&Arc<UnifiedMemoryPool>>
The underlying pool, if one was provided at construction.
Sourcepub fn config(&self) -> &MemoryCreatorConfig
pub fn config(&self) -> &MemoryCreatorConfig
The tunable configuration handed to this creator.
Trait Implementations§
Source§impl Clone for TensorWasmMemoryCreator
impl Clone for TensorWasmMemoryCreator
Source§fn clone(&self) -> TensorWasmMemoryCreator
fn clone(&self) -> TensorWasmMemoryCreator
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read moreSource§impl Default for TensorWasmMemoryCreator
impl Default for TensorWasmMemoryCreator
Source§impl MemoryCreator for TensorWasmMemoryCreator
impl MemoryCreator for TensorWasmMemoryCreator
Source§fn new_memory(
&self,
_ty: MemoryType,
minimum: usize,
maximum: Option<usize>,
reserved_size_in_bytes: Option<usize>,
guard_size_in_bytes: usize,
) -> Result<Box<dyn LinearMemory>, String>
fn new_memory( &self, _ty: MemoryType, minimum: usize, maximum: Option<usize>, reserved_size_in_bytes: Option<usize>, guard_size_in_bytes: usize, ) -> Result<Box<dyn LinearMemory>, String>
LinearMemory object from the specified parameters. Read moreAuto Trait Implementations§
impl !RefUnwindSafe for TensorWasmMemoryCreator
impl !UnwindSafe for TensorWasmMemoryCreator
impl Freeze for TensorWasmMemoryCreator
impl Send for TensorWasmMemoryCreator
impl Sync for TensorWasmMemoryCreator
impl Unpin for TensorWasmMemoryCreator
impl UnsafeUnpin for TensorWasmMemoryCreator
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
Source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more