Module startup_memory 

Device-aware startup policy. Costs come from the compiled execution plan, including its provider workspaces and physical storage layouts.

Structs

DeviceMemorySnapshot

Sampled before model upload. Available bytes are additional allocatable memory, already accounting for other processes and platform ceilings.

StartupMemoryPlan

StartupMemoryRequest

StartupResourceLimits

Enums

StartupWorkload

Independent capacity probes, not a promise that every admitted sequence can simultaneously occupy the maximum context. Runtime admission remains responsible for the actual mixture of live requests.

Functions

fit_startup_resources

Find supported limits using a monotone upper bound supplied by the actual compiled plan. No tensors are allocated by this search. Provider errors are errors, never interpreted as evidence that a smaller request will work.