Skip to main content

Module mem_budget

Module mem_budget 

Source
Expand description

Per-query memory budget accumulator (WS2).

The blunt row-count caps (MAX_SORT_ROWS, MAX_JOIN_ROWS) cannot stop a query that materializes a small number of very large rows, and they don’t cover GROUP BY hash tables or IN-list materialization at all. A crafted query could therefore OOM-kill the server process — fatal on AWS / Railway / Cloudflare where the process has a hard memory ceiling.

This module adds a lightweight byte-budget accumulator that each materialization point charges as it grows its buffer. When the running total would exceed the configured limit we return QueryError::MemoryLimitExceeded cleanly — no panic, no partial state.

§Why a thread-local accumulator

The read path runs concurrently behind Arc<RwLock<Engine>>: many threads call execute_powql_readonly(&self) at once. A single accumulator field on the Engine would (a) make Engine !Sync if it used Cell, and (b) be wrong even with an atomic, because concurrent queries would sum and reset each other’s totals. Each query, however, runs to completion synchronously on a single thread (spawn_blockingdispatch_queryexecute_powql*), so a thread-local running total is both correct and contention-free. The per-query limit is passed explicitly (it lives on the Engine as a plain usize, which is Copy/Sync).

Disk-spill (so over-budget queries still succeed) is explicitly deferred to Phase 3; for now over-budget is a clean error.

Structs§

EnterGuard
RAII guard returned by enter; decrements the reentrancy depth on drop.

Constants§

DEFAULT_QUERY_MEMORY_LIMIT
Default per-query memory budget: 256 MB. Plumbed from POWDB_QUERY_MEMORY_LIMIT by the server.

Functions§

charge
Charge bytes against the current thread’s running total, checking it against limit_bytes. Returns QueryError::MemoryLimitExceeded if this allocation would push the total over the limit. On error nothing is charged.
enter
Enter a budgeted statement frame. Returns an EnterGuard that decrements the depth on drop so the count stays correct even if execution unwinds via ?/panic. Only the outermost entry (depth 0 → 1) zeroes the accumulator; nested entries (e.g. the source query of a create_view/refresh_view recursively calling execute_powql) leave the outer frame’s charged bytes intact. This is a reentrancy guard rather than a save/restore because it is simpler — there is exactly one running total to protect and the guard makes the “outermost statement owns the reset” rule self-evident at the call site.
estimate_row_size
Estimate the in-memory footprint of a fully materialized row, including the Vec<Value> backing allocation.
estimate_value_size
Estimate the in-memory footprint of a single Value, including the heap allocation behind Str/Bytes. The estimate counts the enum slot plus any owned heap bytes — it is intentionally an over-approximation so the guard trips slightly early rather than slightly late.