Skip to main content

Module limits

Module limits 

Source
Expand description

Bounds on caller-supplied SPARQL, checked before the parser sees a byte (ledger #915).

oxigraph’s SPARQL parser (spargebra, a peg grammar) and its evaluator (sparopt, spareval) are recursive, so a query’s shape is a claim on the stack of whatever thread parses it. Running out is not an error a caller gets back: Rust aborts the whole process on a stack overflow, on any thread. SELECT * WHERE { FILTER(((…1…))) } with ~1000 parentheses took down a host through its HTTP door, and the anonymous grant held the read authority that reached the parse.

Two layers, because neither is enough alone:

  1. check_sparql refuses, never truncates. A query or update over MAX_SPARQL_BYTES, or that nests deeper than MAX_SPARQL_NESTING — brackets, and runs of the one prefix operator spargebra recurses on, ! — is refused with a typed InvalidArgument naming the bound, before anything is parsed. Those are the cheapest ways to recurse (ONE byte a level; ~2.3 KiB of stack a level for a bracket in a release build), so this is the check that turns a 3 KB query that killed a host into a refusal.
  2. on_sparql_stack runs the parse AND the evaluation on a thread whose stack is sized from the query’s length. Nesting is not the only recursion: measured on a 2 MiB thread in a release build, 1,879 triple patterns, 2,087 FILTERs, a 2,086-way || or 3,283 + terms all overflow — and those are shapes a generated query can legitimately have, so no lexical bound can refuse them without refusing real work. What bounds them is the byte bound times the stack each byte can cost, and the thread is where that budget lives: STACK_PER_BYTE per byte above a STACK_BASE floor. A thread’s stack is reserved address space, committed only as it is touched, so an ordinary query costs the spawn and nothing else.

⚠ The second layer is a RELEASE-build guarantee. A debug build spends ~20–50× the stack per level (a 2 MiB thread holds 47 + terms when evaluating, unoptimized), so a debug host can still be overflowed by a long enough operator chain inside the byte bound. The bracket bound holds in both: a debug build at MAX_SPARQL_NESTING of the costliest bracket (STR( calls, ~60 KiB a level) needs ~4 MiB, well inside STACK_BASE.

⚠ And on wasm there is no thread: on_sparql_stack runs inline there, so only the first layer applies, on whatever stack the host gave the module.

⚠ Neither layer bounds TIME. The evaluator is quadratic in an operator chain’s length: 40,000 ||1 terms (120 KB) take ~30 s of a core in a release build and 80,000 more than two minutes, and long property paths, long BGPs and deeply nested collections are slow well before they are deep. This module is about the stack — an abort takes down every request at once — and leaves a query budget to the host.

§The scan is exact about what it skips, and that took an automaton

The obvious scan — count (, {, [, skipping strings, <IRI>s and # comments — is wrong in a way an attacker can use, because SPARQL’s lexing of < depends on where the parser is: in a triple pattern <…> is an IRI, but in an expression ?a<… is a less-than. spargebra accepts < + anything-but-> + > as an IRI only where an IRI is allowed and the text inside is a valid IRI; elsewhere it is an operator, and the “IRI” is code. So ( ?a<'> ) ' && ( ?a<'> ) ' && (… reads to a skip-the-IRI scan as one level, while the parser sees (?a < "> ) " && (?a < "> ) " && (… — a string it opens INSIDE what the scan skipped — and nests once per repetition. A # inside such a token does the same with a comment.

So the scan follows every reading the grammar allows, as a small set of branches, and refuses on the deepest. At a < it forks: an IRI (when the text up to the next > could be one), a << triple-term opener (when one follows), and a less-than — the last only when the innermost open bracket is (, because every SPARQL expression sits directly inside parentheses, so that is the only place a less-than can occur. Branches in the same lexical state with the same open brackets are merged, so an ordinary query runs as one branch and the whole scan stays linear. A closer pops only the opener it matches, so no reading can be made to look shallower than it is; and a query that keeps more than MAX_SCAN_BRANCHES readings alive at once is refused as too ambiguous to bound.

⚠ One more reading lives OUTSIDE the grammar: spargebra’s standard-unicode-escaping feature rewrites \uXXXX and \UXXXXXXXX anywhere in the text before tokenizing, so \u0028 is a (. oxigraph does not enable it, but feature unification is global and a sibling could; when the text contains such an escape, the scan also runs over the unescaped text and refuses on either.

Constants§

MAX_SCAN_BRANCHES
How many simultaneous readings the scan follows before refusing the text as too ambiguous to bound. An ordinary query keeps one; it takes a < inside parentheses followed by text that could be an IRI containing ', # or a bracket to make two.
MAX_SPARQL_BYTES
The most bytes a SPARQL query or update may carry: 1 MiB.
MAX_SPARQL_NESTING
The deepest a SPARQL query or update may nest (, {, [ and << — with a run of ! counting one level for each — 64.
STACK_BASE
The stack every SPARQL thread gets before the per-byte share: 16 MiB.
STACK_PER_BYTE
The stack reserved per byte of query, above STACK_BASE: 512 bytes.

Functions§

check_sparql
Refuse a SPARQL query or update that exceeds MAX_SPARQL_BYTES or nests deeper than MAX_SPARQL_NESTING, naming the argument arg and the bound. Never truncates.
on_sparql_stack
Run work — a parse and everything that evaluates what it parsed — on a thread whose stack is sized for text (see STACK_BASE and STACK_PER_BYTE), and wait for it.