Skip to main content

rete_core/
card_queries.rs

1//! The standardized exploration query library (Design B of `dev/pyramid-dev.md`).
2//!
3//! A vetted set of starter SPARQL queries, **auto-instantiated at build** with a
4//! dataset's real vocabulary (`{{TOP_CLASS}}`/`{{TOP_PRED}}`/`{{LABEL_PRED}}`/…)
5//! drawn from the enriched [`DatasetCard`] profile, and emitted **only when the
6//! required signal is present** (no geo query without geometry, no time query
7//! without a time predicate) so the shipped set always returns rows. Every query
8//! carries its own `PREFIX` block (the engine injects none) and is tagged with
9//! the cheapest [`Tier`] that can answer it.
10//!
11//! Bodies are **graph-scope aware** (`GraphScope`): a dataset whose statements
12//! live entirely in named graphs (`triple_count == 0`, `named_graph_count > 0`)
13//! gets `GRAPH ?g`-scoped bodies — a bare `{ ?s ?p ?o }` there can only ever
14//! return zero rows — and a dataset with data in **both** the default graph and
15//! named graphs gets `UNION`-scoped overview bodies so neither half is silently
16//! hidden.
17//!
18//! # Why a generated query may not return zero rows
19//!
20//! A starter query that answers nothing is worse than no starter query: the
21//! reader concludes the **file** is broken. So every template carries a
22//! `NonEmpty` claim — the reason its emitted form cannot come back empty —
23//! and `check_body` enforces the one rule that claim rests on:
24//!
25//! > A body may conjoin substituted vocabulary only when the card **proves the
26//! > pieces co-occur**.
27//!
28//! Presence is not co-occurrence. `{{TOP_CLASS}}` is the class with the most
29//! instances and `{{LABEL_PRED}}` the most-used labelling predicate; each is
30//! certainly *present*, and instances of that class need never carry that
31//! predicate. Conjoining them — which `lb-labels` did — is a guaranteed-empty
32//! query whenever the two peaks fall on different parts of the graph, as they
33//! do on `mtg` (top class `mtg:Ruling`, which has no `schema:name`) and on
34//! `hugging-face` (top class `hf:Model`; `rdfs:label` appears only on the
35//! embedded TBox terms). The fix is not to check harder downstream but to make
36//! the conjunction unrepresentable: capabilities that appear together in a body
37//! must be **jointly derived** (`Cap::joint_with`) — chosen *because* they
38//! meet, with a `class_links` row as the witness.
39//!
40//! Where the card genuinely cannot decide emptiness, the template says so
41//! (`NonEmpty::Undecidable`) instead of pretending; `provably_empty` is the
42//! last gate, dropping a template the card can prove would answer nothing.
43//!
44//! # Why a generated query may not cost more than a starter should
45//!
46//! The other half of "a starter query that answers nothing is worse than
47//! none" is a starter query that does not finish. On the OpenCitations sample
48//! (18.9 M quads, 1.5 M named graphs, 1.07 GB) eight of the old bodies scanned
49//! the whole file — `ov-triples`, `ov-pred-*`, `top-*-hubs`, `top-dangling`,
50//! `ng-*` — and every one was killed in a 4 GiB client; `top-dangling` alone
51//! peaked at 8.7 GiB. A greeting query is the first thing a newcomer runs, often
52//! in a browser tab, so its cost has to be bounded by construction, not by the
53//! size of the dataset it happens to greet.
54//!
55//! Every template therefore declares what its body **reads** (`Scan`), and the
56//! card's own counts turn that into a number of statements before the body is
57//! written. Within [`STARTER_SCAN_BUDGET`] the exact body ships, unchanged. Past
58//! it the template ships its **bounded** form (`Bounded`) — the same question
59//! over a fixed-size sample (`LIMIT` inside a sub-select, which the engine
60//! evaluates lazily), or a weaker question with an exact bounded answer — and a
61//! template with no honest bounded form is not shipped at all: those are the
62//! questions (`how many statements?`, `which predicates, how often?`, `which
63//! hubs?`) the card already answers from its own precomputed profile, index-free.
64//! The build then measures every emitted query through a byte budget
65//! ([`STARTER_MAX_BYTES`]) and drops whatever still reads more, so the bound is
66//! measured, not only estimated.
67//!
68//! This is pure serde generation — no format change, no I/O, no threads. It
69//! lives in `rete-core` (it used to live in the binary-only `rete-cli`, where no
70//! client could reach it — #152) so the CLI, the browser builder and every
71//! language binding instantiate one query library from one table.
72
73use serde::Serialize;
74
75use crate::card_derive::{
76    DatasetCard, ExampleQuery, Tier, CARD_TOP_N, GEO_ASWKT, GEO_HASGEOMETRY,
77    LINK_PREDICATES as LINK_PREDICATE_IRIS, O_LITERAL, WGS_LAT as WGS_LAT_IRI,
78    WGS_LONG as WGS_LONG_IRI,
79};
80use crate::RDF_TYPE;
81
82/// Prepended to every generated query — the engine parses with **zero** implicit
83/// prefixes (`Query::parse(q, None)`), so an undeclared prefix is a parse error.
84const PREFIXES: &str = "\
85PREFIX rdf:  <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
86PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
87PREFIX owl:  <http://www.w3.org/2002/07/owl#>
88PREFIX void: <http://rdfs.org/ns/void#>
89PREFIX dct:  <http://purl.org/dc/terms/>
90PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
91PREFIX geo:  <http://www.opengis.net/ont/geosparql#>
92PREFIX geof: <http://www.opengis.net/def/function/geosparql/>
93PREFIX wgs:  <http://www.w3.org/2003/01/geo/wgs84_pos#>
94PREFIX time: <http://www.w3.org/2006/time#>
95PREFIX xsd:  <http://www.w3.org/2001/XMLSchema#>";
96
97/// A capability the dataset must offer for a template to be emitted (and, for the
98/// string-valued ones, the vocabulary substituted into its placeholder).
99#[derive(Debug, Clone, Copy, PartialEq, Eq)]
100enum Cap {
101    TopClass,
102    TopPred,
103    /// The most populous class the card can **prove** carries [`Cap::LabelPred`].
104    LabeledClass,
105    LabelPred,
106    /// The most frequent predicate the card can **prove** has a non-literal
107    /// object — the only kind a path query can walk past one hop.
108    ObjectPred,
109    TimePred,
110    NumPred,
111    BaseIri,
112    /// An IRI the card records that lies **outside** [`Cap::BaseIri`] — the
113    /// witness that "links pointing out of this dataset" has something to find.
114    ExternalIri,
115    GeoLatLong,
116    /// The path from a subject to its WKT geometry, as the data actually shapes
117    /// it (`geo:asWKT` direct, or through `geo:hasGeometry`).
118    WktPath,
119    /// The query box for the geometry template.
120    Bbox,
121    NamedGraphs,
122    Link,
123    HasLiterals,
124    /// At least one statement in this graph is **about a statement** — its
125    /// subject is a quoted triple. The gate for the bodies that write
126    /// `<< ?s ?p ?o >>` in subject position with nothing substituted into it.
127    AnnotatedStatements,
128    /// The most frequent predicate that annotates a statement — the outer
129    /// predicate of `<< s p o >> :recordedBy :who`.
130    AnnotationPred,
131    /// The predicate **inside** a quoted triple that [`Cap::AnnotationPred`]
132    /// was seen annotating. Jointly derived with it (`annotated_statement` is
133    /// the witness), so a body may pin both.
134    AnnotatedPred,
135    /// The most frequent predicate whose **object** is a quoted triple —
136    /// `:claim :states << s p o >>`, the other position a triple term stands
137    /// in, and a different query from the annotation one.
138    QuotingPred,
139}
140
141/// Every capability, in declaration order — the substitution table and the
142/// placeholder audit both walk it, so a new `Cap` cannot be half-wired.
143const ALL_CAPS: &[Cap] = &[
144    Cap::TopClass,
145    Cap::TopPred,
146    Cap::LabeledClass,
147    Cap::LabelPred,
148    Cap::ObjectPred,
149    Cap::TimePred,
150    Cap::NumPred,
151    Cap::BaseIri,
152    Cap::ExternalIri,
153    Cap::GeoLatLong,
154    Cap::WktPath,
155    Cap::Bbox,
156    Cap::NamedGraphs,
157    Cap::Link,
158    Cap::HasLiterals,
159    Cap::AnnotatedStatements,
160    Cap::AnnotationPred,
161    Cap::AnnotatedPred,
162    Cap::QuotingPred,
163];
164
165impl Cap {
166    /// Stable key recorded in `ExampleQuery.requires`.
167    fn key(self) -> &'static str {
168        match self {
169            Cap::TopClass => "TOP_CLASS",
170            Cap::TopPred => "TOP_PRED",
171            Cap::LabeledClass => "LABELED_CLASS",
172            Cap::LabelPred => "LABEL_PRED",
173            Cap::ObjectPred => "OBJECT_PRED",
174            Cap::TimePred => "TIME_PRED",
175            Cap::NumPred => "NUM_PRED",
176            Cap::BaseIri => "BASE_IRI",
177            Cap::ExternalIri => "EXTERNAL_IRI",
178            Cap::GeoLatLong => "GEO_LATLONG",
179            Cap::WktPath => "WKT_PATH",
180            Cap::Bbox => "BBOX",
181            Cap::NamedGraphs => "NAMED_GRAPHS",
182            Cap::Link => "LINK",
183            Cap::HasLiterals => "HAS_LITERALS",
184            Cap::AnnotatedStatements => "ANNOTATED_STATEMENTS",
185            Cap::AnnotationPred => "ANNOTATION_PRED",
186            Cap::AnnotatedPred => "ANNOTATED_PRED",
187            Cap::QuotingPred => "QUOTING_PRED",
188        }
189    }
190
191    /// The `{{PLACEHOLDER}}` this capability's value is substituted into, for
192    /// the string-valued ones. `None` for the pure gates (they have no text in
193    /// the body — they only decide whether it is emitted).
194    fn placeholder(self) -> Option<&'static str> {
195        Some(match self {
196            Cap::TopClass => "{{TOP_CLASS}}",
197            Cap::TopPred => "{{TOP_PRED}}",
198            Cap::LabeledClass => "{{LABELED_CLASS}}",
199            Cap::LabelPred => "{{LABEL_PRED}}",
200            Cap::ObjectPred => "{{OBJECT_PRED}}",
201            Cap::TimePred => "{{TIME_PRED}}",
202            Cap::NumPred => "{{NUM_PRED}}",
203            Cap::BaseIri => "{{BASE_IRI}}",
204            Cap::WktPath => "{{WKT_PATH}}",
205            Cap::Bbox => "{{BBOX_POLYGON}}",
206            Cap::AnnotationPred => "{{ANNOTATION_PRED}}",
207            Cap::AnnotatedPred => "{{ANNOTATED_PRED}}",
208            Cap::QuotingPred => "{{QUOTING_PRED}}",
209            Cap::ExternalIri
210            | Cap::GeoLatLong
211            | Cap::NamedGraphs
212            | Cap::Link
213            | Cap::HasLiterals
214            | Cap::AnnotatedStatements => return None,
215        })
216    }
217
218    /// The capabilities this one is chosen **together with** — the card holds a
219    /// `class_links` row proving they meet on the same subjects, so a body may
220    /// conjoin them.
221    ///
222    /// This is the whole safety property. Every other pair of substitutions is
223    /// picked from a different ranking (most instances / most statements / most
224    /// labels), and two independent maxima need not describe the same entity;
225    /// see the module docs. The relation is symmetric — `check_body` checks
226    /// both directions.
227    fn joint_with(self) -> &'static [Cap] {
228        match self {
229            // `LABELED_CLASS` *is* "the class that carries `LABEL_PRED`". The
230            // one pair the card can witness, and the one the label queries need.
231            Cap::LabeledClass => &[Cap::LabelPred],
232            Cap::LabelPred => &[Cap::LabeledClass],
233            // `ANNOTATED_PRED` *is* "the predicate inside the statements that
234            // `ANNOTATION_PRED` was seen annotating". `signals
235            // .annotated_statement` holds the two as they occurred together on
236            // one statement, so a body may write them into one pattern. Chosen
237            // apart they would be the two-maxima shape again: the most-used
238            // annotation predicate need not be the one attached to the
239            // most-annotated kind of statement.
240            Cap::AnnotatedPred => &[Cap::AnnotationPred],
241            Cap::AnnotationPred => &[Cap::AnnotatedPred],
242            _ => &[],
243        }
244    }
245}
246
247/// **The** co-occurrence witness: does the card record that some subject of
248/// `class` also carries `pred`?
249///
250/// A `class_links` row is positive evidence — the quotient counts real
251/// statements, so a row exists only because a subject classified as `class`
252/// really made a `pred` statement. Both the generator (choosing
253/// `Cap::LabeledClass`) and the audit ([`audit`]) ask this one question, so
254/// "does the card prove these meet?" has exactly one implementation.
255fn class_carries(card: &DatasetCard, class: &str, pred: &str) -> bool {
256    card.class_links
257        .iter()
258        .any(|l| l.s_class == class && l.predicate == pred)
259}
260
261/// The RDF-star co-occurrence witness, in the one place both halves read it:
262/// `(the predicate inside a quoted triple, the predicate annotating it)` for a
263/// statement this graph was seen to **assert and annotate**.
264///
265/// It plays exactly the role [`class_carries`] plays for `LABELED_CLASS`. The
266/// derivation records the pair only when it matched an asserted statement
267/// against the annotations of that same statement, so a hit proves both halves
268/// of `qt-coverage`'s aggregate are non-zero — `?s p ?o` matches, and the
269/// `OPTIONAL` binds at least once. Both the generator (resolving
270/// [`Cap::AnnotatedPred`]) and [`audit`] ask through here, so "does the card
271/// prove these meet?" keeps having exactly one implementation.
272fn annotated_statement(card: &DatasetCard) -> Option<(&str, &str)> {
273    card.signals
274        .annotated_statement
275        .as_ref()
276        .map(|(quoted, about)| (quoted.as_str(), about.as_str()))
277}
278
279/// Why a template's emitted query cannot come back empty. Declared per template
280/// and enforced by `check_body` plus the fixture tests, which run every
281/// generated query against the very graph it was generated for.
282#[derive(Debug, Clone, Copy, PartialEq, Eq)]
283enum NonEmpty {
284    /// Any non-empty graph answers: the body has no substitution and no filter
285    /// that could exclude every statement.
286    AnyGraph,
287    /// An aggregate with no `GROUP BY` — SPARQL returns exactly one row even
288    /// over an empty solution sequence, so "zero rows" is not reachable. (The
289    /// row's values may still be unbound; see each template's note.)
290    Aggregate,
291    /// Every substitution came from a count the card recorded as `> 0`, and any
292    /// two that meet in the body are [jointly derived](Cap::joint_with).
293    Witnessed,
294    /// The card cannot decide it. The string is the reason, and it is the text
295    /// a reader deserves when the query does answer nothing.
296    Undecidable(&'static str),
297}
298
299/// How many statements a starter query's body may have to read before the
300/// generator swaps it for its `Bounded` form (or drops it).
301///
302/// Calibrated on the heaviest exact bodies, measured on the OpenCitations
303/// sample's 5.97 M-statement default graph: `top-class-graph` peaked at
304/// ~450 B of RSS per statement scanned and the old `top-dangling` at ~570 B, so
305/// two million statements keeps the worst of them near 1 GiB — inside a 4 GiB
306/// client with room for the reader's own caches — while a full scan of one
307/// permutation at this size is a few MB of index. A dataset at or under the
308/// budget keeps every exact body, byte for byte.
309pub const STARTER_SCAN_BUDGET: u64 = 2_000_000;
310
311/// The most bytes a starter query may read when the build (or `rete recard`)
312/// measures it, counted once the file is open — the measured backstop to
313/// [`STARTER_SCAN_BUDGET`]'s estimate. A query that asks for more is dropped
314/// from the card and the drop is recorded ("did not finish"). The open is not
315/// charged: it reads the section directories (~44 MB on a 26 GB file) whatever
316/// the query, a cost no starter can change. 64 MiB is ~3× the heaviest exact body on a
317/// budget-sized graph and ~40× the heaviest bounded one.
318pub const STARTER_MAX_BYTES: u64 = 64 << 20;
319
320/// What a template's body has to read — turned into a number of statements by
321/// [`scan_estimate`] from the card's own counts, and compared against
322/// [`STARTER_SCAN_BUDGET`].
323///
324/// An estimate, not a promise (a join's fan-out is not in the card), which is
325/// why the build measures every emitted query through [`STARTER_MAX_BYTES`]
326/// as well. It errs high: a term the card does not list is charged the
327/// largest count it could have.
328#[derive(Debug, Clone, Copy, PartialEq, Eq)]
329enum Scan {
330    /// The body bounds itself — a `LIMIT` over a pattern the engine evaluates
331    /// lazily, a single seeded lookup, an `ASK`.
332    Bounded,
333    /// Every statement in the body's graph scope.
334    Everything,
335    /// Every `rdf:type` statement.
336    Types,
337    /// Every statement of one substituted predicate.
338    Pred(Cap),
339    /// Every statement of these fixed predicates.
340    Preds(&'static [&'static str]),
341    /// The instances of one substituted class.
342    Class(Cap),
343    /// The instances of one substituted class and every statement they make.
344    ClassStar(Cap),
345    /// Every statement in a named graph.
346    NamedGraphs,
347    /// Every statement whose subject is a quoted triple.
348    Annotations,
349}
350
351/// The form a template takes when its [`Scan`] is over the budget.
352///
353/// Each bounded body reads a fixed-size slice: a `LIMIT`ed sub-select (the
354/// engine stops scanning when it is full) feeding the template's question, or
355/// a weaker question whose exact answer is a handful of lookups. The title and
356/// question change with it, so the card never presents a sample as the whole
357/// graph. Default-graph shaped like a [`Fallback`]: sound on a mixed file (its
358/// profile is the default graph's), and skipped on a named-graph-only one
359/// unless `named_body` is given.
360struct Bounded {
361    title: &'static str,
362    question: &'static str,
363    body: &'static str,
364    named_body: Option<&'static str>,
365    requires: &'static [Cap],
366    nonempty: NonEmpty,
367}
368
369/// One query template: a body with `{{PLACEHOLDER}}`s, the tier that answers it,
370/// and the capabilities it needs (which also gate emission).
371struct Template {
372    id: &'static str,
373    title: &'static str,
374    dimension: &'static str,
375    question: &'static str,
376    /// The default-graph body — used verbatim when the data lives in the
377    /// default graph, so classic (graph-free) cards are byte-identical to
378    /// before scope awareness existed.
379    body: &'static str,
380    /// Body when **every** statement lives in a named graph and the default
381    /// graph is empty. On such a file the default-graph `body` can only return
382    /// zero rows, so a template without a named-scope body is **skipped**
383    /// (unless it is already `GRAPH`-native, i.e. requires [`Cap::NamedGraphs`]).
384    named_body: Option<&'static str>,
385    /// Body when **both** the default graph and named graphs hold data.
386    /// `None` falls back to `body`: sound — the profile that instantiated it
387    /// was derived from the (non-empty) default graph — but scoped to it, so
388    /// the whole-dataset templates (counts, histograms, hubs) provide a
389    /// `UNION`-scoped variant instead of silently hiding the named half.
390    mixed_body: Option<&'static str>,
391    tier: Tier,
392    requires: &'static [Cap],
393    /// Why the emitted query cannot return zero rows.
394    nonempty: NonEmpty,
395    /// A weaker body for when `requires` is **not** met: the template degrades
396    /// to a still-answerable question instead of vanishing. Without it a
397    /// capability that only sharpens a query (`LABELED_CLASS`) would also be
398    /// able to delete it — on `hugging-face` the label query would disappear
399    /// even though the dataset has labels, which is not an improvement over
400    /// shipping one that returns nothing.
401    fallback: Option<Fallback>,
402    /// Decides emptiness from the card, in **default-graph scope only** (the
403    /// card profiles the default graph, so its counts witness nothing about a
404    /// `GRAPH`-scoped body). `Some(f)` and `f(card)` true ⇒ the query would
405    /// answer nothing ⇒ it is not emitted. The generator's last gate.
406    provably_empty: Option<fn(&DatasetCard) -> bool>,
407    /// Is `Template::provably_empty` an **equivalence** in default-graph
408    /// scope — does `false` prove the query answers, and not merely fail to
409    /// prove it empty?
410    ///
411    /// True only where the card's precompute *is* this query's result:
412    /// `top-in-hubs` is `card.in_hubs`, row for row and by construction (see
413    /// the in-degree tally in `derive_card_from`). `top-dangling` shares the
414    /// hook — no IRI object means nothing can dangle — without the converse, so
415    /// it stays honestly undecidable. Nothing in generation reads this; it is
416    /// what lets [`audit`] tell a query the card decides from one it does not,
417    /// instead of counting both as unknown.
418    hook_is_exact: bool,
419    /// What the body reads — see [`Scan`] and [`STARTER_SCAN_BUDGET`].
420    scan: Scan,
421    /// What ships instead when `scan` is over the budget. `None` ⇒ nothing
422    /// ships: the template is a question the card's own profile already
423    /// answers index-free (counts, predicate and class histograms, hubs,
424    /// datatypes, extents), so a sample of it would only be a worse copy.
425    bounded: Option<Bounded>,
426}
427
428/// The reduced form of a [`Template`], used when its full `requires` are not
429/// available. Default-graph shaped (no scope variants), so it is skipped on a
430/// named-graph-only file exactly like a template without a `named_body`.
431struct Fallback {
432    body: &'static str,
433    requires: &'static [Cap],
434    nonempty: NonEmpty,
435}
436
437/// The library. Order is fixed → deterministic output (folded into the hash).
438const TEMPLATES: &[Template] = &[
439    // --- Overview ---
440    Template {
441        id: "ov-one-row",
442        title: "One statement, now",
443        dimension: "overview",
444        question: "Return exactly one statement — did this file open and answer?",
445        // The smoke test (issue #153): guaranteed one row on ANY non-empty
446        // file. A COUNT can honestly return 0 (named-graph-only files) and
447        // read as failure; one concrete row is unambiguous.
448        body: "SELECT ?s ?p ?o WHERE { ?s ?p ?o } LIMIT 1",
449        named_body: Some("SELECT ?g ?s ?p ?o WHERE { GRAPH ?g { ?s ?p ?o } } LIMIT 1"),
450        mixed_body: Some(
451            "SELECT ?s ?p ?o WHERE { { ?s ?p ?o } UNION { GRAPH ?g { ?s ?p ?o } } } LIMIT 1",
452        ),
453        tier: Tier::Index,
454        requires: &[],
455        nonempty: NonEmpty::AnyGraph,
456        fallback: None,
457        provably_empty: None,
458        hook_is_exact: false,
459        scan: Scan::Bounded,
460        bounded: None,
461    },
462    Template {
463        id: "ov-triples",
464        title: "How many statements?",
465        dimension: "overview",
466        question: "How big is this graph — how many triples?",
467        body: "SELECT (COUNT(*) AS ?n) WHERE { ?s ?p ?o }",
468        named_body: Some("SELECT (COUNT(*) AS ?n) WHERE { GRAPH ?g { ?s ?p ?o } }"),
469        mixed_body: Some(
470            "SELECT (COUNT(*) AS ?n) WHERE { { ?s ?p ?o } UNION { GRAPH ?g { ?s ?p ?o } } }",
471        ),
472        tier: Tier::Summary,
473        requires: &[],
474        nonempty: NonEmpty::Aggregate,
475        fallback: None,
476        provably_empty: None,
477        hook_is_exact: false,
478        scan: Scan::Everything,
479        bounded: None,
480    },
481    Template {
482        id: "ov-pred-list",
483        title: "What relationships exist?",
484        dimension: "overview",
485        question: "Which predicates (relationships) appear in the data?",
486        body: "SELECT DISTINCT ?p WHERE { ?s ?p ?o }",
487        named_body: Some("SELECT DISTINCT ?p WHERE { GRAPH ?g { ?s ?p ?o } }"),
488        mixed_body: Some(
489            "SELECT DISTINCT ?p WHERE { { ?s ?p ?o } UNION { GRAPH ?g { ?s ?p ?o } } }",
490        ),
491        tier: Tier::Summary,
492        requires: &[],
493        nonempty: NonEmpty::AnyGraph,
494        fallback: None,
495        provably_empty: None,
496        hook_is_exact: false,
497        scan: Scan::Everything,
498        bounded: None,
499    },
500    Template {
501        id: "ov-pred-hist",
502        title: "How frequent is each relationship?",
503        dimension: "overview",
504        question: "How many statements use each predicate?",
505        // No ORDER BY — keeps it on the index-free summary fast path; sort client-side.
506        body: "SELECT ?p (COUNT(*) AS ?n) WHERE { ?s ?p ?o } GROUP BY ?p",
507        named_body: Some("SELECT ?p (COUNT(*) AS ?n) WHERE { GRAPH ?g { ?s ?p ?o } } GROUP BY ?p"),
508        mixed_body: Some(
509            "SELECT ?p (COUNT(*) AS ?n) WHERE { { ?s ?p ?o } UNION { GRAPH ?g { ?s ?p ?o } } } GROUP BY ?p",
510        ),
511        tier: Tier::Summary,
512        requires: &[],
513        // A GROUP BY yields no groups over an empty sequence — but every
514        // statement has a predicate, so a non-empty graph has at least one.
515        nonempty: NonEmpty::AnyGraph,
516        fallback: None,
517        provably_empty: None,
518        hook_is_exact: false,
519        scan: Scan::Everything,
520        bounded: None,
521    },
522    Template {
523        id: "ov-ask-pred",
524        title: "Does the top predicate occur?",
525        dimension: "overview",
526        question: "Is the most common predicate present at all?",
527        body: "ASK { ?s {{TOP_PRED}} ?o }",
528        named_body: Some("ASK { GRAPH ?g { ?s {{TOP_PRED}} ?o } }"),
529        mixed_body: None, // the profiled predicate lives in the default graph
530        tier: Tier::Summary,
531        requires: &[Cap::TopPred],
532        // One substitution, from a predicate the card counted; nothing to
533        // conjoin it with, so the ASK cannot be false.
534        nonempty: NonEmpty::Witnessed,
535        fallback: None,
536        provably_empty: None,
537        hook_is_exact: false,
538        scan: Scan::Bounded,
539        bounded: None,
540    },
541    // --- Identity & labels ---
542    Template {
543        id: "id-sample",
544        title: "A real entity to start from",
545        dimension: "identity",
546        question: "Give me one concrete entity of the most common type.",
547        body: "SELECT ?s WHERE { ?s a {{TOP_CLASS}} } LIMIT 1",
548        named_body: None,
549        mixed_body: None,
550        tier: Tier::Index,
551        requires: &[Cap::TopClass],
552        nonempty: NonEmpty::Witnessed,
553        fallback: None,
554        provably_empty: None,
555        hook_is_exact: false,
556        scan: Scan::Bounded,
557        bounded: None,
558    },
559    Template {
560        id: "lb-labels",
561        title: "Readable names for entities",
562        dimension: "labels",
563        question: "What are the human-readable names of some entities?",
564        //
565        // The class is `{{LABELED_CLASS}}`, NOT `{{TOP_CLASS}}`. Both name a
566        // class the dataset certainly has, and only one of them is proven to
567        // carry the label predicate this body joins it to. Picking the two
568        // maxima independently — the class with the most instances, the
569        // predicate with the most uses — ships a guaranteed-empty query
570        // whenever they fall on different parts of the graph:
571        //
572        //   mtg           top class `mtg:Ruling` (76,990), which has no
573        //                 `schema:name` at all — 0 rows, on a plain
574        //                 default-graph file.
575        //   hugging-face  top class `hf:Model` (2.9 M); `rdfs:label` occurs
576        //                 only on the 64 embedded ontology terms — 0 rows.
577        //
578        // `LABELED_CLASS` walks `card.classes` in descending instance order and
579        // takes the first one a `class_links` row proves carries the predicate.
580        // That row is a positive witness (some subject has both statements), so
581        // the join cannot be empty — and on a dataset where the top class *is*
582        // labelled (the common case: geoadmin, lombardi) it resolves to exactly
583        // the top class, leaving those cards byte-identical.
584        //
585        // Two other shapes were considered:
586        //
587        //   Drop the class (`?s {{LABEL_PRED}} ?label`). Simplest, and it also
588        //   cannot fail — but it answers a strictly weaker question, throws away
589        //   the join that is the single most reusable pattern a newcomer copies
590        //   out of a starter query, and would rewrite the query on every card
591        //   that is already correct.
592        //
593        //   Emit only when the top class is labelled. Silent and lossy: a
594        //   dataset whose labels live off the top class loses its label query
595        //   entirely, which is how a reader concludes there are no labels.
596        //
597        // So: prefer the typed form when the card can prove it, and fall back to
598        // the class-free form when it cannot — the case where `class_links` (top
599        // 100 rows) truncated the witness away, or where labels sit on untyped
600        // subjects. `hugging-face` takes the fallback and answers with 50 rows.
601        body: "SELECT ?s ?label WHERE { ?s a {{LABELED_CLASS}} ; {{LABEL_PRED}} ?label } LIMIT 50",
602        named_body: None,
603        mixed_body: None,
604        tier: Tier::Index,
605        requires: &[Cap::LabeledClass, Cap::LabelPred],
606        nonempty: NonEmpty::Witnessed,
607        fallback: Some(Fallback {
608            body: "SELECT ?s ?label WHERE { ?s {{LABEL_PRED}} ?label } LIMIT 50",
609            requires: &[Cap::LabelPred],
610            nonempty: NonEmpty::Witnessed,
611        }),
612        provably_empty: None,
613        hook_is_exact: false,
614        scan: Scan::Bounded,
615        bounded: None,
616    },
617    // --- Types ---
618    Template {
619        id: "ty-class-hist",
620        title: "What kinds of things, how many?",
621        dimension: "types",
622        question: "How many entities of each class are there?",
623        body: "SELECT ?c (COUNT(?s) AS ?n) WHERE { ?s a ?c } GROUP BY ?c ORDER BY DESC(?n) LIMIT 50",
624        named_body: None,
625        mixed_body: None,
626        tier: Tier::Index,
627        requires: &[Cap::TopClass],
628        // Gated on a class existing, and the body substitutes nothing.
629        nonempty: NonEmpty::Witnessed,
630        fallback: None,
631        provably_empty: None,
632        hook_is_exact: false,
633        scan: Scan::Types,
634        bounded: None,
635    },
636    Template {
637        id: "ty-entities",
638        title: "How many distinct typed entities?",
639        dimension: "types",
640        question: "How many distinct entities carry a type?",
641        body: "SELECT (COUNT(DISTINCT ?s) AS ?n) WHERE { ?s a [] }",
642        named_body: None,
643        mixed_body: None,
644        tier: Tier::Index,
645        requires: &[Cap::TopClass],
646        nonempty: NonEmpty::Aggregate,
647        fallback: None,
648        provably_empty: None,
649        hook_is_exact: false,
650        scan: Scan::Types,
651        bounded: None,
652    },
653    Template {
654        id: "ty-class-shape",
655        title: "What does the top class look like?",
656        dimension: "types",
657        question: "Which predicates describe the most common class?",
658        body: "SELECT ?p (COUNT(*) AS ?n) WHERE { ?s a {{TOP_CLASS}} ; ?p ?o } GROUP BY ?p ORDER BY DESC(?n)",
659        named_body: None,
660        mixed_body: None,
661        tier: Tier::Index,
662        requires: &[Cap::TopClass],
663        // One substitution, conjoined only with a wholly-variable pattern that
664        // the class assertion itself already satisfies (`?p ?o` matches the
665        // `rdf:type` statement) — so this is not the two-maxima shape.
666        nonempty: NonEmpty::Witnessed,
667        fallback: None,
668        provably_empty: None,
669        hook_is_exact: false,
670        scan: Scan::ClassStar(Cap::TopClass),
671        bounded: Some(Bounded {
672            title: "What does the top class look like? (1,000 instances)",
673            question: "Which predicates describe a sample of 1,000 instances of the most common class?",
674            body: "SELECT ?p (COUNT(*) AS ?n) WHERE { { SELECT ?s WHERE { ?s a {{TOP_CLASS}} } LIMIT 1000 } ?s ?p ?o } GROUP BY ?p ORDER BY DESC(?n)",
675            named_body: None,
676            requires: &[Cap::TopClass],
677            nonempty: NonEmpty::Witnessed,
678        }),
679    },
680    // --- Topology ---
681    Template {
682        id: "top-class-graph",
683        title: "Which classes link to which?",
684        dimension: "topology",
685        question: "What is the class-to-class schema graph?",
686        body: "SELECT ?sC ?p ?oC (COUNT(*) AS ?n) WHERE { ?s a ?sC ; ?p ?o OPTIONAL { ?o a ?oC } } GROUP BY ?sC ?p ?oC ORDER BY DESC(?n)",
687        named_body: None,
688        mixed_body: None,
689        tier: Tier::Index,
690        requires: &[Cap::TopClass],
691        nonempty: NonEmpty::Witnessed,
692        fallback: None,
693        provably_empty: None,
694        hook_is_exact: false,
695        scan: Scan::Everything,
696        // No bounded form, deliberately. A sample would join each of its
697        // subjects and objects to `?x a ?class`, and an unbound class is a
698        // pattern the planner answers by materializing every `rdf:type`
699        // statement (2.6 GiB on the OpenCitations sample, for a
700        // 1,000-statement sample) — and the answer is `card.class_links`,
701        // which the card already carries, exact, index-free.
702        bounded: None,
703    },
704    Template {
705        id: "top-out-hubs",
706        title: "Most connected sources",
707        dimension: "topology",
708        question: "Which subjects make the most statements?",
709        body: "SELECT ?s (COUNT(*) AS ?d) WHERE { ?s ?p ?o } GROUP BY ?s ORDER BY DESC(?d) LIMIT 25",
710        named_body: Some(
711            "SELECT ?s (COUNT(*) AS ?d) WHERE { GRAPH ?g { ?s ?p ?o } } GROUP BY ?s ORDER BY DESC(?d) LIMIT 25",
712        ),
713        mixed_body: Some(
714            "SELECT ?s (COUNT(*) AS ?d) WHERE { { ?s ?p ?o } UNION { GRAPH ?g { ?s ?p ?o } } } GROUP BY ?s ORDER BY DESC(?d) LIMIT 25",
715        ),
716        tier: Tier::Index,
717        requires: &[],
718        nonempty: NonEmpty::AnyGraph,
719        fallback: None,
720        provably_empty: None,
721        hook_is_exact: false,
722        scan: Scan::Everything,
723        bounded: None,
724    },
725    Template {
726        id: "top-in-hubs",
727        title: "Most referenced entities",
728        dimension: "topology",
729        question: "Which resources are referenced most often?",
730        body: "SELECT ?o (COUNT(*) AS ?d) WHERE { ?s ?p ?o FILTER(!isLiteral(?o)) } GROUP BY ?o ORDER BY DESC(?d) LIMIT 25",
731        named_body: Some(
732            "SELECT ?o (COUNT(*) AS ?d) WHERE { GRAPH ?g { ?s ?p ?o FILTER(!isLiteral(?o)) } } GROUP BY ?o ORDER BY DESC(?d) LIMIT 25",
733        ),
734        mixed_body: Some(
735            "SELECT ?o (COUNT(*) AS ?d) WHERE { { ?s ?p ?o } UNION { GRAPH ?g { ?s ?p ?o } } FILTER(!isLiteral(?o)) } GROUP BY ?o ORDER BY DESC(?d) LIMIT 25",
736        ),
737        tier: Tier::Index,
738        requires: &[],
739        // A graph whose objects are ALL literals (a flat table of values, no
740        // `rdf:type` anywhere) has no group to return. `card.in_hubs` is
741        // exactly this query's precompute, so in default-graph scope the
742        // emptiness is decidable — see `provably_empty`. In `GRAPH` scope the
743        // card profiles the wrong half and cannot say.
744        nonempty: NonEmpty::Undecidable(
745            "no non-literal object exists; decidable (and refused) in default-graph scope only, \
746             since the card profiles the default graph and this body may be GRAPH-scoped",
747        ),
748        fallback: None,
749        provably_empty: Some(|card| card.in_hubs.is_empty()),
750        // `in_hubs` is this query's own precompute, so the hook decides it in
751        // both directions once the body is default-graph scoped.
752        hook_is_exact: true,
753        scan: Scan::Everything,
754        bounded: None,
755    },
756    Template {
757        id: "top-dangling",
758        title: "Referenced but undescribed IRIs",
759        dimension: "topology",
760        question: "Which IRIs are referenced as objects but never described?",
761        // `OPTIONAL … FILTER(!BOUND(…))`, not `FILTER NOT EXISTS`: the same
762        // rows, but the engine probes the OPTIONAL once per streamed row and
763        // stops at the LIMIT, while NOT EXISTS materializes its whole inner
764        // pattern first. On the OpenCitations sample's 5.97 M-statement default
765        // graph that is 18 MB and 0.02 s against 3.4 GB and 4.8 s.
766        body: "SELECT ?o WHERE { ?s ?p ?o FILTER(isIRI(?o)) OPTIONAL { ?o ?p2 ?o2 } FILTER(!BOUND(?p2)) } LIMIT 100",
767        named_body: Some(
768            "SELECT ?o WHERE { GRAPH ?g { ?s ?p ?o FILTER(isIRI(?o)) } FILTER NOT EXISTS { GRAPH ?h { ?o ?p2 ?o2 } } } LIMIT 100",
769        ),
770        mixed_body: Some(
771            "SELECT ?o WHERE { { ?s ?p ?o } UNION { GRAPH ?g { ?s ?p ?o } } FILTER(isIRI(?o)) FILTER NOT EXISTS { ?o ?p2 ?o2 } FILTER NOT EXISTS { GRAPH ?h { ?o ?p3 ?o3 } } } LIMIT 100",
772        ),
773        tier: Tier::Index,
774        requires: &[],
775        // The honest one: a fully-described graph legitimately has none, and
776        // "is every referenced IRI described?" is precisely what the card does
777        // not record. Only the trivial half is decidable (no IRI object at all).
778        nonempty: NonEmpty::Undecidable(
779            "a fully-described graph has no dangling IRI — emptiness here is the answer, \
780             not a failure, and the card does not record which objects are also subjects",
781        ),
782        fallback: None,
783        provably_empty: Some(|card| card.in_hubs.is_empty()),
784        hook_is_exact: false,
785        scan: Scan::Everything,
786        bounded: Some(Bounded {
787            title: "Referenced but undescribed IRIs (1,000 statements)",
788            question: "Which IRIs referenced by a sample of 1,000 statements are never described in the default graph?",
789            body: "SELECT DISTINCT ?o WHERE { { SELECT ?o WHERE { ?s ?p ?o } LIMIT 1000 } FILTER(isIRI(?o)) OPTIONAL { ?o ?p2 ?o2 } FILTER(!BOUND(?p2)) } LIMIT 100",
790            named_body: None,
791            requires: &[],
792            nonempty: NonEmpty::Undecidable(
793                "a sample of a fully-described graph has no dangling IRI, and the card does not record which objects are also subjects",
794            ),
795        }),
796    },
797    Template {
798        id: "top-reach",
799        title: "What's reachable by following one relation?",
800        dimension: "connectivity",
801        question: "Following the most common entity-to-entity relation, what is reachable from a node?",
802        //
803        // Was `{{HUB_IRI}} {{TOP_PRED}}+ ?y`: the busiest subject and the most
804        // frequent predicate, chosen independently and then conjoined — the
805        // same defect as `lb-labels`, and live. On `hugging-face` the hub is a
806        // User and the top predicate `schema:keywords` (which only Models
807        // carry): 0 rows. Nothing in the card ties a specific subject to a
808        // specific predicate, so that pair can never be witnessed and the seed
809        // has to come from the relation itself.
810        //
811        // The sub-SELECT picks one real subject OF the relation, so the outer
812        // path always has at least the seed's own edge to return; the seed stays
813        // a single bound term, so the traversal costs what it did before (a
814        // bare `?x p+ ?y` would enumerate the whole closure, and these queries
815        // are measured at build time).
816        //
817        // `OBJECT_PRED`, not `TOP_PRED`: the most frequent predicate is usually
818        // a labelling one, whose objects are literals — a `+` over it can never
819        // walk past one hop. A `class_links` row with a non-literal `o_class`
820        // proves the relation goes entity-to-entity. A self-referential one
821        // (`s_class == o_class`) would chain deeper still, but its closure is
822        // unbounded — `hf:follows` is 886 k edges of social graph — so
823        // frequency, not recursion, picks the winner.
824        body: "SELECT ?x ?y WHERE { { SELECT ?x WHERE { ?x {{OBJECT_PRED}} ?o } LIMIT 1 } ?x {{OBJECT_PRED}}+ ?y } LIMIT 100",
825        named_body: None,
826        mixed_body: None,
827        tier: Tier::Index,
828        requires: &[Cap::ObjectPred],
829        nonempty: NonEmpty::Witnessed,
830        fallback: None,
831        provably_empty: None,
832        hook_is_exact: false,
833        scan: Scan::Bounded,
834        bounded: None,
835    },
836    // --- Connectivity / completeness ---
837    Template {
838        id: "cmp-coverage",
839        title: "Do all entities of the labelled class have a label?",
840        dimension: "connectivity",
841        question: "How complete is labelling on the class that carries labels?",
842        // `{{LABELED_CLASS}}` for the same reason as `lb-labels`, plus one of
843        // its own: with `{{TOP_CLASS}}` this reported `76990 / 0` on mtg — a
844        // row, so never caught by a zero-rows gate, and a measurement of the
845        // wrong class. Pairing it with `lb-labels` also keeps the two label
846        // queries talking about the same entities.
847        body: "SELECT (COUNT(?s) AS ?total) (COUNT(?l) AS ?have) WHERE { ?s a {{LABELED_CLASS}} OPTIONAL { ?s {{LABEL_PRED}} ?l } }",
848        named_body: None,
849        mixed_body: None,
850        tier: Tier::Index,
851        requires: &[Cap::LabeledClass, Cap::LabelPred],
852        nonempty: NonEmpty::Aggregate,
853        fallback: None,
854        provably_empty: None,
855        hook_is_exact: false,
856        scan: Scan::Class(Cap::LabeledClass),
857        bounded: Some(Bounded {
858            title: "Do the labelled class's entities have a label? (1,000 instances)",
859            question: "How complete is labelling on a sample of 1,000 instances of the class that carries labels?",
860            body: "SELECT (COUNT(?s) AS ?total) (COUNT(?l) AS ?have) WHERE { { SELECT ?s WHERE { ?s a {{LABELED_CLASS}} } LIMIT 1000 } OPTIONAL { ?s {{LABEL_PRED}} ?l } }",
861            named_body: None,
862            requires: &[Cap::LabeledClass, Cap::LabelPred],
863            nonempty: NonEmpty::Aggregate,
864        }),
865    },
866    // --- Links ---
867    Template {
868        id: "lk-sameas",
869        title: "Aligned to which external datasets?",
870        dimension: "links",
871        question: "Which entity-alignment predicates are used, and how often?",
872        // The VALUES list must cover the gate exactly. It listed three of the
873        // four predicates `signals.link_predicates` is detected from, so a
874        // dataset aligned only with `skos:closeMatch` passed the gate and then
875        // grouped nothing. Adding the fourth makes emptiness unreachable.
876        body: "SELECT ?p (COUNT(*) AS ?n) WHERE { ?s ?p ?o VALUES ?p { owl:sameAs skos:exactMatch skos:closeMatch rdfs:seeAlso } } GROUP BY ?p",
877        named_body: None,
878        mixed_body: None,
879        tier: Tier::Index,
880        requires: &[Cap::Link],
881        nonempty: NonEmpty::Witnessed,
882        fallback: None,
883        provably_empty: None,
884        hook_is_exact: false,
885        scan: Scan::Preds(LINK_PREDICATE_IRIS),
886        bounded: Some(Bounded {
887            title: "Aligned to which external datasets?",
888            question: "Which entity-alignment predicates are used?",
889            body: "SELECT ?p WHERE { VALUES ?p { owl:sameAs skos:exactMatch skos:closeMatch rdfs:seeAlso } FILTER EXISTS { ?s ?p ?o } }",
890            named_body: None,
891            requires: &[Cap::Link],
892            nonempty: NonEmpty::Witnessed,
893        }),
894    },
895    Template {
896        id: "lk-external",
897        title: "Which predicates link out?",
898        dimension: "links",
899        question: "Which predicates point to IRIs outside this dataset?",
900        // `EXTERNAL_IRI` is the witness that the FILTER keeps something: an IRI
901        // in `classes` or `in_hubs` that does not start with the base IRI.
902        // Without it a self-contained dataset — every object either a literal
903        // or an in-namespace IRI — emitted a query that grouped nothing.
904        body: "SELECT ?p (COUNT(*) AS ?n) WHERE { ?s ?p ?o FILTER(isIRI(?o) && !STRSTARTS(STR(?o), \"{{BASE_IRI}}\")) } GROUP BY ?p ORDER BY DESC(?n) LIMIT 50",
905        named_body: None,
906        mixed_body: None,
907        tier: Tier::Index,
908        requires: &[Cap::BaseIri, Cap::ExternalIri],
909        nonempty: NonEmpty::Witnessed,
910        fallback: None,
911        provably_empty: None,
912        hook_is_exact: false,
913        scan: Scan::Everything,
914        bounded: Some(Bounded {
915            title: "Which predicates link out? (1,000 statements)",
916            question: "In a sample of 1,000 statements, which predicates point to IRIs outside this dataset?",
917            body: "SELECT ?p (COUNT(*) AS ?n) WHERE { { SELECT ?p ?o WHERE { ?s ?p ?o } LIMIT 1000 } FILTER(isIRI(?o) && !STRSTARTS(STR(?o), \"{{BASE_IRI}}\")) } GROUP BY ?p ORDER BY DESC(?n) LIMIT 50",
918            named_body: None,
919            requires: &[Cap::BaseIri, Cap::ExternalIri],
920            nonempty: NonEmpty::Undecidable(
921                "the card records an IRI outside the base IRI somewhere in the graph, not in the 1,000-statement sample this body reads",
922            ),
923        }),
924    },
925    // --- Literals ---
926    Template {
927        id: "lt-datatypes",
928        title: "What value types appear?",
929        dimension: "literals",
930        question: "Which literal datatypes are used, and how often?",
931        body: "SELECT ?dt (COUNT(*) AS ?n) WHERE { ?s ?p ?o FILTER(isLiteral(?o)) } GROUP BY (DATATYPE(?o) AS ?dt) ORDER BY DESC(?n)",
932        named_body: None,
933        mixed_body: None,
934        tier: Tier::Index,
935        requires: &[Cap::HasLiterals],
936        nonempty: NonEmpty::Witnessed,
937        fallback: None,
938        provably_empty: None,
939        hook_is_exact: false,
940        scan: Scan::Everything,
941        bounded: None,
942    },
943    Template {
944        id: "lt-langs",
945        title: "What languages?",
946        dimension: "literals",
947        question: "Which language tags appear on literals, and how often?",
948        body: "SELECT ?l (COUNT(*) AS ?n) WHERE { ?s ?p ?o FILTER(isLiteral(?o)) } GROUP BY (LANG(?o) AS ?l) ORDER BY DESC(?n)",
949        named_body: None,
950        mixed_body: None,
951        tier: Tier::Index,
952        requires: &[Cap::HasLiterals],
953        nonempty: NonEmpty::Witnessed,
954        fallback: None,
955        provably_empty: None,
956        hook_is_exact: false,
957        scan: Scan::Everything,
958        bounded: None,
959    },
960    Template {
961        id: "lt-numrange",
962        title: "Range of a numeric property",
963        dimension: "literals",
964        question: "What is the min / average / max of the top numeric property?",
965        // BIND the cast first — aggregating over a function expression is rejected.
966        body: "SELECT (MIN(?v) AS ?lo) (AVG(?v) AS ?avg) (MAX(?v) AS ?hi) WHERE { ?s {{NUM_PRED}} ?o BIND(xsd:double(?o) AS ?v) }",
967        named_body: None,
968        mixed_body: None,
969        tier: Tier::Index,
970        requires: &[Cap::NumPred],
971        nonempty: NonEmpty::Aggregate,
972        fallback: None,
973        provably_empty: None,
974        hook_is_exact: false,
975        scan: Scan::Pred(Cap::NumPred),
976        bounded: None,
977    },
978    // --- Time ---
979    Template {
980        id: "ti-extent",
981        title: "What period does it span?",
982        dimension: "time",
983        question: "What is the earliest and latest value of the top time predicate?",
984        body: "SELECT (MIN(?d) AS ?from) (MAX(?d) AS ?to) WHERE { ?s {{TIME_PRED}} ?d }",
985        named_body: None,
986        mixed_body: None,
987        tier: Tier::Index,
988        requires: &[Cap::TimePred],
989        nonempty: NonEmpty::Aggregate,
990        fallback: None,
991        provably_empty: None,
992        hook_is_exact: false,
993        scan: Scan::Pred(Cap::TimePred),
994        bounded: None,
995    },
996    Template {
997        id: "ti-histogram",
998        title: "Distribution over time",
999        dimension: "time",
1000        question: "How are entities distributed by year?",
1001        body: "SELECT ?yr (COUNT(*) AS ?n) WHERE { ?s {{TIME_PRED}} ?d BIND(SUBSTR(STR(?d),1,4) AS ?yr) } GROUP BY ?yr ORDER BY ?yr",
1002        named_body: None,
1003        mixed_body: None,
1004        tier: Tier::Index,
1005        requires: &[Cap::TimePred],
1006        nonempty: NonEmpty::Witnessed,
1007        fallback: None,
1008        provably_empty: None,
1009        hook_is_exact: false,
1010        scan: Scan::Pred(Cap::TimePred),
1011        bounded: None,
1012    },
1013    // --- Space (WKT is always CRS84 lon/lat) ---
1014    Template {
1015        id: "sp-bbox",
1016        title: "Geographic extent",
1017        dimension: "space",
1018        question: "What is the bounding box of the wgs84 coordinates? (lon/lat)",
1019        // NOTE: the two predicates are conjoined on ONE subject, while the gate
1020        // (`geo_latlong`) is `lat seen` AND `long seen`, tallied independently —
1021        // the same shape as the `lb-labels` bug. It cannot produce zero rows
1022        // (an un-grouped aggregate always returns one), so the worst case is a
1023        // row of unbound values rather than a missing answer, and the card
1024        // cannot do better: `class_links` collapses every untyped subject into
1025        // one `(untyped)` bucket, so rows for `wgs:lat` and `wgs:long` under it
1026        // are not evidence that any single subject carries both.
1027        body: "SELECT (MIN(?lon) AS ?minLon) (MIN(?lat) AS ?minLat) (MAX(?lon) AS ?maxLon) (MAX(?lat) AS ?maxLat) WHERE { ?s wgs:long ?lon ; wgs:lat ?lat }",
1028        named_body: None,
1029        mixed_body: None,
1030        tier: Tier::Index,
1031        requires: &[Cap::GeoLatLong],
1032        nonempty: NonEmpty::Aggregate,
1033        fallback: None,
1034        provably_empty: None,
1035        hook_is_exact: false,
1036        scan: Scan::Preds(&[WGS_LAT_IRI, WGS_LONG_IRI]),
1037        bounded: None,
1038    },
1039    Template {
1040        id: "sp-within",
1041        title: "Features inside a box",
1042        dimension: "space",
1043        question: "Which features fall inside the dataset's bounding box? (WKT is lon/lat)",
1044        //
1045        // The access path is `{{WKT_PATH}}`, read off the predicates the card
1046        // recorded, because the fixed `geo:hasGeometry/geo:asWKT` was a third
1047        // instance of the same bug — and a live one. The gate, `signals.geo_wkt`,
1048        // is a three-way OR (`geo:asWKT` seen, OR `geo:hasGeometry` seen, OR a
1049        // `wktLiteral` datatype seen) while the body demanded BOTH steps of the
1050        // chain: `geoadmin` hangs `geo:asWKT` straight off each District and has
1051        // no `geo:hasGeometry` at all, so its published card ships a query that
1052        // cannot match — 0 rows on 52,959 geometries. `geo:hasGeometry?/geo:asWKT`
1053        // covers both layouts in one path (the optional step binds `?s` to the
1054        // feature when there is a geometry node, to the feature itself when the
1055        // WKT hangs off it directly).
1056        body: "SELECT ?s WHERE { ?s {{WKT_PATH}} ?w FILTER(geof:sfWithin(?w, \"{{BBOX_POLYGON}}\"^^geo:wktLiteral)) } LIMIT 100",
1057        named_body: None,
1058        mixed_body: None,
1059        tier: Tier::Index,
1060        requires: &[Cap::WktPath, Cap::Bbox],
1061        // The path is witnessed; the FILTER is not, and cannot be. The box is
1062        // derived from `wgs84:lat`/`wgs84:long` literals — a different signal
1063        // from the geometries this body reads, on possibly different subjects —
1064        // and a GeoSPARQL `wktLiteral` may carry a projected CRS whose
1065        // coordinates fall outside any lon/lat box, so not even the whole-world
1066        // fallback box is a proof. Emptiness here means "nothing in that box",
1067        // which is a real answer about the data, and the box is the parameter
1068        // the reader is meant to edit.
1069        nonempty: NonEmpty::Undecidable(
1070            "the box is derived from wgs84 lat/long literals, not from the WKT geometries this \
1071             query reads, and the card records no coordinate extent for them",
1072        ),
1073        fallback: None,
1074        provably_empty: None,
1075        hook_is_exact: false,
1076        scan: Scan::Preds(&[GEO_ASWKT]),
1077        bounded: Some(Bounded {
1078            title: "Features inside a box (1,000 geometries)",
1079            question: "Which of a sample of 1,000 geometries fall inside the dataset's bounding box? (WKT is lon/lat)",
1080            body: "SELECT ?s WHERE { { SELECT ?s ?w WHERE { ?s {{WKT_PATH}} ?w } LIMIT 1000 } FILTER(geof:sfWithin(?w, \"{{BBOX_POLYGON}}\"^^geo:wktLiteral)) } LIMIT 100",
1081            named_body: None,
1082            requires: &[Cap::WktPath, Cap::Bbox],
1083            nonempty: NonEmpty::Undecidable(
1084                "the box is derived from wgs84 lat/long literals, not from the WKT geometries this query reads, and the card records no coordinate extent for them",
1085            ),
1086        }),
1087    },
1088    // --- Named graphs ---
1089    Template {
1090        id: "ng-list",
1091        title: "What graphs partition this?",
1092        dimension: "graphs",
1093        question: "Which named graphs does the dataset contain?",
1094        body: "SELECT DISTINCT ?g WHERE { GRAPH ?g { ?s ?p ?o } }",
1095        named_body: None,
1096        mixed_body: None,
1097        tier: Tier::Index,
1098        requires: &[Cap::NamedGraphs],
1099        nonempty: NonEmpty::Witnessed,
1100        fallback: None,
1101        provably_empty: None,
1102        hook_is_exact: false,
1103        scan: Scan::NamedGraphs,
1104        bounded: Some(Bounded {
1105            title: "What graphs partition this? (first 100)",
1106            question: "Which named graphs does the dataset contain? (the first 100)",
1107            body: "SELECT DISTINCT ?g WHERE { GRAPH ?g { ?s ?p ?o } } LIMIT 100",
1108            named_body: Some(
1109                "SELECT DISTINCT ?g WHERE { GRAPH ?g { ?s ?p ?o } } LIMIT 100",
1110            ),
1111            requires: &[Cap::NamedGraphs],
1112            nonempty: NonEmpty::Witnessed,
1113        }),
1114    },
1115    Template {
1116        id: "ng-sizes",
1117        title: "Which graphs are biggest?",
1118        dimension: "graphs",
1119        question: "Which named graphs hold the most statements?",
1120        // LIMIT keeps this a starter on a file with tens of thousands of graphs.
1121        body: "SELECT ?g (COUNT(*) AS ?n) WHERE { GRAPH ?g { ?s ?p ?o } } GROUP BY ?g ORDER BY DESC(?n) LIMIT 25",
1122        named_body: None,
1123        mixed_body: None,
1124        tier: Tier::Index,
1125        requires: &[Cap::NamedGraphs],
1126        nonempty: NonEmpty::Witnessed,
1127        fallback: None,
1128        provably_empty: None,
1129        hook_is_exact: false,
1130        scan: Scan::NamedGraphs,
1131        bounded: None,
1132    },
1133    Template {
1134        id: "ng-sample",
1135        title: "A peek inside the graphs",
1136        dimension: "graphs",
1137        question: "What do statements in the named graphs look like?",
1138        // `?g` in the projection on purpose: each sample row says where it lives.
1139        body: "SELECT ?g ?s ?p ?o WHERE { GRAPH ?g { ?s ?p ?o } } LIMIT 10",
1140        named_body: None,
1141        mixed_body: None,
1142        tier: Tier::Index,
1143        requires: &[Cap::NamedGraphs],
1144        nonempty: NonEmpty::Witnessed,
1145        fallback: None,
1146        provably_empty: None,
1147        hook_is_exact: false,
1148        scan: Scan::Bounded,
1149        bounded: None,
1150    },
1151    // --- Statements about statements (RDF-star) ---
1152    //
1153    // Emitted only for a dataset whose own statements were seen to quote
1154    // other statements, and written in the RDF-star surface `<< s p o >>`,
1155    // which is what the SPARQL parser accepts (`docs/sparql.md#rdf-star`).
1156    // The RDF 1.2 spelling `<<( s p o )>>` is an EXPORT surface, not a query
1157    // one, and a body written in it would not parse — the worst thing a card
1158    // can carry.
1159    //
1160    // None of these has a `named_body`: the profile they are instantiated from
1161    // is derived from the default graph, so on a named-graph-only file the
1162    // capabilities are simply absent and nothing is emitted.
1163    Template {
1164        id: "qt-sample",
1165        title: "A statement someone made a statement about",
1166        dimension: "statements",
1167        question: "Which statements does this dataset say something about, and what does it say?",
1168        // The family's entry point, and the one shape worth copying: the inner
1169        // variables bind the quoted statement's own subject, predicate and
1170        // object, so one row shows both the assertion and what qualifies it.
1171        // The annotation predicate is pinned rather than left a variable so
1172        // the scan routes on a predicate instead of walking the whole index.
1173        body: "SELECT ?s ?p ?o ?value WHERE { << ?s ?p ?o >> {{ANNOTATION_PRED}} ?value } LIMIT 25",
1174        named_body: None,
1175        mixed_body: None,
1176        tier: Tier::Index,
1177        // One substitution, from a predicate the derivation counted on a
1178        // statement whose subject really is a quoted triple.
1179        requires: &[Cap::AnnotationPred],
1180        nonempty: NonEmpty::Witnessed,
1181        fallback: None,
1182        provably_empty: None,
1183        hook_is_exact: false,
1184        scan: Scan::Bounded,
1185        bounded: None,
1186    },
1187    Template {
1188        id: "qt-about",
1189        title: "What gets said about statements?",
1190        dimension: "statements",
1191        question: "Which predicates qualify a statement rather than an entity, and how often?",
1192        // The annotation schema: the `ov-pred-hist` of the star layer. The
1193        // annotation predicate is a variable here on purpose — that IS the
1194        // question — so the gate is the pure "such statements exist" bool.
1195        body: "SELECT ?about (COUNT(*) AS ?n) WHERE { << ?s ?p ?o >> ?about ?value } GROUP BY ?about ORDER BY DESC(?n) LIMIT 50",
1196        named_body: None,
1197        mixed_body: None,
1198        tier: Tier::Index,
1199        requires: &[Cap::AnnotatedStatements],
1200        // A GROUP BY yields no group over an empty sequence — but the card
1201        // counted at least one statement whose subject is a quoted triple, and
1202        // every statement has a predicate to group by.
1203        nonempty: NonEmpty::Witnessed,
1204        fallback: None,
1205        provably_empty: None,
1206        hook_is_exact: false,
1207        scan: Scan::Annotations,
1208        bounded: None,
1209    },
1210    Template {
1211        id: "qt-qualifiers",
1212        title: "Who (or what) qualifies the assertions?",
1213        dimension: "statements",
1214        question: "How many assertions does each distinct qualifier value account for?",
1215        // "Count assertions by whatever qualifies them" — the recorder, the
1216        // source, the confidence band. Grouping on the VALUE, not on the
1217        // predicate, is what makes this a different question from `qt-about`.
1218        body: "SELECT ?value (COUNT(*) AS ?n) WHERE { << ?s ?p ?o >> {{ANNOTATION_PRED}} ?value } GROUP BY ?value ORDER BY DESC(?n) LIMIT 50",
1219        named_body: None,
1220        mixed_body: None,
1221        tier: Tier::Index,
1222        requires: &[Cap::AnnotationPred],
1223        nonempty: NonEmpty::Witnessed,
1224        fallback: None,
1225        provably_empty: None,
1226        hook_is_exact: false,
1227        scan: Scan::Pred(Cap::AnnotationPred),
1228        bounded: None,
1229    },
1230    Template {
1231        id: "qt-coverage",
1232        title: "How much of the data is qualified?",
1233        dimension: "statements",
1234        question: "Of the statements that get annotated most, how many actually carry an annotation?",
1235        // The `cmp-coverage` of the star layer, and the question a provenance
1236        // graph is really about: 40,000 assertions, 300 of them sourced.
1237        //
1238        // `{{ANNOTATED_PRED}}` is written twice — once in a regular pattern to
1239        // enumerate the assertions, once inside the quoted pattern to reach
1240        // their annotations. `?s`/`?o` are bound by the regular pattern, so
1241        // the quoted occurrence lowers to a join on them (`sameTerm`), not a
1242        // second enumeration. The two predicates conjoined here are the ONE
1243        // pair the derivation saw on a single statement
1244        // (`signals.annotated_statement`), never two independent maxima.
1245        body: "SELECT (COUNT(*) AS ?asserted) (COUNT(?value) AS ?qualified) WHERE { ?s {{ANNOTATED_PRED}} ?o OPTIONAL { << ?s {{ANNOTATED_PRED}} ?o >> {{ANNOTATION_PRED}} ?value } }",
1246        named_body: None,
1247        mixed_body: None,
1248        tier: Tier::Index,
1249        requires: &[Cap::AnnotatedPred, Cap::AnnotationPred],
1250        nonempty: NonEmpty::Aggregate,
1251        fallback: None,
1252        provably_empty: None,
1253        hook_is_exact: false,
1254        scan: Scan::Pred(Cap::AnnotatedPred),
1255        bounded: Some(Bounded {
1256            title: "How much of the data is qualified? (1,000 assertions)",
1257            question: "Of a sample of 1,000 statements of the most-annotated kind, how many carry an annotation?",
1258            body: "SELECT (COUNT(*) AS ?asserted) (COUNT(?value) AS ?qualified) WHERE { { SELECT ?s ?o WHERE { ?s {{ANNOTATED_PRED}} ?o } LIMIT 1000 } OPTIONAL { << ?s {{ANNOTATED_PRED}} ?o >> {{ANNOTATION_PRED}} ?value } }",
1259            named_body: None,
1260            requires: &[Cap::AnnotatedPred, Cap::AnnotationPred],
1261            nonempty: NonEmpty::Aggregate,
1262        }),
1263    },
1264    Template {
1265        id: "qt-quoted-object",
1266        title: "Statements held as values",
1267        dimension: "statements",
1268        question: "Which statements are referenced as the value of another statement?",
1269        // The OTHER position a triple term stands in — `:claim :states
1270        // << s p o >>` — and a separate capability, because a dataset can have
1271        // one shape and not the other: a query written for the subject
1272        // position returns nothing at all on an object-position file (and the
1273        // header's presence bit cannot tell them apart).
1274        body: "SELECT ?s ?qs ?qp ?qo WHERE { ?s {{QUOTING_PRED}} << ?qs ?qp ?qo >> } LIMIT 25",
1275        named_body: None,
1276        mixed_body: None,
1277        tier: Tier::Index,
1278        requires: &[Cap::QuotingPred],
1279        nonempty: NonEmpty::Witnessed,
1280        fallback: None,
1281        provably_empty: None,
1282        hook_is_exact: false,
1283        scan: Scan::Bounded,
1284        bounded: None,
1285    },
1286];
1287
1288/// What the template behind an emitted query claims about its emptiness — the
1289/// `NonEmpty` declaration, read back by id.
1290#[derive(Debug, Clone, Copy, PartialEq, Eq)]
1291pub enum Claim {
1292    /// The template asserts the emitted query cannot come back empty
1293    /// (`NonEmpty::AnyGraph` / `NonEmpty::Aggregate` / `NonEmpty::Witnessed`).
1294    CannotBeEmpty,
1295    /// The template admits the card cannot decide it
1296    /// (`NonEmpty::Undecidable`).
1297    Undecidable,
1298    /// No template of this revision owns the id — a curated query, or one an
1299    /// older revision wrote. It claims nothing, so nothing is contradicted.
1300    Unknown,
1301}
1302
1303/// The [`Claim`] behind a shipped query id.
1304///
1305/// The build measures every starter query against the finished file
1306/// (`commands::buildinfo`), which makes emptiness a **measured** fact rather
1307/// than a derived one — and where measurement and this claim disagree, the
1308/// claim is wrong. A query the table swears cannot be empty and that then
1309/// answers nothing is a defect in the table, not a fact about the dataset, and
1310/// `commands::build` says so out loud and records it. `Undecidable` is the
1311/// opposite: the template said in advance it could not know, so a measured zero
1312/// is expected news, not a contradiction.
1313///
1314/// This is the claim of the template's **exact** body. A template can also
1315/// ship a `Bounded` form with a claim of its own (a sample of a graph that has
1316/// dangling IRIs need not contain one); [`claim_of_query`] reads the body and
1317/// answers for whichever form it is.
1318pub fn claim_of(id: &str) -> Claim {
1319    match TEMPLATES.iter().find(|t| t.id == id) {
1320        Some(t) => claim_from(t.nonempty),
1321        None => Claim::Unknown,
1322    }
1323}
1324
1325/// The [`Claim`] behind a shipped query, read from its **body** as well as its
1326/// id — the form to use whenever the SPARQL is at hand, because the exact and
1327/// the bounded form of one template claim different things.
1328pub fn claim_of_query(id: &str, sparql: &str) -> Claim {
1329    let Some(t) = TEMPLATES.iter().find(|t| t.id == id) else {
1330        return Claim::Unknown;
1331    };
1332    match variant_of(t, strip_prefixes(sparql).trim()) {
1333        Some(v) => claim_from(v.nonempty),
1334        None => claim_from(t.nonempty),
1335    }
1336}
1337
1338fn claim_from(nonempty: NonEmpty) -> Claim {
1339    match nonempty {
1340        NonEmpty::Undecidable(_) => Claim::Undecidable,
1341        _ => Claim::CannotBeEmpty,
1342    }
1343}
1344
1345/// Which of a template's forms a published body is, and the bindings it gave
1346/// the placeholders.
1347struct Variant {
1348    pattern: &'static str,
1349    binds: Binds,
1350    nonempty: NonEmpty,
1351    /// The [`Bounded`] form — a sample, so the card's whole-graph precomputes
1352    /// stop deciding it in both directions (see `Template::hook_is_exact`).
1353    bounded: bool,
1354}
1355
1356/// Match a published body against every form of `t`: the exact body in each
1357/// scope, the fallback, the bounded form, then the bodies earlier revisions
1358/// wrote. The *values* decide a verdict, so two revisions sharing a pattern is
1359/// not a problem — `revision` already says which one a re-card would write.
1360fn variant_of(t: &Template, body: &str) -> Option<Variant> {
1361    let mut forms: Vec<(&'static str, NonEmpty, bool)> = Vec::new();
1362    for p in [Some(t.body), t.named_body, t.mixed_body]
1363        .into_iter()
1364        .flatten()
1365    {
1366        forms.push((p, t.nonempty, false));
1367    }
1368    if let Some(f) = &t.fallback {
1369        forms.push((f.body, f.nonempty, false));
1370    }
1371    if let Some(b) = &t.bounded {
1372        for p in [Some(b.body), b.named_body].into_iter().flatten() {
1373            forms.push((p, b.nonempty, true));
1374        }
1375    }
1376    for l in LEGACY_BODIES.iter().filter(|l| l.id == t.id) {
1377        forms.push((l.body, t.nonempty, false));
1378    }
1379    // The bodies of one template are not prefixes of each other, so an exact
1380    // match is unambiguous.
1381    forms.into_iter().find_map(|(pattern, nonempty, bounded)| {
1382        match_pattern(pattern, body).map(|binds| Variant {
1383            pattern,
1384            binds,
1385            nonempty,
1386            bounded,
1387        })
1388    })
1389}
1390
1391/// A query's body without the `PREFIX` lines every generated query carries.
1392fn strip_prefixes(sparql: &str) -> String {
1393    sparql
1394        .lines()
1395        .filter(|l| !l.starts_with("PREFIX"))
1396        .collect::<Vec<_>>()
1397        .join("\n")
1398}
1399
1400/// Where the dataset's statements live, decided from the card's own counts —
1401/// so the emitted queries address the graph(s) that actually hold data instead
1402/// of scanning a default graph the card itself records as empty.
1403#[derive(Debug, Clone, Copy, PartialEq, Eq)]
1404enum GraphScope {
1405    /// No named graphs — bodies are used verbatim, so classic cards are
1406    /// byte-identical to pre-scope-awareness builds.
1407    DefaultOnly,
1408    /// Every statement lives in a named graph and the default graph is empty
1409    /// (`triple_count == 0 && named_graph_count > 0`) — a bare `{ ?s ?p ?o }`
1410    /// can only ever return zero rows here.
1411    NamedOnly,
1412    /// Both the default graph and named graphs hold statements.
1413    Mixed,
1414}
1415
1416impl GraphScope {
1417    fn of(card: &DatasetCard) -> Self {
1418        if card.named_graph_count == 0 {
1419            GraphScope::DefaultOnly
1420        } else if card.triple_count == 0 {
1421            GraphScope::NamedOnly
1422        } else {
1423            GraphScope::Mixed
1424        }
1425    }
1426}
1427
1428/// The dataset's resolved vocabulary, drawn from the card profile.
1429///
1430/// The *jointly derived* fields — [`Caps::labeled_class`], [`Caps::object_pred`],
1431/// [`Caps::wkt_path`], [`Caps::external_iri`] — are the point of this struct: each
1432/// is resolved from a `class_links` row (or a predicate list) that **witnesses**
1433/// the pattern its template will write, rather than from an independent ranking
1434/// that merely says the term exists somewhere in the graph.
1435struct Caps {
1436    top_class: Option<String>,
1437    top_pred: Option<String>,
1438    labeled_class: Option<String>,
1439    label_pred: Option<String>,
1440    object_pred: Option<String>,
1441    time_pred: Option<String>,
1442    num_pred: Option<String>,
1443    base_iri: Option<String>,
1444    external_iri: bool,
1445    wkt_path: Option<String>,
1446    bbox_polygon: Option<String>,
1447    geo_latlong: bool,
1448    named_graphs: bool,
1449    link: bool,
1450    has_literals: bool,
1451    annotated_statements: bool,
1452    annotation_pred: Option<String>,
1453    annotated_pred: Option<String>,
1454    quoting_pred: Option<String>,
1455}
1456
1457impl Caps {
1458    fn from_card(card: &DatasetCard) -> Self {
1459        let s = &card.signals;
1460        // The most frequent predicate that is not rdf:type (the interesting
1461        // relation), falling back to whatever is most frequent.
1462        let top_pred = card
1463            .predicates
1464            .iter()
1465            .map(|(p, _)| p)
1466            .find(|p| p.as_str() != RDF_TYPE)
1467            .or_else(|| card.predicates.first().map(|(p, _)| p))
1468            .cloned();
1469
1470        // LABELED_CLASS — the most populous class the card can PROVE carries the
1471        // label predicate. `card.classes` is ordered by instance count, so the
1472        // first hit is the biggest labelled class, and it equals `top_class`
1473        // whenever the top class is labelled (the common case, which therefore
1474        // produces exactly the query it produced before). A `class_links` row is
1475        // a positive witness — some subject really has both statements — so a
1476        // hit is proof, and a miss (a truncated top-100 list, or labels only on
1477        // untyped subjects) is merely absence of proof, which is why the
1478        // template keeps a class-free fallback rather than vanishing.
1479        let label_pred = s.label_predicate.clone();
1480        let labeled_class = label_pred.as_ref().and_then(|lp| {
1481            card.classes
1482                .iter()
1483                .map(|(c, _)| c)
1484                .find(|c| class_carries(card, c, lp))
1485                .cloned()
1486        });
1487
1488        // OBJECT_PRED — the most frequent relation the card can PROVE points at
1489        // something other than a literal, i.e. one a path query can walk.
1490        // `class_links` is ordered by count and excludes `rdf:type` rows.
1491        let object_pred = card
1492            .class_links
1493            .iter()
1494            .find(|l| l.o_class != O_LITERAL && l.predicate != RDF_TYPE)
1495            .map(|l| l.predicate.clone());
1496
1497        // WKT_PATH — how this dataset actually hangs a geometry off a subject.
1498        // Read from the predicate list (capped at the top 100, so a miss can
1499        // only lose the query, never emit a broken one).
1500        let has_pred = |iri: &str| card.predicates.iter().any(|(p, _)| p == iri);
1501        let wkt_path = has_pred(GEO_ASWKT).then(|| {
1502            if has_pred(GEO_HASGEOMETRY) {
1503                // The optional step covers BOTH layouts at once: `?s` binds to
1504                // the feature when a geometry node sits in between, and to the
1505                // feature itself when the WKT hangs off it directly.
1506                "geo:hasGeometry?/geo:asWKT".to_string()
1507            } else {
1508                "geo:asWKT".to_string()
1509            }
1510        });
1511        let bbox_polygon = wkt_path.is_some().then(|| bbox_polygon(s.spatial_bbox));
1512
1513        // EXTERNAL_IRI — an IRI the card records that does NOT start with the
1514        // dataset's base IRI, so the "links pointing out" filter keeps at least
1515        // one statement. Classes count: `?s a foaf:Person` is an IRI object like
1516        // any other. Blank nodes do not — the query filters on `isIRI`.
1517        let external_iri = s.base_iri.as_deref().is_some_and(|base| {
1518            card.classes
1519                .iter()
1520                .chain(card.in_hubs.iter())
1521                .any(|(term, _)| {
1522                    let iri = term.trim_start_matches('<').trim_end_matches('>');
1523                    !term.starts_with("_:") && !iri.starts_with(base)
1524                })
1525        });
1526
1527        // RDF-star. The presence signal the card *displays* is read off the
1528        // header at display time — true for every file ever built, which is
1529        // what makes it honest about the already-published corpus, and useless
1530        // here: it names no vocabulary and cannot be written into a query. So
1531        // the `qt-*` family is gated on the profile derived from the
1532        // statements instead, which a card only has if it was derived after
1533        // this existed. Presence keeps answering on old files; queries arrive
1534        // with the next build.
1535        //
1536        // `ANNOTATED_PRED` and `ANNOTATION_PRED` come from the ONE pair the
1537        // derivation saw on a single statement, never from the two rankings
1538        // independently — see `Cap::joint_with`.
1539        let witness = annotated_statement(card);
1540        let annotated_pred = witness.map(|(quoted, _)| quoted.to_string());
1541        // The witnessed annotating predicate when there is one — so the pair
1542        // `qt-coverage` writes really met, and so the whole family talks about
1543        // one predicate (the reason `cmp-coverage` shares `lb-labels`' class).
1544        // Otherwise the plain ranking, which is all the single-substitution
1545        // bodies need; `ANNOTATED_PRED` is then absent and nothing conjoins.
1546        let annotation_pred = witness
1547            .map(|(_, about)| about.to_string())
1548            .or_else(|| s.annotation_predicates.first().cloned());
1549
1550        Caps {
1551            top_class: card.classes.first().map(|(c, _)| c.clone()),
1552            top_pred,
1553            labeled_class,
1554            label_pred,
1555            object_pred,
1556            time_pred: s.time_predicates.first().cloned(),
1557            num_pred: s.numeric_predicates.first().cloned(),
1558            base_iri: s.base_iri.clone(),
1559            external_iri,
1560            wkt_path,
1561            bbox_polygon,
1562            geo_latlong: s.geo_latlong,
1563            named_graphs: card.named_graph_count > 0,
1564            link: !s.link_predicates.is_empty(),
1565            has_literals: !card.datatypes.is_empty(),
1566            annotated_statements: !s.annotation_predicates.is_empty(),
1567            annotation_pred,
1568            annotated_pred,
1569            quoting_pred: s.quoting_predicates.first().cloned(),
1570        }
1571    }
1572
1573    /// The text substituted for a capability's placeholder, if it has one.
1574    fn value(&self, cap: Cap) -> Option<&str> {
1575        match cap {
1576            Cap::TopClass => self.top_class.as_deref(),
1577            Cap::TopPred => self.top_pred.as_deref(),
1578            Cap::LabeledClass => self.labeled_class.as_deref(),
1579            Cap::LabelPred => self.label_pred.as_deref(),
1580            Cap::ObjectPred => self.object_pred.as_deref(),
1581            Cap::TimePred => self.time_pred.as_deref(),
1582            Cap::NumPred => self.num_pred.as_deref(),
1583            Cap::BaseIri => self.base_iri.as_deref(),
1584            Cap::WktPath => self.wkt_path.as_deref(),
1585            Cap::Bbox => self.bbox_polygon.as_deref(),
1586            Cap::AnnotationPred => self.annotation_pred.as_deref(),
1587            Cap::AnnotatedPred => self.annotated_pred.as_deref(),
1588            Cap::QuotingPred => self.quoting_pred.as_deref(),
1589            Cap::ExternalIri
1590            | Cap::GeoLatLong
1591            | Cap::NamedGraphs
1592            | Cap::Link
1593            | Cap::HasLiterals
1594            | Cap::AnnotatedStatements => None,
1595        }
1596    }
1597
1598    /// Is a capability available (string caps must resolve; bool caps must hold)?
1599    fn available(&self, cap: Cap) -> bool {
1600        match cap {
1601            Cap::ExternalIri => self.external_iri,
1602            Cap::GeoLatLong => self.geo_latlong,
1603            Cap::NamedGraphs => self.named_graphs,
1604            Cap::Link => self.link,
1605            Cap::HasLiterals => self.has_literals,
1606            Cap::AnnotatedStatements => self.annotated_statements,
1607            other => self.value(other).is_some(),
1608        }
1609    }
1610
1611    /// Substitute every placeholder we have a value for.
1612    fn substitute(&self, body: &str) -> String {
1613        let mut out = body.to_string();
1614        for &cap in ALL_CAPS {
1615            if let (Some(ph), Some(v)) = (cap.placeholder(), self.value(cap)) {
1616                out = out.replace(ph, v);
1617            }
1618        }
1619        out
1620    }
1621}
1622
1623/// Audit one body against the co-occurrence rule (see the module docs).
1624///
1625/// Two things are checked, and both are properties of the **table**, not of any
1626/// dataset — so this runs over every template at test time and, in debug builds,
1627/// over every body the generator actually emits:
1628///
1629/// 1. Every `{{PLACEHOLDER}}` names a capability the template *requires*. A body
1630///    can therefore never be shipped with an unsubstituted hole, whatever the
1631///    card looks like.
1632/// 2. Unless the template declares itself `NonEmpty::Undecidable`, any two
1633///    distinct capabilities appearing in the same body are [jointly
1634///    derived](Cap::joint_with). This is the rule `lb-labels` broke.
1635fn check_body(id: &str, body: &str, requires: &[Cap], nonempty: NonEmpty) -> Result<(), String> {
1636    let used: Vec<Cap> = ALL_CAPS
1637        .iter()
1638        .copied()
1639        .filter(|c| c.placeholder().is_some_and(|ph| body.contains(ph)))
1640        .collect();
1641    // (1) Placeholders are accounted for, in both directions.
1642    for cap in &used {
1643        if !requires.contains(cap) {
1644            return Err(format!(
1645                "{id}: body uses {} but does not require {}",
1646                cap.placeholder().unwrap_or_default(),
1647                cap.key()
1648            ));
1649        }
1650    }
1651    for chunk in body.split("{{").skip(1) {
1652        let name = chunk.split("}}").next().unwrap_or(chunk);
1653        let ph = format!("{{{{{name}}}}}");
1654        if !ALL_CAPS.iter().any(|c| c.placeholder() == Some(&ph[..])) {
1655            return Err(format!("{id}: body has an unknown placeholder {ph}"));
1656        }
1657    }
1658    // (2) Whatever meets in the body must have been chosen together.
1659    if matches!(nonempty, NonEmpty::Undecidable(_)) {
1660        return Ok(());
1661    }
1662    for (i, a) in used.iter().enumerate() {
1663        for b in &used[i + 1..] {
1664            if !a.joint_with().contains(b) {
1665                return Err(format!(
1666                    "{id}: conjoins {} with {}, which are chosen independently — \
1667                     make one jointly derived (Cap::joint_with) or declare NonEmpty::Undecidable",
1668                    a.key(),
1669                    b.key()
1670                ));
1671            }
1672        }
1673    }
1674    Ok(())
1675}
1676
1677/// A CRS84 lon/lat WKT polygon for the bounding box, or the whole world when the
1678/// dataset has geometry but no wgs84 extent was derived.
1679fn bbox_polygon(bbox: Option<[f64; 4]>) -> String {
1680    let [min_lon, min_lat, max_lon, max_lat] = bbox.unwrap_or([-180.0, -90.0, 180.0, 90.0]);
1681    format!(
1682        "POLYGON(({min_lon} {min_lat}, {max_lon} {min_lat}, {max_lon} {max_lat}, {min_lon} {max_lat}, {min_lon} {min_lat}))"
1683    )
1684}
1685
1686/// The count the card records for a predicate, or — when the term fell off the
1687/// capped list — the most it could be: a list sorted by count that stops at the
1688/// cap bounds everything below it by its last entry. A complete list that does
1689/// not name the term proves it absent.
1690fn bounded_count(card: &DatasetCard, list: &[(String, u64)], term: &str) -> u64 {
1691    if let Some((_, n)) = list.iter().find(|(t, _)| t == term) {
1692        return *n;
1693    }
1694    if complete(card, list) {
1695        0
1696    } else {
1697        list.last().map_or(u64::MAX, |(_, n)| *n)
1698    }
1699}
1700
1701/// Which form of a template's body was picked — the [`Scan::Everything`]
1702/// estimate depends on it (a `UNION`/`GRAPH` body reads the named graphs too).
1703#[derive(Clone, Copy, PartialEq, Eq)]
1704enum BodyScope {
1705    Default,
1706    AllQuads,
1707}
1708
1709/// How many statements a body of this [`Scan`] reads, from the card's counts.
1710fn scan_estimate(scan: Scan, card: &DatasetCard, caps: &Caps, body: BodyScope) -> u64 {
1711    let pred = |iri: &str| bounded_count(card, &card.predicates, iri);
1712    let class = |iri: &str| bounded_count(card, &card.classes, iri);
1713    let named = card.quad_count.saturating_sub(card.triple_count);
1714    match scan {
1715        Scan::Bounded => 0,
1716        Scan::Everything => match body {
1717            BodyScope::Default => card.triple_count,
1718            BodyScope::AllQuads => card.quad_count,
1719        },
1720        Scan::Types => pred(RDF_TYPE),
1721        Scan::Pred(cap) => caps.value(cap).map_or(u64::MAX, pred),
1722        Scan::Preds(list) => list.iter().map(|p| pred(p)).fold(0u64, u64::saturating_add),
1723        Scan::Class(cap) => caps.value(cap).map_or(u64::MAX, class),
1724        Scan::ClassStar(cap) => {
1725            let Some(c) = caps.value(cap) else {
1726                return u64::MAX;
1727            };
1728            // Every statement a subject of the class makes, as the quotient
1729            // counts them. A class with no row of its own in a truncated
1730            // quotient is charged the whole graph — the estimate errs high.
1731            let rows: u64 = card
1732                .class_links
1733                .iter()
1734                .filter(|l| l.s_class == c)
1735                .map(|l| l.count)
1736                .sum();
1737            if rows == 0 && !complete(card, &card.class_links) {
1738                return card.triple_count;
1739            }
1740            class(c).saturating_add(rows)
1741        }
1742        Scan::NamedGraphs => named,
1743        Scan::Annotations => card
1744            .signals
1745            .annotation_predicates
1746            .iter()
1747            .map(|p| pred(p))
1748            .fold(0u64, u64::saturating_add),
1749    }
1750}
1751
1752/// Generate the tiered starter-query library for a card: emit each template whose
1753/// required capabilities are all present, with the body picked for where the data
1754/// lives (`GraphScope`), placeholders substituted, and the shared PREFIX block
1755/// prepended. A body whose `Scan` is over [`STARTER_SCAN_BUDGET`] statements is
1756/// replaced by its bounded form, or dropped when it has none.
1757pub fn generate(card: &DatasetCard) -> Vec<ExampleQuery> {
1758    generate_with_budget(card, STARTER_SCAN_BUDGET)
1759}
1760
1761/// [`generate`] under an explicit scan budget — the tests' way to exercise the
1762/// bounded forms on a fixture small enough to build in a unit test.
1763fn generate_with_budget(card: &DatasetCard, budget: u64) -> Vec<ExampleQuery> {
1764    let caps = Caps::from_card(card);
1765    let scope = GraphScope::of(card);
1766    let mut out = Vec::new();
1767    for t in TEMPLATES {
1768        // 1. Capability gate — the full template, or its reduced fallback. A
1769        //    fallback carries no scope variants, so it is default-graph shaped.
1770        let all = |rs: &[Cap]| rs.iter().all(|&c| caps.available(c));
1771        let (base, named, mixed, requires, nonempty) = if all(t.requires) {
1772            (t.body, t.named_body, t.mixed_body, t.requires, t.nonempty)
1773        } else if let Some(f) = t.fallback.as_ref().filter(|f| all(f.requires)) {
1774            (f.body, None, None, f.requires, f.nonempty)
1775        } else {
1776            continue;
1777        };
1778
1779        // 2. Pick the body for where the data lives. A scope-variant body is
1780        //    always Index-tier: GRAPH/UNION patterns are not summary-shaped
1781        //    (`summary_query_shape` recognizes only a bare default-graph
1782        //    pattern), and the pyramid summary covers the default graph only.
1783        let (mut body, mut tier, reads) = match scope {
1784            GraphScope::DefaultOnly => (base, t.tier, BodyScope::Default),
1785            GraphScope::NamedOnly => match named {
1786                Some(b) => (b, Tier::Index, BodyScope::AllQuads),
1787                // GRAPH-native templates (the ng-* family) already address the
1788                // named graphs; any other default-graph body would be a
1789                // guaranteed-zero-rows query on a file whose default graph the
1790                // card itself records as empty — skip it.
1791                None if requires.contains(&Cap::NamedGraphs) => (base, t.tier, BodyScope::AllQuads),
1792                None => continue,
1793            },
1794            GraphScope::Mixed => match mixed {
1795                Some(b) => (b, Tier::Index, BodyScope::AllQuads),
1796                // The default-graph body is sound here — the profile that
1797                // instantiated it was derived from the (non-empty) default
1798                // graph — and the ng-* family surfaces the named half.
1799                None => (base, t.tier, BodyScope::Default),
1800            },
1801        };
1802
1803        // 2b. The cost gate: a body that would read more than the budget is
1804        //     replaced by its bounded form, or not shipped. See the module docs.
1805        let (mut title, mut question, mut requires, mut nonempty) =
1806            (t.title, t.question, requires, nonempty);
1807        if scan_estimate(t.scan, card, &caps, reads) > budget {
1808            let Some(b) = t.bounded.as_ref().filter(|b| all(b.requires)) else {
1809                continue;
1810            };
1811            body = match scope {
1812                GraphScope::NamedOnly => match b.named_body {
1813                    Some(nb) => nb,
1814                    None => continue,
1815                },
1816                _ => b.body,
1817            };
1818            tier = Tier::Index;
1819            (title, question, requires, nonempty) = (b.title, b.question, b.requires, b.nonempty);
1820        }
1821
1822        // 3. The last gate: refuse a query the card can PROVE would answer
1823        //    nothing. Only in default-graph scope — the card's counts are over
1824        //    the default graph, so they witness nothing about a GRAPH-scoped
1825        //    body (and a `GRAPH`-native template is never asking about the
1826        //    default graph in the first place).
1827        if scope == GraphScope::DefaultOnly && t.provably_empty.is_some_and(|f| f(card)) {
1828            continue;
1829        }
1830
1831        debug_assert!(
1832            check_body(t.id, body, requires, nonempty).is_ok(),
1833            "{}",
1834            check_body(t.id, body, requires, nonempty).unwrap_err()
1835        );
1836        let body = caps.substitute(body);
1837        debug_assert!(
1838            !body.contains("{{"),
1839            "template {} left an unsubstituted placeholder",
1840            t.id
1841        );
1842        out.push(ExampleQuery {
1843            id: t.id.to_string(),
1844            title: title.to_string(),
1845            dimension: t.dimension.to_string(),
1846            question: question.to_string(),
1847            sparql: format!("{PREFIXES}\n{body}"),
1848            tier,
1849            requires: requires.iter().map(|c| c.key().to_string()).collect(),
1850        });
1851    }
1852    out
1853}
1854
1855// ===========================================================================
1856// Auditing a card that is ALREADY published
1857// ===========================================================================
1858//
1859// [`generate`] decides what a card *should* ship. The audit answers the other
1860// half of the same question — what does a card that is already on the wire
1861// ship, and can it answer? — for a file whose queries were written by an older
1862// revision of the table above.
1863//
1864// It is deliberately in this module and not beside it: the one judgement it
1865// makes ("does the card prove these two pieces meet?") is
1866// `class_carries`, the same function `Caps::from_card` resolves
1867// `Cap::LabeledClass` with, and the emptiness gates it applies are the
1868// templates' own `Template::provably_empty` hooks and `NonEmpty` claims. A
1869// second, drifting copy of "is this empty" living in a script is how the
1870// original defect survived four releases.
1871
1872/// What a published starter query is worth on the file that carries it.
1873#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize)]
1874#[serde(rename_all = "kebab-case")]
1875pub enum Verdict {
1876    /// The card **proves** at least one row comes back.
1877    Answers,
1878    /// The card **proves** none does. Airtight for the shapes that turn on a
1879    /// term's existence (a path step missing from a complete predicate list, a
1880    /// `VALUES` list disjoint from the link predicates, a default-graph body on
1881    /// an empty default graph); for the class-∧-predicate shape it rests on the
1882    /// quotient, whose one blind spot `Refuted` documents and narrows.
1883    Empty,
1884    /// A row comes back (an un-grouped aggregate always returns one) but it is
1885    /// vacuous — the thing being counted is provably zero.
1886    Vacuous,
1887    /// The card cannot prove the join is non-empty, and bounds how much could
1888    /// survive it. Not the same as `Empty`: reported separately, never merged.
1889    Suspect,
1890    /// Honestly undecidable from a card — the template says so, or the query
1891    /// conjoins terms nothing in the card can relate.
1892    Undecidable,
1893    /// The body matches no known revision of its template, so nothing is
1894    /// claimed about it.
1895    Unrecognized,
1896    /// The query does not parse. Only a curated `example_queries` entry can be
1897    /// this — the generator's bodies are parsed by the test suite — and it is
1898    /// the one verdict that needs no run to be certain of.
1899    Invalid,
1900}
1901
1902impl Verdict {
1903    pub fn as_str(self) -> &'static str {
1904        match self {
1905            Verdict::Answers => "answers",
1906            Verdict::Empty => "empty",
1907            Verdict::Vacuous => "vacuous",
1908            Verdict::Suspect => "suspect",
1909            Verdict::Undecidable => "undecidable",
1910            Verdict::Unrecognized => "unrecognized",
1911            Verdict::Invalid => "invalid",
1912        }
1913    }
1914}
1915
1916/// The id `rete card-audit` gives the curated `example_queries` entry at
1917/// `index` (0-based): `example-1`, `example-2`, … — curated queries carry no id
1918/// of their own, and a stable positional one is what `--only` can name.
1919pub fn example_query_id(index: usize) -> String {
1920    format!("example-{}", index + 1)
1921}
1922
1923/// One published query's verdict, with the evidence behind it.
1924#[derive(Debug, Clone, Serialize)]
1925pub struct Finding {
1926    pub id: String,
1927    pub verdict: Verdict,
1928    /// The evidence, in the words a maintainer needs to act on it.
1929    pub why: String,
1930    /// What a re-card would do to this query: `current` (byte-identical to what
1931    /// this generator writes for this very card — nothing to fix),
1932    /// `superseded` (the generator writes a different body now), or `dropped`
1933    /// (it would no longer be emitted at all).
1934    pub revision: &'static str,
1935    /// The terms substituted into the body, each tagged with the card ranking
1936    /// it was drawn from. Two terms from **different** rankings conjoined in
1937    /// one body is the #172 shape — recorded for every query, so the census is
1938    /// empirical rather than a re-reading of the table.
1939    pub substitutions: Vec<Substitution>,
1940    /// True when the body conjoins two or more independently-ranked terms.
1941    pub conjoined: bool,
1942    /// What running the query actually did — absent unless `--measure` ran it.
1943    /// Kept **beside** `verdict`, never merged into it: one is what a card can
1944    /// prove about a file, the other is what the file did. Where they disagree
1945    /// the observation wins, and the disagreement is the finding.
1946    #[serde(default, skip_serializing_if = "Option::is_none")]
1947    pub observed: Option<Observation>,
1948}
1949
1950/// One starter query, actually run.
1951#[derive(Debug, Clone, Serialize)]
1952pub struct Observation {
1953    /// `answers` (rows with bindings), `vacuous` (rows that bind nothing),
1954    /// `empty` (no rows at all), or `error` (the run did not finish).
1955    pub outcome: &'static str,
1956    pub rows: u64,
1957    /// Bytes the run read, cold, open included.
1958    pub bytes: u64,
1959    /// Range requests the run made, cold, open included.
1960    pub requests: u64,
1961    /// Wall clock on the measuring machine — a reference, not a property of the
1962    /// file. Read it next to `bytes`/`requests`, never on its own.
1963    pub debug_ms: u64,
1964    #[serde(skip_serializing_if = "Option::is_none")]
1965    pub error: Option<String>,
1966    /// What the file's own build record says this query cost — present only
1967    /// when the file carries one. That is the single case where a
1968    /// re-measurement has a **known answer to check itself against**, so it is
1969    /// checked, and the answer is reported rather than assumed.
1970    #[serde(skip_serializing_if = "Option::is_none")]
1971    pub recorded: Option<Recorded>,
1972}
1973
1974/// The figures a build wrote into the file, beside the ones just measured.
1975#[derive(Debug, Clone, Serialize)]
1976pub struct Recorded {
1977    pub bytes: u64,
1978    pub requests: u64,
1979    pub rows: u64,
1980    /// All three agree with the run. `bytes` and `requests` are properties of
1981    /// layout + query, so they should — through the same transport. A
1982    /// disagreement is worth knowing about: a different reader fan-out, a
1983    /// different engine, or a file that is not the one the record describes.
1984    pub agrees: bool,
1985}
1986
1987impl Observation {
1988    /// Does the observation contradict what the card alone concluded? Only
1989    /// `Answers` and `Empty` are claims strong enough to be wrong; `Suspect`
1990    /// and `Undecidable` say in so many words that they are not claims.
1991    pub fn contradicts(&self, verdict: Verdict) -> bool {
1992        match verdict {
1993            Verdict::Answers => self.outcome != "answers",
1994            Verdict::Empty => self.outcome == "answers",
1995            Verdict::Vacuous => self.outcome == "answers",
1996            _ => false,
1997        }
1998    }
1999}
2000
2001/// A substituted term and where in the card profile it came from.
2002#[derive(Debug, Clone, Serialize)]
2003pub struct Substitution {
2004    pub placeholder: String,
2005    pub value: String,
2006    /// e.g. `classes[0]`, `predicates[3]`, `signals.label_predicate`. Empty
2007    /// when the term is in no ranking the card records (a hard-coded IRI, or a
2008    /// value that fell off a capped list).
2009    pub origin: Vec<String>,
2010}
2011
2012/// Bodies that **earlier revisions** of a template shipped. A published file is
2013/// not re-cardable for free, so the audit has to read the query text that is
2014/// actually out there; only the revisions the current bodies no longer match
2015/// need an entry. Every one is a body `check_body` would reject today — that
2016/// is why it has an entry.
2017struct Legacy {
2018    id: &'static str,
2019    body: &'static str,
2020}
2021
2022const LEGACY_BODIES: &[Legacy] = &[
2023    // pre-#172 `top-reach`: the busiest subject and the most frequent
2024    // predicate, chosen from two independent rankings and then conjoined.
2025    Legacy {
2026        id: "top-reach",
2027        body: "SELECT ?y WHERE { {{HUB_IRI}} {{TOP_PRED}}+ ?y } LIMIT 100",
2028    },
2029    // pre-#172 `lk-sameas`: the VALUES list covered three of the four
2030    // predicates the `signals.link_predicates` gate is detected from.
2031    Legacy {
2032        id: "lk-sameas",
2033        body: "SELECT ?p (COUNT(*) AS ?n) WHERE { ?s ?p ?o VALUES ?p { owl:sameAs skos:exactMatch rdfs:seeAlso } } GROUP BY ?p",
2034    },
2035    // pre-0.4 `top-dangling`: the same rows as today's body, but `FILTER NOT
2036    // EXISTS` materializes its inner pattern — 8.7 GiB on the OpenCitations
2037    // sample. Recognized so a published card audits as `superseded`, not as an
2038    // unreadable body.
2039    Legacy {
2040        id: "top-dangling",
2041        body: "SELECT ?o WHERE { ?s ?p ?o FILTER(isIRI(?o)) FILTER NOT EXISTS { ?o ?p2 ?o2 } } LIMIT 100",
2042    },
2043];
2044
2045/// The effective top-N cap a card's profile lists were derived under. Cards
2046/// built before the field existed omit it; they were built by this same builder
2047/// family, whose cap is a compile-time constant.
2048fn cap_of(card: &DatasetCard) -> usize {
2049    if card.top_n > 0 {
2050        card.top_n as usize
2051    } else {
2052        CARD_TOP_N
2053    }
2054}
2055
2056/// Is a capped profile list **complete** — did nothing fall off the end? A list
2057/// shorter than the cap was never truncated, so absence from it is evidence of
2058/// absence. A list *at* the cap proves nothing either way.
2059fn complete<T>(card: &DatasetCard, list: &[T]) -> bool {
2060    list.len() < cap_of(card)
2061}
2062
2063/// How far the card can go towards proving that no instance of `class` carries
2064/// `pred` — the negative of `class_carries`, which a card can rarely settle
2065/// outright.
2066/// The refutation is exact **for singly-typed data only**, and that is the
2067/// audit's one standing caveat: the quotient gives every subject exactly one
2068/// class (the last `rdf:type` wins), so a subject typed both `class` and
2069/// `other` is counted under `other` alone. `?s a class ; pred ?o` then matches a
2070/// graph whose quotient shows no such row — `vidy` types all 39,723 of its units
2071/// both `schema:ArchiveComponent` and `vidy:Unit`, the quotient files every one
2072/// of them under `vidy:Unit`, and the label query the card ships returns 50 rows
2073/// regardless.
2074///
2075/// A card cannot record multi-typing, but it does leave two tells, and
2076/// [`refute_class_pred`] refuses to claim a refutation when either shows:
2077///
2078///  1. **The class is absent from the quotient.** A class with instances but no
2079///     `class_links` row of its own is a class the quotient is not describing —
2080///     shadowed, or making no statements at all — so its silence about one
2081///     predicate means nothing.
2082///  2. **Another class has exactly the same instance count.** Two classes over
2083///     the same population is what dual typing looks like from the card (a
2084///     schema.org type beside a native one — `vidy` and `arxiu` both), and the
2085///     labelled half may be filed under the twin.
2086///
2087/// Neither is a proof of single-typing; together they are what the card can see.
2088/// Running the query is what closes it.
2089enum Refuted {
2090    /// Every use of `pred` is accounted for in the `class_links` quotient, none
2091    /// of them falls on `class`, and neither multi-typing tell shows.
2092    Proved,
2093    /// The quotient is truncated: at most `budget` uses of `pred` are
2094    /// unaccounted for, so at most that many of the class's `instances` could
2095    /// be joined. Suspicion, not proof.
2096    AtMost { budget: u64, instances: u64 },
2097    /// The quotient accounts for every use of `pred` and none falls on `class`,
2098    /// but a multi-typing tell shows, so that silence proves nothing.
2099    Shadowed { described: bool, twin: bool },
2100    /// `pred` is not in the (capped) predicate list, so there is no total to
2101    /// account against.
2102    Unknown,
2103}
2104
2105fn refute_class_pred(card: &DatasetCard, class: &str, pred: &str) -> Refuted {
2106    let instances = card
2107        .classes
2108        .iter()
2109        .find(|(c, _)| c == class)
2110        .map(|(_, n)| *n)
2111        .unwrap_or(0);
2112    // Every non-`rdf:type` statement contributes exactly one `class_links` row,
2113    // so the rows for a predicate sum to its total use **iff** none was
2114    // truncated away — the completeness test that makes absence conclusive.
2115    let accounted: u64 = card
2116        .class_links
2117        .iter()
2118        .filter(|l| l.predicate == pred)
2119        .map(|l| l.count)
2120        .sum();
2121    let Some((_, total)) = card.predicates.iter().find(|(p, _)| p == pred) else {
2122        return Refuted::Unknown;
2123    };
2124    let accounted_for_all = accounted >= *total || complete(card, &card.class_links);
2125    if !accounted_for_all {
2126        return Refuted::AtMost {
2127            budget: total - accounted,
2128            instances,
2129        };
2130    }
2131    // The two multi-typing tells (see `Refuted`). Either one and the quotient's
2132    // silence about this class stops being evidence.
2133    let described = card.class_links.iter().any(|l| l.s_class == class);
2134    let twin = card
2135        .classes
2136        .iter()
2137        .any(|(c, n)| c != class && *n == instances);
2138    if described && !twin {
2139        Refuted::Proved
2140    } else {
2141        Refuted::Shadowed { described, twin }
2142    }
2143}
2144
2145/// Which card rankings a substituted term appears in. Read off the card rather
2146/// than declared by the template, so it also describes a body this table no
2147/// longer contains.
2148fn origins(card: &DatasetCard, v: &str) -> Vec<String> {
2149    let mut o = Vec::new();
2150    if let Some(i) = card.classes.iter().position(|(c, _)| c == v) {
2151        o.push(format!("classes[{i}]"));
2152    }
2153    if let Some(i) = card.predicates.iter().position(|(p, _)| p == v) {
2154        o.push(format!("predicates[{i}]"));
2155    }
2156    if let Some(i) = card.top_hubs.iter().position(|(h, _)| h == v) {
2157        o.push(format!("top_hubs[{i}]"));
2158    }
2159    let s = &card.signals;
2160    if s.label_predicate.as_deref() == Some(v) {
2161        o.push("signals.label_predicate".to_string());
2162    }
2163    if let Some(i) = s.time_predicates.iter().position(|p| p == v) {
2164        o.push(format!("signals.time_predicates[{i}]"));
2165    }
2166    if let Some(i) = s.numeric_predicates.iter().position(|p| p == v) {
2167        o.push(format!("signals.numeric_predicates[{i}]"));
2168    }
2169    if s.base_iri.as_deref() == Some(v) {
2170        o.push("signals.base_iri".to_string());
2171    }
2172    if let Some(i) = s.annotation_predicates.iter().position(|p| p == v) {
2173        o.push(format!("signals.annotation_predicates[{i}]"));
2174    }
2175    if let Some(i) = s.quoting_predicates.iter().position(|p| p == v) {
2176        o.push(format!("signals.quoting_predicates[{i}]"));
2177    }
2178    if let Some((quoted, about)) = annotated_statement(card) {
2179        if v == quoted {
2180            o.push("signals.annotated_statement.0".to_string());
2181        }
2182        if v == about {
2183            o.push("signals.annotated_statement.1".to_string());
2184        }
2185    }
2186    o
2187}
2188
2189/// The bindings a published body gave a pattern's placeholders.
2190type Binds = Vec<(String, String)>;
2191
2192/// The **distinct** placeholders a body substitutes with a term the card
2193/// ranks. Distinct by name on purpose: the current `top-reach` writes
2194/// `{{OBJECT_PRED}}` twice — once to pick the seed, once to walk from it — and
2195/// a term conjoined with itself is the opposite of the two-maxima shape, it is
2196/// how the template guarantees the seed has an edge.
2197fn distinct_ranked<'a>(card: &DatasetCard, binds: &'a Binds) -> Vec<&'a str> {
2198    let mut names: Vec<&str> = binds
2199        .iter()
2200        .filter(|(_, v)| !origins(card, v).is_empty())
2201        .map(|(n, _)| n.as_str())
2202        .collect();
2203    names.sort_unstable();
2204    names.dedup();
2205    names
2206}
2207
2208fn bound<'a>(binds: &'a Binds, name: &str) -> Option<&'a str> {
2209    binds
2210        .iter()
2211        .find(|(n, _)| n == name)
2212        .map(|(_, v)| v.as_str())
2213}
2214
2215/// Match a published body against a template pattern, recovering the value each
2216/// `{{PLACEHOLDER}}` took. Every literal segment must appear, in order, and the
2217/// body must end exactly where the pattern does — a match is exact, so a
2218/// mismatched revision is reported as such instead of being misread.
2219fn match_pattern(pattern: &str, body: &str) -> Option<Binds> {
2220    let mut binds: Binds = Vec::new();
2221    let mut pat = pattern;
2222    let mut rest = body;
2223    loop {
2224        let Some(i) = pat.find("{{") else {
2225            return (pat == rest).then_some(binds);
2226        };
2227        let (lit, tail) = pat.split_at(i);
2228        rest = rest.strip_prefix(lit)?;
2229        let end = tail.find("}}")?;
2230        let name = &tail[2..end];
2231        pat = &tail[end + 2..];
2232        // The value runs up to the pattern's next literal segment.
2233        let next_lit = &pat[..pat.find("{{").unwrap_or(pat.len())];
2234        let cut = if next_lit.is_empty() {
2235            rest.len()
2236        } else {
2237            rest.find(next_lit)?
2238        };
2239        if cut == 0 {
2240            return None; // an empty substitution is not a substitution
2241        }
2242        binds.push((name.to_string(), rest[..cut].to_string()));
2243        rest = &rest[cut..];
2244    }
2245}
2246
2247/// The `?s a {{CLASS}} … {{PRED}}` shape — a class and a predicate conjoined on
2248/// **one** subject. Recognized from the pattern text, not from the template id,
2249/// so a future template that writes the same shape is audited too.
2250fn class_pred_conjunction(pattern: &str) -> Option<(&str, &str)> {
2251    let after = pattern.split("?s a {{").nth(1)?;
2252    let class = after.split("}}").next()?;
2253    let tail = after.split_once("}}")?.1;
2254    for sep in ["; {{", "?s {{"] {
2255        if let Some(x) = tail.split(sep).nth(1) {
2256            return Some((class, x.split("}}").next()?));
2257        }
2258    }
2259    None
2260}
2261
2262/// The `VALUES ?p { … }` list a body pins itself to, as bracketed IRIs. The
2263/// gate that admits the query counts predicates; the body only matches the ones
2264/// it names, and the two can disagree.
2265fn values_predicates(body: &str) -> Option<Vec<String>> {
2266    let inner = body.split("VALUES ?p {").nth(1)?.split('}').next()?;
2267    Some(
2268        inner
2269            .split_whitespace()
2270            .filter_map(|t| {
2271                let (p, local) = t.split_once(':')?;
2272                let ns = match p {
2273                    "owl" => "http://www.w3.org/2002/07/owl#",
2274                    "skos" => "http://www.w3.org/2004/02/skos/core#",
2275                    "rdfs" => "http://www.w3.org/2000/01/rdf-schema#",
2276                    _ => return None,
2277                };
2278                Some(format!("<{ns}{local}>"))
2279            })
2280            .collect(),
2281    )
2282}
2283
2284/// The predicate IRIs a property path **must** traverse — the steps that are
2285/// not optional. `geo:hasGeometry?/geo:asWKT` requires only the second; the
2286/// fixed `geo:hasGeometry/geo:asWKT` the pre-#172 template wrote requires both,
2287/// which is why it could not match a file that has no `geo:hasGeometry`.
2288fn required_path_steps(path: &str) -> Vec<String> {
2289    path.split('/')
2290        .filter(|step| !step.ends_with('?') && !step.ends_with('*'))
2291        .filter_map(|step| match step.trim_end_matches('+') {
2292            "geo:asWKT" => Some(GEO_ASWKT.to_string()),
2293            "geo:hasGeometry" => Some(GEO_HASGEOMETRY.to_string()),
2294            _ => None,
2295        })
2296        .collect()
2297}
2298
2299/// Audit every starter query a card already ships.
2300///
2301/// Two things happen per query, and they answer different questions. [`generate`]
2302/// is re-run over the same profile — what a re-card would ship — which says
2303/// whether the query is *stale*. The verdict says whether it *answers*, which is
2304/// the thing a reader of the published file experiences today.
2305pub fn audit(card: &DatasetCard) -> Vec<Finding> {
2306    let caps = Caps::from_card(card);
2307    let scope = GraphScope::of(card);
2308    let fresh = generate(card);
2309    card.queries
2310        .iter()
2311        .map(|q| {
2312            let revision = match fresh.iter().find(|r| r.id == q.id) {
2313                Some(r) if r.sparql == q.sparql => "current",
2314                Some(_) => "superseded",
2315                None => "dropped",
2316            };
2317            finding(card, &caps, scope, q, revision)
2318        })
2319        .collect()
2320}
2321
2322/// Audit the **curated** `example_queries` a card carries — the half of a card a
2323/// publisher wrote, which [`audit`] (the generated half) never looks at.
2324///
2325/// A curated query makes no claim, so there is no template to hold it to; what
2326/// the card *can* still decide is narrower and stated as such:
2327///
2328/// * it does not parse — [`Verdict::Invalid`], certain;
2329/// * it reads only the default graph of a file whose default graph the card
2330///   records as empty — [`Verdict::Empty`], the same scope rule the starters
2331///   obey (#170);
2332/// * it names a predicate the card's **complete** predicate list does not
2333///   contain — [`Verdict::Suspect`], not `Empty`: the predicate may sit in an
2334///   `OPTIONAL`, `UNION` or `MINUS` branch, and the profile covers the default
2335///   graph only;
2336/// * anything else — [`Verdict::Undecidable`]; `card-audit --measure` runs it.
2337///
2338/// Findings use the ids of [`example_query_id`] and `revision: "curated"`, and
2339/// are reported apart from the generated queries' — they answer to a
2340/// different author.
2341pub fn audit_examples(card: &DatasetCard) -> Vec<Finding> {
2342    card.example_queries
2343        .iter()
2344        .enumerate()
2345        .map(|(i, sparql)| curated_finding(card, example_query_id(i), sparql))
2346        .collect()
2347}
2348
2349fn curated_finding(card: &DatasetCard, id: String, sparql: &str) -> Finding {
2350    let mk = |verdict, why: String| Finding {
2351        id: id.clone(),
2352        verdict,
2353        why,
2354        revision: "curated",
2355        substitutions: Vec::new(),
2356        conjoined: false,
2357        observed: None,
2358    };
2359    let predicates = match crate::query_predicates(sparql) {
2360        Ok(p) => p,
2361        Err(e) => return mk(Verdict::Invalid, format!("does not parse: {e}")),
2362    };
2363    // Keyword tests on the text, as the starter audit's scope rule does: a
2364    // `GRAPH` or a dataset clause anywhere means the query does not read only
2365    // the default graph.
2366    let upper = strip_prefixes(sparql).to_ascii_uppercase();
2367    let scoped = upper.contains("GRAPH") || upper.contains("FROM");
2368    if !scoped && card.triple_count == 0 && card.named_graph_count > 0 {
2369        return mk(
2370            Verdict::Empty,
2371            format!(
2372                "it reads only the default graph, which the card records as empty \
2373                 ({} named graph(s) hold all {} statements)",
2374                card.named_graph_count, card.quad_count
2375            ),
2376        );
2377    }
2378    let missing: Vec<&String> = predicates
2379        .iter()
2380        .filter(|p| !card.predicates.iter().any(|(q, _)| q == *p))
2381        .collect();
2382    if !missing.is_empty() && !card.predicates.is_empty() {
2383        let names = missing
2384            .iter()
2385            .map(|s| s.as_str())
2386            .collect::<Vec<_>>()
2387            .join(" ");
2388        return if complete(card, &card.predicates) && card.named_graph_count == 0 {
2389            mk(
2390                Verdict::Suspect,
2391                format!(
2392                    "names {names}, which the card's complete predicate list does not contain \
2393                     — no row can match unless it sits in an OPTIONAL/UNION/MINUS branch"
2394                ),
2395            )
2396        } else {
2397            mk(
2398                Verdict::Undecidable,
2399                format!(
2400                    "names {names}, which the card's predicate list does not show — the list is \
2401                     capped, or the predicate may live in a named graph the profile does not cover"
2402                ),
2403            )
2404        };
2405    }
2406    mk(
2407        Verdict::Undecidable,
2408        "a curated query makes no claim the card can check; `card-audit --measure` runs it"
2409            .to_string(),
2410    )
2411}
2412
2413fn finding(
2414    card: &DatasetCard,
2415    caps: &Caps,
2416    scope: GraphScope,
2417    q: &ExampleQuery,
2418    revision: &'static str,
2419) -> Finding {
2420    let body = strip_prefixes(&q.sparql);
2421    let body = body.trim();
2422
2423    let mk = |verdict, why: String, revision, binds: &Binds| {
2424        let substitutions: Vec<Substitution> = binds
2425            .iter()
2426            .map(|(n, v)| Substitution {
2427                placeholder: n.clone(),
2428                value: v.clone(),
2429                origin: origins(card, v),
2430            })
2431            .collect();
2432        Finding {
2433            id: q.id.clone(),
2434            verdict,
2435            why,
2436            revision,
2437            conjoined: distinct_ranked(card, binds).len() > 1,
2438            substitutions,
2439            observed: None,
2440        }
2441    };
2442
2443    let Some(t) = TEMPLATES.iter().find(|t| t.id == q.id) else {
2444        return mk(
2445            Verdict::Unrecognized,
2446            "no template of this id is in the library any more".to_string(),
2447            "retired",
2448            &Vec::new(),
2449        );
2450    };
2451
2452    // Recover the substituted terms: match the body against the template's own
2453    // patterns (every scope variant, the fallback and the bounded form), then
2454    // against the bodies earlier revisions wrote. See `variant_of`.
2455    let Some(Variant {
2456        pattern,
2457        binds,
2458        nonempty,
2459        bounded,
2460    }) = variant_of(t, body)
2461    else {
2462        return mk(
2463            Verdict::Unrecognized,
2464            "the body matches no known revision of this template".to_string(),
2465            "unknown",
2466            &Vec::new(),
2467        );
2468    };
2469
2470    // --- 1. Scope. A default-graph body on a file whose default graph the card
2471    //        itself records as empty cannot return a row (issue #170).
2472    if !body.contains("GRAPH") && card.triple_count == 0 && card.named_graph_count > 0 {
2473        return mk(
2474            Verdict::Empty,
2475            format!(
2476                "the body scans the default graph, which the card records as empty \
2477                 ({} named graph(s) hold all {} statements)",
2478                card.named_graph_count, card.quad_count
2479            ),
2480            revision,
2481            &binds,
2482        );
2483    }
2484
2485    // --- 2. The template's own last gate, unchanged — and, where the table says
2486    //        the hook is an equivalence, its other direction.
2487    if scope == GraphScope::DefaultOnly {
2488        if let Some(hook) = t.provably_empty {
2489            if hook(card) {
2490                return mk(
2491                    Verdict::Empty,
2492                    "the card's own precompute for this query is empty (the generator \
2493                     refuses to emit it today)"
2494                        .to_string(),
2495                    revision,
2496                    &binds,
2497                );
2498            }
2499            if t.hook_is_exact && !bounded {
2500                return mk(
2501                    Verdict::Answers,
2502                    "the card precomputes this query's own result, and it is not empty".to_string(),
2503                    revision,
2504                    &binds,
2505                );
2506            }
2507        }
2508    }
2509
2510    // --- 3. The co-occurrence rule, applied to the body that shipped.
2511    if let Some((cn, pn)) = class_pred_conjunction(pattern) {
2512        if let (Some(class), Some(pred)) = (bound(&binds, cn), bound(&binds, pn)) {
2513            let aggregate = matches!(nonempty, NonEmpty::Aggregate);
2514            if class_carries(card, class, pred) {
2515                return mk(
2516                    Verdict::Answers,
2517                    format!("class_links witnesses {class} carrying {pred}"),
2518                    revision,
2519                    &binds,
2520                );
2521            }
2522            return match refute_class_pred(card, class, pred) {
2523                Refuted::Proved => mk(
2524                    if aggregate {
2525                        Verdict::Vacuous
2526                    } else {
2527                        Verdict::Empty
2528                    },
2529                    format!(
2530                        "the class_links quotient accounts for every use of {pred} and \
2531                         none falls on {class}{}",
2532                        if aggregate {
2533                            " — the aggregate returns one row whose count is 0"
2534                        } else {
2535                            ""
2536                        }
2537                    ),
2538                    revision,
2539                    &binds,
2540                ),
2541                Refuted::AtMost { budget, instances } => mk(
2542                    Verdict::Suspect,
2543                    format!(
2544                        "no class_links row witnesses {class} carrying {pred}; the quotient \
2545                         is truncated, so at most {budget} of the class's {instances} \
2546                         instances could be joined"
2547                    ),
2548                    revision,
2549                    &binds,
2550                ),
2551                Refuted::Shadowed { described, twin } => mk(
2552                    Verdict::Suspect,
2553                    format!(
2554                        "the quotient accounts for every use of {pred} and none falls on \
2555                         {class}, but {} — the quotient files a multi-typed subject under \
2556                         one class only, so its silence is not evidence",
2557                        match (described, twin) {
2558                            (false, true) =>
2559                                "the class has no class_links row of its own AND another class \
2560                                 has exactly its instance count",
2561                            (false, false) => "the class has no class_links row of its own",
2562                            _ => "another class has exactly its instance count",
2563                        }
2564                    ),
2565                    revision,
2566                    &binds,
2567                ),
2568                Refuted::Unknown => mk(
2569                    Verdict::Undecidable,
2570                    format!("{pred} is not in the card's (capped) predicate list"),
2571                    revision,
2572                    &binds,
2573                ),
2574            };
2575        }
2576    }
2577
2578    // --- 4. A property path can only match if the steps it insists on exist.
2579    if let Some(path) = bound(&binds, "WKT_PATH") {
2580        let missing: Vec<String> = required_path_steps(path)
2581            .into_iter()
2582            .filter(|iri| !card.predicates.iter().any(|(p, _)| p == iri))
2583            .collect();
2584        if !missing.is_empty() && complete(card, &card.predicates) {
2585            return mk(
2586                Verdict::Empty,
2587                format!(
2588                    "the path {path} requires {} , which the card's complete predicate \
2589                     list does not contain",
2590                    missing.join(", ")
2591                ),
2592                revision,
2593                &binds,
2594            );
2595        }
2596    }
2597
2598    // --- 5. A pinned VALUES list must cover the gate that admitted the query.
2599    if let Some(listed) = values_predicates(body) {
2600        let present: Vec<&String> = card
2601            .signals
2602            .link_predicates
2603            .iter()
2604            .filter(|p| listed.contains(p))
2605            .collect();
2606        if present.is_empty() {
2607            return mk(
2608                Verdict::Empty,
2609                format!(
2610                    "the body pins VALUES ?p to {} , none of which this dataset uses \
2611                     (it links with {})",
2612                    listed.join(" "),
2613                    card.signals.link_predicates.join(" ")
2614                ),
2615                revision,
2616                &binds,
2617            );
2618        }
2619    }
2620
2621    // --- 6. "Which predicates point OUT?" needs something outside the base IRI.
2622    if q.id == "lk-external" && !caps.external_iri {
2623        let why = "no IRI the card records lies outside the dataset's base IRI".to_string();
2624        return if complete(card, &card.in_hubs) && complete(card, &card.classes) {
2625            mk(
2626                Verdict::Empty,
2627                format!("{why}, and both lists are complete — the FILTER keeps nothing"),
2628                revision,
2629                &binds,
2630            )
2631        } else {
2632            mk(
2633                Verdict::Suspect,
2634                format!("{why}, but the lists are truncated so one could have fallen off"),
2635                revision,
2636                &binds,
2637            )
2638        };
2639    }
2640
2641    // --- 6b. The RDF-star conjunction. `ANNOTATED_PRED` (inside the quoted
2642    //         triple) and `ANNOTATION_PRED` (outside it) are the second pair
2643    //         the card can witness, so step 7 below — which would see two
2644    //         ranked predicates and decline — must not reach them. What the
2645    //         published body actually bound is compared against the card's own
2646    //         witness: a re-derivation can move the pair, and a body carrying
2647    //         the old one is no longer proven.
2648    if let (Some(quoted), Some(about)) = (
2649        bound(&binds, "ANNOTATED_PRED"),
2650        bound(&binds, "ANNOTATION_PRED"),
2651    ) {
2652        return match annotated_statement(card) {
2653            Some((q, a)) if q == quoted && a == about => mk(
2654                Verdict::Answers,
2655                format!(
2656                    "signals.annotated_statement witnesses a {quoted} statement this graph \
2657                     both asserts and annotates with {about}"
2658                ),
2659                revision,
2660                &binds,
2661            ),
2662            Some((q, a)) => mk(
2663                Verdict::Suspect,
2664                format!(
2665                    "the body pins {quoted} / {about}, but the card's witnessed pair is \
2666                     {q} / {a} — nothing here proves these two meet on one statement"
2667                ),
2668                revision,
2669                &binds,
2670            ),
2671            None => mk(
2672                Verdict::Undecidable,
2673                "the card records no annotated-statement witness, so it cannot say whether \
2674                 the quoted predicate and the annotating one ever meet"
2675                    .to_string(),
2676                revision,
2677                &binds,
2678            ),
2679        };
2680    }
2681
2682    // --- 7. Two independently-ranked terms conjoined, with nothing in the card
2683    //        able to relate them: the shape, without a decision.
2684    let ranked = distinct_ranked(card, &binds);
2685    if ranked.len() > 1 {
2686        return mk(
2687            Verdict::Undecidable,
2688            format!(
2689                "conjoins {} , drawn from different rankings — the card relates no \
2690                 specific subject to a specific predicate",
2691                ranked.join(" and ")
2692            ),
2693            revision,
2694            &binds,
2695        );
2696    }
2697
2698    // --- 8. An un-grouped aggregate that joins two predicates on one subject
2699    //        always returns its row; whether the row says anything is another
2700    //        matter, and one no card can settle (`class_links` collapses every
2701    //        untyped subject into one bucket, so rows for the two predicates
2702    //        under it are not evidence that any single subject carries both).
2703    if matches!(nonempty, NonEmpty::Aggregate) && body.contains(" ; ") && !body.contains("GROUP BY")
2704    {
2705        return mk(
2706            Verdict::Undecidable,
2707            "an un-grouped aggregate always returns one row, but its body conjoins two \
2708             predicates the card only counted separately — the row may be vacuous"
2709                .to_string(),
2710            revision,
2711            &binds,
2712        );
2713    }
2714
2715    // --- 9. Nothing above bit: the template's own claim stands.
2716    match nonempty {
2717        NonEmpty::Undecidable(reason) => {
2718            mk(Verdict::Undecidable, reason.to_string(), revision, &binds)
2719        }
2720        NonEmpty::AnyGraph | NonEmpty::Aggregate | NonEmpty::Witnessed => mk(
2721            Verdict::Answers,
2722            match nonempty {
2723                NonEmpty::AnyGraph => "any non-empty graph answers this",
2724                NonEmpty::Aggregate => "an un-grouped aggregate always returns one row",
2725                _ => "every substituted term came from a count the card records as > 0",
2726            }
2727            .to_string(),
2728            revision,
2729            &binds,
2730        ),
2731    }
2732}
2733
2734#[cfg(test)]
2735mod tests {
2736    use super::*;
2737    use crate::card_derive::{derive_card, CardInput};
2738    use crate::{eval_query, ingest, summary_query_shape, QueryOutput, Rete};
2739
2740    const TYPE: &str = "<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>";
2741
2742    fn q(s: &str, p: &str, o: &str) -> (String, String, String, Option<String>) {
2743        (s.into(), p.into(), o.into(), None)
2744    }
2745
2746    /// Ids whose template admits it cannot prove non-emptiness. The allow-list
2747    /// is **read off the table** rather than written out here, so a template
2748    /// cannot quietly join it: adding `NonEmpty::Undecidable` to one means
2749    /// writing down the reason, in the source, next to the body.
2750    fn may_be_empty(id: &str) -> bool {
2751        claim_of(id) == Claim::Undecidable
2752    }
2753
2754    /// The property that would have caught the named-graph-only card bug — and
2755    /// the `lb-labels` one: every query the card ships **returns a non-empty
2756    /// result** when run against the very graph it was generated for.
2757    ///
2758    /// The only exemptions are the templates that declare themselves
2759    /// undecidable; everything else answering nothing is a generator bug, not a
2760    /// fact about the dataset.
2761    fn assert_every_query_returns_rows(card: &DatasetCard, rete: &Rete) {
2762        assert!(!card.queries.is_empty(), "card generated no queries at all");
2763        for eq in &card.queries {
2764            let allow_empty = may_be_empty(&eq.id);
2765            match eval_query(rete, &eq.sparql) {
2766                Ok(QueryOutput::Select(_, rows)) => assert!(
2767                    allow_empty || !rows.is_empty(),
2768                    "{}: returned zero rows:\n{}",
2769                    eq.id,
2770                    eq.sparql
2771                ),
2772                Ok(QueryOutput::Ask(b)) => assert!(allow_empty || b, "{}: ASK false", eq.id),
2773                Ok(QueryOutput::Construct(ts)) => {
2774                    assert!(
2775                        allow_empty || !ts.is_empty(),
2776                        "{}: constructed nothing",
2777                        eq.id
2778                    )
2779                }
2780                // No catch-all arm: `QueryOutput`'s `#[non_exhaustive]` does not
2781                // apply inside its own crate, so the three arms above are
2782                // exhaustive here — and a fourth result form should red this
2783                // test until someone decides what the query library does with it.
2784                Err(e) => panic!("{}: failed to run: {e:?}", eq.id),
2785            }
2786        }
2787    }
2788
2789    /// Build the fixture and return `(card, image)` — the pair every
2790    /// returns-rows test needs.
2791    fn card_and_graph(
2792        quads: Vec<(String, String, String, Option<String>)>,
2793        term_count: u64,
2794        named_graphs: u64,
2795    ) -> (DatasetCard, Rete) {
2796        let card = derive_card(&quads, term_count, named_graphs, CardInput::default());
2797        let (bytes, _) =
2798            ingest::assemble_dataset_with_opts(quads, true, false, None, |_, _| Vec::new());
2799        (card, Rete::open(&bytes).unwrap())
2800    }
2801
2802    fn sparql_of<'a>(card: &'a DatasetCard, id: &str) -> Option<&'a str> {
2803        card.queries
2804            .iter()
2805            .find(|q| q.id == id)
2806            .map(|q| q.sparql.as_str())
2807    }
2808
2809    /// The single COUNT value of an aggregate query's one-row result.
2810    fn count_value(rete: &Rete, sparql: &str, var: &str) -> u64 {
2811        match eval_query(rete, sparql).unwrap() {
2812            QueryOutput::Select(_, rows) => {
2813                assert_eq!(rows.len(), 1, "expected one aggregate row");
2814                let lit = rows[0].get(var).expect("aggregate variable bound");
2815                lit.trim_start_matches('"')
2816                    .split('"')
2817                    .next()
2818                    .unwrap()
2819                    .parse()
2820                    .expect("count literal parses")
2821            }
2822            other => panic!("expected a SELECT result, got {other:?}"),
2823        }
2824    }
2825
2826    /// A fixture exercising every signal so (almost) every template emits.
2827    fn rich_quads() -> Vec<(String, String, String, Option<String>)> {
2828        let label = "<http://www.w3.org/2000/01/rdf-schema#label>";
2829        let sameas = "<http://www.w3.org/2002/07/owl#sameAs>";
2830        let lat = "<http://www.w3.org/2003/01/geo/wgs84_pos#lat>";
2831        let long = "<http://www.w3.org/2003/01/geo/wgs84_pos#long>";
2832        let aswkt = "<http://www.opengis.net/ont/geosparql#asWKT>";
2833        let hasgeom = "<http://www.opengis.net/ont/geosparql#hasGeometry>";
2834        let gyear = "http://www.w3.org/2001/XMLSchema#gYear";
2835        let xint = "http://www.w3.org/2001/XMLSchema#integer";
2836        let wkt = "http://www.opengis.net/ont/geosparql#wktLiteral";
2837        let dbl = "http://www.w3.org/2001/XMLSchema#double";
2838        let mut v = Vec::new();
2839        for i in 0..6 {
2840            let s = format!("<http://ex/person/{i}>");
2841            v.push(q(&s, TYPE, "<http://ex/Person>"));
2842            v.push(q(&s, label, &format!("\"Person {i}\"@en")));
2843            v.push(q(&s, label, &format!("\"Personne {i}\"@fr")));
2844            v.push(q(
2845                &s,
2846                "<http://ex/birthYear>",
2847                &format!("\"{}\"^^<{gyear}>", 1800 + i),
2848            ));
2849            v.push(q(
2850                &s,
2851                "<http://ex/friendCount>",
2852                &format!("\"{}\"^^<{xint}>", i * 3),
2853            ));
2854            v.push(q(
2855                &s,
2856                "<http://ex/knows>",
2857                &format!("<http://ex/person/{}>", (i + 1) % 6),
2858            ));
2859            v.push(q(&s, "<http://ex/bornIn>", "<http://ex/place/0>"));
2860            if i % 2 == 0 {
2861                v.push(q(
2862                    &s,
2863                    sameas,
2864                    &format!("<http://www.wikidata.org/entity/Q{}>", 100 + i),
2865                ));
2866            }
2867        }
2868        v.push(q("<http://ex/place/0>", TYPE, "<http://ex/Place>"));
2869        v.push(q("<http://ex/place/0>", label, "\"Rome\"@en"));
2870        v.push(q("<http://ex/place/0>", lat, &format!("\"41.9\"^^<{dbl}>")));
2871        v.push(q(
2872            "<http://ex/place/0>",
2873            long,
2874            &format!("\"12.5\"^^<{dbl}>"),
2875        ));
2876        v.push(q(
2877            "<http://ex/place/0>",
2878            hasgeom,
2879            "<http://ex/place/0/geom>",
2880        ));
2881        v.push(q(
2882            "<http://ex/place/0/geom>",
2883            aswkt,
2884            &format!("\"POINT(12.5 41.9)\"^^<{wkt}>"),
2885        ));
2886        v
2887    }
2888
2889    #[test]
2890    fn generated_queries_parse_run_and_are_tiered() {
2891        let quads = rich_quads();
2892        let card = derive_card(&quads, 50, 0, CardInput::default());
2893        assert!(
2894            card.queries.len() >= 12,
2895            "expected >= 12 starter queries, got {}",
2896            card.queries.len()
2897        );
2898
2899        let (bytes, _) =
2900            ingest::assemble_dataset_with_opts(quads, true, false, None, |_, _| Vec::new());
2901        let rete = Rete::open(&bytes).unwrap();
2902
2903        for eq in &card.queries {
2904            assert!(
2905                eq.sparql.starts_with("PREFIX"),
2906                "{}: no PREFIX block",
2907                eq.id
2908            );
2909            assert!(
2910                !eq.sparql.contains("{{"),
2911                "{}: unsubstituted placeholder",
2912                eq.id
2913            );
2914            assert!(!eq.dimension.is_empty(), "{}: missing dimension", eq.id);
2915
2916            // Parses AND runs against the real engine (>= 0 rows, never an error).
2917            let result = eval_query(&rete, &eq.sparql);
2918            assert!(
2919                result.is_ok(),
2920                "{}: failed to run: {:?}",
2921                eq.id,
2922                result.err()
2923            );
2924
2925            // Every Summary-tagged query is recognized by the index-free summary
2926            // classifier — the tier is truthful, not aspirational.
2927            if eq.tier == Tier::Summary {
2928                let shape = summary_query_shape(&eq.sparql);
2929                assert!(
2930                    matches!(shape, Ok(Some(_))),
2931                    "{}: tagged Summary but not summary-shaped: {:?}",
2932                    eq.id,
2933                    shape
2934                );
2935            }
2936        }
2937
2938        // The default-only scope gets the same returns-rows contract the
2939        // named-only and mixed ones have had since #154 — the gap through which
2940        // `lb-labels`, `top-reach` and `sp-within` shipped broken on
2941        // default-graph files, which is the population most datasets are in.
2942        assert_every_query_returns_rows(&card, &rete);
2943    }
2944
2945    /// Every statement in a named graph, the default graph empty — the shape
2946    /// (e.g. nkod.rete) whose cards used to ship six guaranteed-zero-rows
2947    /// queries. Includes an IRI that is referenced but never described, so
2948    /// `top-dangling` has rows to find.
2949    fn named_only_quads() -> Vec<(String, String, String, Option<String>)> {
2950        let mut v = Vec::new();
2951        for gi in 0..3 {
2952            let g = format!("<http://ex/graph/{gi}>");
2953            for i in 0..4 {
2954                let s = format!("<http://ex/item/{gi}-{i}>");
2955                v.push((
2956                    s.clone(),
2957                    TYPE.to_string(),
2958                    "<http://ex/Item>".to_string(),
2959                    Some(g.clone()),
2960                ));
2961                v.push((
2962                    s.clone(),
2963                    "<http://ex/rel>".to_string(),
2964                    format!("<http://ex/item/{gi}-{}>", (i + 1) % 4),
2965                    Some(g.clone()),
2966                ));
2967                v.push((
2968                    s,
2969                    "<http://ex/ref>".to_string(),
2970                    "<http://other/never-described>".to_string(),
2971                    Some(g.clone()),
2972                ));
2973            }
2974        }
2975        v
2976    }
2977
2978    #[test]
2979    fn named_graph_only_queries_are_scoped_and_return_rows() {
2980        let quads = named_only_quads();
2981        let total = quads.len() as u64;
2982        let card = derive_card(&quads, 20, 3, CardInput::default());
2983        // The precondition of the bug: the card itself knows the default graph
2984        // is empty and the data lives in named graphs.
2985        assert_eq!(card.triple_count, 0);
2986        assert_eq!(card.named_graph_count, 3);
2987
2988        // No emitted query may scan only the (empty) default graph.
2989        for eq in &card.queries {
2990            assert!(
2991                eq.sparql.contains("GRAPH"),
2992                "{}: default-graph-only query on a named-graph-only file:\n{}",
2993                eq.id,
2994                eq.sparql
2995            );
2996            // The pyramid summary covers the default graph only, so no
2997            // GRAPH-scoped query may claim the Summary tier.
2998            assert_ne!(eq.tier, Tier::Summary, "{}: untruthful Summary tier", eq.id);
2999        }
3000
3001        let (bytes, _) =
3002            ingest::assemble_dataset_with_opts(quads, true, false, None, |_, _| Vec::new());
3003        let rete = Rete::open(&bytes).unwrap();
3004        assert_every_query_returns_rows(&card, &rete);
3005
3006        // The headline count now reflects the data, not the empty default graph.
3007        let ov = card.queries.iter().find(|q| q.id == "ov-triples").unwrap();
3008        assert_eq!(count_value(&rete, &ov.sparql, "n"), total);
3009    }
3010
3011    #[test]
3012    fn mixed_graph_queries_cover_both_sides_and_return_rows() {
3013        // Data in BOTH the default graph and a named graph.
3014        let mut quads = Vec::new();
3015        let label = "<http://www.w3.org/2000/01/rdf-schema#label>";
3016        for i in 0..4 {
3017            let s = format!("<http://ex/doc/{i}>");
3018            quads.push(q(&s, TYPE, "<http://ex/Doc>"));
3019            quads.push(q(&s, label, &format!("\"Doc {i}\"@en")));
3020            quads.push(q(
3021                &s,
3022                "<http://ex/rel>",
3023                &format!("<http://ex/doc/{}>", (i + 1) % 4),
3024            ));
3025            quads.push(q(&s, "<http://ex/rel>", "<http://other/never-described>"));
3026        }
3027        let default_count = quads.len() as u64;
3028        let g = "<http://ex/graph/annotations>".to_string();
3029        for i in 0..5 {
3030            quads.push((
3031                format!("<http://ex/note/{i}>"),
3032                "<http://ex/about>".to_string(),
3033                format!("<http://ex/doc/{}>", i % 4),
3034                Some(g.clone()),
3035            ));
3036        }
3037        let total = quads.len() as u64;
3038
3039        let card = derive_card(&quads, 30, 1, CardInput::default());
3040        assert_eq!(card.triple_count, default_count);
3041        assert_eq!(card.quad_count, total);
3042        assert_eq!(card.named_graph_count, 1);
3043
3044        let (bytes, _) =
3045            ingest::assemble_dataset_with_opts(quads, true, false, None, |_, _| Vec::new());
3046        let rete = Rete::open(&bytes).unwrap();
3047        assert_every_query_returns_rows(&card, &rete);
3048
3049        // The overview count covers BOTH sides — neither half silently hidden.
3050        let ov = card.queries.iter().find(|q| q.id == "ov-triples").unwrap();
3051        assert_eq!(count_value(&rete, &ov.sparql, "n"), total);
3052        // And the named-graph family is present alongside the profile queries.
3053        let ids: Vec<&str> = card.queries.iter().map(|q| q.id.as_str()).collect();
3054        for id in ["ng-list", "ng-sizes", "ng-sample", "id-sample", "lb-labels"] {
3055            assert!(ids.contains(&id), "{id} missing from a mixed-graph card");
3056        }
3057    }
3058
3059    #[test]
3060    fn default_graph_bodies_are_unchanged() {
3061        // A graph with no named graphs must get the classic bodies verbatim —
3062        // scope awareness may not perturb existing default-graph cards.
3063        let quads = rich_quads();
3064        let card = derive_card(&quads, 50, 0, CardInput::default());
3065        let ov = card.queries.iter().find(|q| q.id == "ov-triples").unwrap();
3066        assert!(ov
3067            .sparql
3068            .ends_with("SELECT (COUNT(*) AS ?n) WHERE { ?s ?p ?o }"));
3069        assert_eq!(ov.tier, Tier::Summary);
3070        for eq in &card.queries {
3071            assert!(
3072                !eq.sparql.contains("GRAPH ?g") && !eq.sparql.contains("UNION"),
3073                "{}: graph-scoped body leaked into a default-graph card",
3074                eq.id
3075            );
3076        }
3077    }
3078
3079    /// The one-row smoke query returns EXACTLY one row in every graph scope —
3080    /// the unambiguous "did this file open and answer?" probe (a COUNT returns
3081    /// an honest 0 on a named-graph-only file and reads as failure).
3082    #[test]
3083    fn one_row_smoke_query_returns_exactly_one_row_in_every_scope() {
3084        type Quads = Vec<(String, String, String, Option<String>)>;
3085        let cases: Vec<(Quads, u64)> = vec![
3086            (rich_quads(), 0),       // default-only
3087            (named_only_quads(), 3), // named-graph-only
3088        ];
3089        for (quads, ng) in cases {
3090            let card = derive_card(&quads, 50, ng, CardInput::default());
3091            let smoke = card
3092                .queries
3093                .iter()
3094                .find(|q| q.id == "ov-one-row")
3095                .expect("ov-one-row always emits");
3096            let (bytes, _) =
3097                ingest::assemble_dataset_with_opts(quads, true, false, None, |_, _| Vec::new());
3098            let rete = Rete::open(&bytes).unwrap();
3099            match eval_query(&rete, &smoke.sparql).unwrap() {
3100                QueryOutput::Select(_, rows) => {
3101                    assert_eq!(rows.len(), 1, "ov-one-row must return exactly one row")
3102                }
3103                other => panic!("unexpected result form: {other:?}"),
3104            }
3105        }
3106    }
3107
3108    #[test]
3109    fn families_are_gated_by_signal() {
3110        // A bare graph: no labels, no time, no geometry, no links, no named graphs.
3111        let quads = vec![
3112            q("<http://ex/a>", TYPE, "<http://ex/T>"),
3113            q("<http://ex/a>", "<http://ex/rel>", "<http://ex/b>"),
3114        ];
3115        let card = derive_card(&quads, 4, 0, CardInput::default());
3116        let ids: Vec<&str> = card.queries.iter().map(|q| q.id.as_str()).collect();
3117
3118        // Signal-gated families are absent.
3119        for absent in [
3120            "sp-bbox",
3121            "sp-within",
3122            "ti-extent",
3123            "ti-histogram",
3124            "lb-labels",
3125            "ng-list",
3126            "ng-sample",
3127        ] {
3128            assert!(!ids.contains(&absent), "{absent} should be gated out");
3129        }
3130        // Unconditional overview queries are still present.
3131        for present in ["ov-triples", "ov-pred-list", "ov-pred-hist"] {
3132            assert!(ids.contains(&present), "{present} should always emit");
3133        }
3134        // Every object IRI is inside the dataset's own namespace, so "which
3135        // predicates link out?" has nothing to find — and is not emitted.
3136        assert!(
3137            !ids.contains(&"lk-external"),
3138            "lk-external has no external IRI"
3139        );
3140    }
3141
3142    // ---------------------------------------------------------------------
3143    // The co-occurrence property: a body may conjoin only what the card proves
3144    // meets. Three fixtures, each reproducing the shape of a real dataset whose
3145    // published card ships a query that answers nothing.
3146    // ---------------------------------------------------------------------
3147
3148    /// The **mtg** shape: the most populous class is NOT the one that carries
3149    /// the label predicate.
3150    ///
3151    /// `mtg:Ruling` has 76,990 instances and no `schema:name`; `mtg:Card` and
3152    /// its subclasses are fewer and carry all the names. A card generated from
3153    /// this graph must not join the top class to the label predicate, because
3154    /// the two never meet — that query returns zero rows on a plain
3155    /// default-graph file, and a reader concludes the file is broken.
3156    fn mtg_shaped_quads() -> Vec<(String, String, String, Option<String>)> {
3157        let name = "<http://schema.org/name>";
3158        let mut v = Vec::new();
3159        // 30 Rulings: the most populous class, no name anywhere on it.
3160        for i in 0..30 {
3161            let s = format!("<http://ex/ruling/{i}>");
3162            v.push(q(&s, TYPE, "<http://ex/Ruling>"));
3163            v.push(q(&s, "<http://ex/text>", &format!("\"Ruling text {i}\"")));
3164            v.push(q(
3165                &s,
3166                "<http://ex/about>",
3167                &format!("<http://ex/card/{}>", i % 5),
3168            ));
3169        }
3170        // 5 Cards: fewer instances, and the only things with a name.
3171        for i in 0..5 {
3172            let s = format!("<http://ex/card/{i}>");
3173            v.push(q(&s, TYPE, "<http://ex/Card>"));
3174            v.push(q(&s, name, &format!("\"Card {i}\"")));
3175            v.push(q(&s, "<http://ex/legalIn>", "<http://ex/format/standard>"));
3176        }
3177        v
3178    }
3179
3180    #[test]
3181    fn top_class_without_the_label_predicate_still_answers() {
3182        let (card, rete) = card_and_graph(mtg_shaped_quads(), 60, 0);
3183
3184        // The precondition of the bug, asserted so the fixture cannot drift
3185        // into the easy case: the top class is the one WITHOUT the label.
3186        assert_eq!(card.classes[0].0, "<http://ex/Ruling>");
3187        assert_eq!(
3188            card.signals.label_predicate.as_deref(),
3189            Some("<http://schema.org/name>")
3190        );
3191        assert!(
3192            !card
3193                .class_links
3194                .iter()
3195                .any(|l| l.s_class == "<http://ex/Ruling>"
3196                    && l.predicate == "<http://schema.org/name>"),
3197            "fixture precondition: the top class carries no label"
3198        );
3199
3200        // The label query is still emitted — and it names the class that
3201        // actually has labels, not the biggest one.
3202        let lb = sparql_of(&card, "lb-labels").expect("lb-labels emitted");
3203        assert!(
3204            lb.contains("?s a <http://ex/Card> ; <http://schema.org/name> ?label"),
3205            "lb-labels must join the LABELLED class:\n{lb}"
3206        );
3207        assert!(
3208            !lb.contains("Ruling"),
3209            "the unlabelled top class leaked in:\n{lb}"
3210        );
3211
3212        // Same class for the coverage query, so the two label queries are not
3213        // measuring different things (with TOP_CLASS this reported total/0).
3214        let cov = sparql_of(&card, "cmp-coverage").expect("cmp-coverage emitted");
3215        assert!(
3216            cov.contains("<http://ex/Card>"),
3217            "cmp-coverage on the wrong class"
3218        );
3219
3220        // And the whole library answers.
3221        assert_every_query_returns_rows(&card, &rete);
3222    }
3223
3224    /// The **hugging-face** shape: a label predicate exists, but no class the
3225    /// card can see carries it (there, `rdfs:label` sits only on the embedded
3226    /// ontology terms, whose `class_links` rows fall outside the top 100).
3227    /// The label query must degrade to the class-free form, not disappear.
3228    #[test]
3229    fn labels_off_every_known_class_fall_back_to_the_class_free_query() {
3230        let label = "<http://www.w3.org/2000/01/rdf-schema#label>";
3231        let mut quads = Vec::new();
3232        for i in 0..8 {
3233            let s = format!("<http://ex/model/{i}>");
3234            quads.push(q(&s, TYPE, "<http://ex/Model>"));
3235            quads.push(q(&s, "<http://ex/downloads>", &format!("\"{i}\"")));
3236        }
3237        // The only labels in the graph hang off UNTYPED subjects, so no
3238        // `class_links` row can witness a class that carries the predicate.
3239        for i in 0..3 {
3240            quads.push(q(
3241                &format!("<http://ex/term/{i}>"),
3242                label,
3243                &format!("\"Term {i}\"@en"),
3244            ));
3245        }
3246        let (card, rete) = card_and_graph(quads, 30, 0);
3247
3248        assert_eq!(card.signals.label_predicate.as_deref(), Some(label));
3249        let lb = sparql_of(&card, "lb-labels").expect("lb-labels still emitted");
3250        assert!(
3251            lb.ends_with("SELECT ?s ?label WHERE { ?s <http://www.w3.org/2000/01/rdf-schema#label> ?label } LIMIT 50"),
3252            "expected the class-free fallback:\n{lb}"
3253        );
3254        // The fallback advertises the reduced gate it was emitted under.
3255        let req = &card
3256            .queries
3257            .iter()
3258            .find(|x| x.id == "lb-labels")
3259            .unwrap()
3260            .requires;
3261        assert_eq!(req, &["LABEL_PRED".to_string()]);
3262        // Coverage needs a labelled class and honestly has none, so it is gone
3263        // rather than reporting a percentage of the wrong class.
3264        assert!(sparql_of(&card, "cmp-coverage").is_none());
3265        assert_every_query_returns_rows(&card, &rete);
3266    }
3267
3268    /// The **geoadmin** shape: `geo:asWKT` hangs straight off each feature and
3269    /// there is no `geo:hasGeometry` anywhere, so the fixed two-step path the
3270    /// template used to write could not match a single one of its 52,959
3271    /// geometries.
3272    #[test]
3273    fn geometry_hung_directly_off_the_feature_is_reachable() {
3274        let aswkt = "<http://www.opengis.net/ont/geosparql#asWKT>";
3275        let wkt = "http://www.opengis.net/ont/geosparql#wktLiteral";
3276        let mut quads = Vec::new();
3277        for i in 0..5 {
3278            let s = format!("<http://ex/district/{i}>");
3279            quads.push(q(&s, TYPE, "<http://ex/District>"));
3280            quads.push(q(&s, aswkt, &format!("\"POINT({} 4{i})\"^^<{wkt}>", i + 7)));
3281        }
3282        let (card, rete) = card_and_graph(quads, 20, 0);
3283
3284        let sp = sparql_of(&card, "sp-within").expect("sp-within emitted");
3285        assert!(
3286            sp.contains("?s geo:asWKT ?w"),
3287            "expected the direct geometry path:\n{sp}"
3288        );
3289        assert!(
3290            !sp.contains("hasGeometry"),
3291            "a hasGeometry step the data does not have:\n{sp}"
3292        );
3293        match eval_query(&rete, sp).unwrap() {
3294            QueryOutput::Select(_, rows) => {
3295                assert_eq!(rows.len(), 5, "every feature is in the box")
3296            }
3297            other => panic!("unexpected result form: {other:?}"),
3298        }
3299        assert_every_query_returns_rows(&card, &rete);
3300    }
3301
3302    /// Both geometry layouts in one graph: the optional step binds `?s` to the
3303    /// feature either way, so neither half is hidden.
3304    #[test]
3305    fn both_geometry_layouts_are_covered_by_one_path() {
3306        let (card, rete) = card_and_graph(rich_quads(), 50, 0);
3307        let sp = sparql_of(&card, "sp-within").expect("sp-within emitted");
3308        assert!(
3309            sp.contains("?s geo:hasGeometry?/geo:asWKT ?w"),
3310            "expected the optional-step path:\n{sp}"
3311        );
3312        match eval_query(&rete, sp).unwrap() {
3313            QueryOutput::Select(_, rows) => assert!(!rows.is_empty()),
3314            other => panic!("unexpected result form: {other:?}"),
3315        }
3316    }
3317
3318    /// A graph of pure values — every object a literal, no `rdf:type` — has no
3319    /// group for the in-degree query and no IRI for the dangling-IRI one. The
3320    /// card knows (`in_hubs` is that precompute), so the generator refuses to
3321    /// emit either rather than shipping two queries that answer nothing.
3322    #[test]
3323    fn a_literal_only_graph_gets_no_object_queries() {
3324        let quads: Vec<_> = (0..6)
3325            .map(|i| {
3326                q(
3327                    &format!("<http://ex/row/{i}>"),
3328                    "<http://ex/value>",
3329                    &format!("\"v{i}\""),
3330                )
3331            })
3332            .collect();
3333        let (card, rete) = card_and_graph(quads, 12, 0);
3334
3335        assert!(
3336            card.in_hubs.is_empty(),
3337            "fixture precondition: no IRI objects"
3338        );
3339        let ids: Vec<&str> = card.queries.iter().map(|x| x.id.as_str()).collect();
3340        for absent in ["top-in-hubs", "top-dangling"] {
3341            assert!(!ids.contains(&absent), "{absent} would return zero rows");
3342        }
3343        assert_every_query_returns_rows(&card, &rete);
3344    }
3345
3346    // ---------------------------------------------------------------------
3347    // RDF-star: the `qt-*` family.
3348    // ---------------------------------------------------------------------
3349
3350    const RECORDED_BY: &str = "<http://ex/recordedBy>";
3351    const CONFIDENCE: &str = "<http://ex/confidence>";
3352    const STATES: &str = "<http://ex/states>";
3353    const COUNT_OF: &str = "<http://ex/count>";
3354
3355    /// A provenance graph: every occurrence is typed, most of those type
3356    /// assertions are themselves annotated with who recorded them, and two
3357    /// claims hold a statement as their *value* — the object position, which
3358    /// is a different capability from the subject one.
3359    ///
3360    /// Deliberately **partial**: `occ4`'s type assertion carries no annotation,
3361    /// so `qt-coverage` reports a real ratio instead of a vacuous 100 %.
3362    fn annotated_quads() -> Vec<(String, String, String, Option<String>)> {
3363        let mut v = Vec::new();
3364        for i in 0..4 {
3365            let s = format!("<http://ex/occ/{i}>");
3366            let species = if i < 2 {
3367                "<http://ex/Swallow>"
3368            } else {
3369                "<http://ex/Robin>"
3370            };
3371            v.push(q(&s, TYPE, species));
3372            v.push(q(&s, COUNT_OF, &format!("\"{}\"", i + 1)));
3373            // The star layer: a statement about the type assertion above.
3374            if i < 3 {
3375                let quoted = format!("<<{s} {TYPE} {species}>>");
3376                v.push(q(
3377                    &quoted,
3378                    RECORDED_BY,
3379                    if i == 0 {
3380                        "<http://ex/jsmith>"
3381                    } else {
3382                        "<http://ex/adoe>"
3383                    },
3384                ));
3385                if i == 0 {
3386                    v.push(q(&quoted, CONFIDENCE, "\"0.9\""));
3387                }
3388            }
3389            // …and the other position: a claim whose VALUE is a statement.
3390            if i < 2 {
3391                v.push(q(
3392                    &format!("<http://ex/claim/{i}>"),
3393                    STATES,
3394                    &format!("<<{s} {COUNT_OF} \"{}\">>", i + 1),
3395                ));
3396            }
3397        }
3398        v
3399    }
3400
3401    /// The family emits, every query answers on the graph it came from, and
3402    /// the coverage aggregate reports the ratio the fixture really has.
3403    #[test]
3404    fn statement_queries_are_generated_and_return_rows() {
3405        let quads = annotated_quads();
3406        let (card, rete) = card_and_graph(quads, 40, 0);
3407
3408        // The derive-time profile, which is what makes any of this possible:
3409        // the header's presence bit names no vocabulary.
3410        let s = &card.signals;
3411        assert_eq!(s.annotation_predicates, vec![RECORDED_BY, CONFIDENCE]);
3412        assert_eq!(s.quoting_predicates, vec![STATES]);
3413        assert_eq!(
3414            s.annotated_statement,
3415            Some((TYPE.to_string(), RECORDED_BY.to_string())),
3416            "the witness must be a statement the graph BOTH asserts and annotates"
3417        );
3418
3419        let ids: Vec<&str> = card.queries.iter().map(|x| x.id.as_str()).collect();
3420        for want in [
3421            "qt-sample",
3422            "qt-about",
3423            "qt-qualifiers",
3424            "qt-coverage",
3425            "qt-quoted-object",
3426        ] {
3427            assert!(ids.contains(&want), "{want} should be emitted: {ids:?}");
3428        }
3429        for eq in card.queries.iter().filter(|x| x.id.starts_with("qt-")) {
3430            assert_eq!(eq.dimension, "statements");
3431            // The query surface is RDF-star (`<< s p o >>`). The ratified
3432            // RDF 1.2 spelling `<<( s p o )>>` is an EXPORT surface; a body in
3433            // it would not parse, which is the worst content a card can carry.
3434            assert!(
3435                !eq.sparql.contains("<<("),
3436                "{}: RDF 1.2 triple-term syntax in a QUERY",
3437                eq.id
3438            );
3439        }
3440
3441        // The whole point: every one of them answers on this graph.
3442        assert_every_query_returns_rows(&card, &rete);
3443
3444        // And the coverage aggregate counts what the fixture actually holds:
3445        // four type assertions, three of them annotated.
3446        let cov = sparql_of(&card, "qt-coverage").unwrap();
3447        assert_eq!(count_value(&rete, cov, "asserted"), 4);
3448        assert_eq!(count_value(&rete, cov, "qualified"), 3);
3449    }
3450
3451    /// The overwhelmingly common case: no quoted triples, so no `qt-*` query,
3452    /// no empty section, and — because all three new signal fields serialize
3453    /// only when non-empty — a card byte-identical to one built before the
3454    /// family existed.
3455    #[test]
3456    fn a_graph_without_quoted_triples_gains_nothing() {
3457        let quads = rich_quads();
3458        let card = derive_card(&quads, 50, 0, CardInput::default());
3459        let s = &card.signals;
3460        assert!(s.annotation_predicates.is_empty());
3461        assert!(s.quoting_predicates.is_empty());
3462        assert!(s.annotated_statement.is_none());
3463        assert!(
3464            !card.queries.iter().any(|x| x.id.starts_with("qt-")),
3465            "a graph with no quoted triple must gain no statement query"
3466        );
3467        // The card's own bytes say nothing about a layer it does not have.
3468        let json = String::from_utf8(card.to_json_bytes()).unwrap();
3469        for key in [
3470            "annotation_predicates",
3471            "quoting_predicates",
3472            "annotated_statement",
3473        ] {
3474            assert!(!json.contains(key), "{key} reached a card that has none");
3475        }
3476    }
3477
3478    /// Presence is not position. A graph that only ever holds a statement as a
3479    /// *value* has nothing for a subject-position body to match — the header's
3480    /// `FLAG_HAS_QUOTED_TRIPLES` cannot tell the two apart, and a query written
3481    /// for the wrong one is exactly the zero-rows failure this library exists
3482    /// to prevent.
3483    #[test]
3484    fn a_statement_held_only_as_a_value_gets_only_the_object_position_query() {
3485        let quads = vec![
3486            q("<http://ex/a>", TYPE, "<http://ex/T>"),
3487            q("<http://ex/a>", "<http://ex/p>", "<http://ex/b>"),
3488            q(
3489                "<http://ex/claim>",
3490                STATES,
3491                "<<<http://ex/a> <http://ex/p> <http://ex/b>>>",
3492            ),
3493        ];
3494        let (card, rete) = card_and_graph(quads, 12, 0);
3495        assert!(card.signals.annotation_predicates.is_empty());
3496        assert_eq!(card.signals.quoting_predicates, vec![STATES]);
3497
3498        let ids: Vec<&str> = card.queries.iter().map(|x| x.id.as_str()).collect();
3499        assert!(ids.contains(&"qt-quoted-object"));
3500        for absent in ["qt-sample", "qt-about", "qt-qualifiers", "qt-coverage"] {
3501            assert!(
3502                !ids.contains(&absent),
3503                "{absent} has no subject-position statement to match"
3504            );
3505        }
3506        assert_every_query_returns_rows(&card, &rete);
3507    }
3508
3509    /// A statement this graph annotates but never asserts cannot witness the
3510    /// coverage query: `?s p ?o` would match nothing and the aggregate would
3511    /// report `0 / 0`, which is the `mtg` "76990 / 0" failure one level up.
3512    /// The other four still emit — the quoted layer is real, it is only the
3513    /// join to the asserted one that is missing.
3514    #[test]
3515    fn an_annotation_of_an_unasserted_statement_witnesses_no_coverage() {
3516        let quads = vec![
3517            q("<http://ex/a>", TYPE, "<http://ex/T>"),
3518            q(
3519                "<<<http://ex/a> <http://ex/p> <http://ex/b>>>",
3520                RECORDED_BY,
3521                "<http://ex/jsmith>",
3522            ),
3523        ];
3524        let (card, rete) = card_and_graph(quads, 10, 0);
3525        assert_eq!(card.signals.annotation_predicates, vec![RECORDED_BY]);
3526        assert!(
3527            card.signals.annotated_statement.is_none(),
3528            "nothing asserts <http://ex/a> <http://ex/p> <http://ex/b>"
3529        );
3530        let ids: Vec<&str> = card.queries.iter().map(|x| x.id.as_str()).collect();
3531        assert!(
3532            !ids.contains(&"qt-coverage"),
3533            "no witness, no coverage query"
3534        );
3535        for want in ["qt-sample", "qt-about", "qt-qualifiers"] {
3536            assert!(ids.contains(&want), "{want} needs only the star layer");
3537        }
3538        assert_every_query_returns_rows(&card, &rete);
3539    }
3540
3541    /// The audit reaches the RDF-star conjunction through the card's witness
3542    /// rather than through step 7, which would see two ranked predicates and
3543    /// decline — and it stops vouching for a published body whose pair the
3544    /// card no longer witnesses.
3545    #[test]
3546    fn the_audit_decides_the_star_conjunction_from_the_witness() {
3547        let mut card = derive_card(&annotated_quads(), 40, 0, CardInput::default());
3548        let findings = audit(&card);
3549        assert_eq!(verdict_of(&findings, "qt-coverage"), Verdict::Answers);
3550
3551        // The same body on a card whose witness has moved: the pair is no
3552        // longer proven, and the audit says so instead of vouching for it.
3553        card.signals.annotated_statement = Some((COUNT_OF.to_string(), CONFIDENCE.to_string()));
3554        let findings = audit(&card);
3555        assert_eq!(verdict_of(&findings, "qt-coverage"), Verdict::Suspect);
3556    }
3557
3558    /// The table-level audit: nothing in the library may conjoin two
3559    /// independently-chosen substitutions, and every placeholder must be
3560    /// declared. This is the check that makes the bug unrepresentable rather
3561    /// than merely fixed — it runs over every body of every template, in every
3562    /// graph scope, with no dataset involved.
3563    #[test]
3564    fn every_template_body_is_witnessed() {
3565        let mut problems = Vec::new();
3566        for t in TEMPLATES {
3567            for body in [Some(t.body), t.named_body, t.mixed_body]
3568                .into_iter()
3569                .flatten()
3570            {
3571                if let Err(e) = check_body(t.id, body, t.requires, t.nonempty) {
3572                    problems.push(e);
3573                }
3574            }
3575            if let Some(f) = &t.fallback {
3576                if let Err(e) = check_body(t.id, f.body, f.requires, f.nonempty) {
3577                    problems.push(e);
3578                }
3579                assert!(
3580                    f.requires.len() < t.requires.len(),
3581                    "{}: a fallback must need LESS than the template it stands in for",
3582                    t.id
3583                );
3584            }
3585            if let Some(b) = &t.bounded {
3586                for body in [Some(b.body), b.named_body].into_iter().flatten() {
3587                    if let Err(e) = check_body(t.id, body, b.requires, b.nonempty) {
3588                        problems.push(e);
3589                    }
3590                }
3591            }
3592        }
3593        assert!(problems.is_empty(), "{}", problems.join("\n"));
3594    }
3595
3596    // ---------------------------------------------------------------------
3597    // B4: the cost gate. Every emitted starter is bounded — by the budget
3598    // when the card says the exact body is cheap, by its bounded form when
3599    // not, and by not being emitted when it has no honest bounded form.
3600    // ---------------------------------------------------------------------
3601
3602    /// The ids whose exact question the card's own profile already answers,
3603    /// and which therefore ship nothing at all past the budget.
3604    const PROFILE_ANSWERS: &[&str] = &[
3605        "ov-triples",
3606        "ov-pred-list",
3607        "ov-pred-hist",
3608        "ty-class-hist",
3609        "ty-entities",
3610        "top-out-hubs",
3611        "top-in-hubs",
3612        "lt-datatypes",
3613        "lt-langs",
3614        "lt-numrange",
3615        "ti-extent",
3616        "ti-histogram",
3617        "sp-bbox",
3618        "ng-sizes",
3619        "qt-about",
3620        "qt-qualifiers",
3621        "top-class-graph",
3622    ];
3623
3624    /// Generate under a budget of zero — every template whose body reads
3625    /// anything takes its bounded form or is dropped — and run every emitted
3626    /// query against the graph it was generated for.
3627    fn assert_bounded_library(quads: Vec<(String, String, String, Option<String>)>, ng: u64) {
3628        let card = derive_card(&quads, 60, ng, CardInput::default());
3629        let (bytes, _) =
3630            ingest::assemble_dataset_with_opts(quads, true, false, None, |_, _| Vec::new());
3631        let rete = Rete::open(&bytes).unwrap();
3632        let bounded = generate_with_budget(&card, 0);
3633        assert!(!bounded.is_empty(), "nothing survived a zero budget");
3634        for eq in &bounded {
3635            let t = TEMPLATES.iter().find(|t| t.id == eq.id).unwrap();
3636            // Either self-bounded, or the bounded form — never the exact body
3637            // of a template that reads something.
3638            if t.scan != Scan::Bounded {
3639                let b = t
3640                    .bounded
3641                    .as_ref()
3642                    .unwrap_or_else(|| panic!("{}: emitted past the budget", eq.id));
3643                assert!(
3644                    eq.sparql.ends_with(b.body)
3645                        || b.named_body.is_some_and(|nb| eq.sparql.ends_with(nb))
3646                        || {
3647                            // Substituted bodies cannot be compared verbatim; the
3648                            // variant matcher is the arbiter.
3649                            variant_of(t, strip_prefixes(&eq.sparql).trim())
3650                                .is_some_and(|v| v.bounded)
3651                        },
3652                    "{}: not the bounded form:\n{}",
3653                    eq.id,
3654                    eq.sparql
3655                );
3656                assert_eq!(
3657                    eq.title, b.title,
3658                    "{}: the title must say it is a sample",
3659                    eq.id
3660                );
3661            }
3662            assert!(
3663                !PROFILE_ANSWERS.contains(&eq.id.as_str()),
3664                "{}: a profile-answered question shipped past the budget",
3665                eq.id
3666            );
3667            let undecidable = claim_of_query(&eq.id, &eq.sparql) == Claim::Undecidable;
3668            match eval_query(&rete, &eq.sparql) {
3669                Ok(QueryOutput::Select(_, rows)) => assert!(
3670                    undecidable || !rows.is_empty(),
3671                    "{}: bounded form returned zero rows:\n{}",
3672                    eq.id,
3673                    eq.sparql
3674                ),
3675                Ok(QueryOutput::Ask(b)) => assert!(undecidable || b, "{}: ASK false", eq.id),
3676                Ok(QueryOutput::Construct(ts)) => assert!(undecidable || !ts.is_empty()),
3677                Err(e) => panic!("{}: failed to run: {e:?}\n{}", eq.id, eq.sparql),
3678            }
3679        }
3680        // And the audit reads every bounded body as a known form of its
3681        // template — never "unrecognized".
3682        let mut shipped = card.clone();
3683        shipped.queries = bounded;
3684        for f in audit(&shipped) {
3685            assert_ne!(f.verdict, Verdict::Unrecognized, "{}: {}", f.id, f.why);
3686        }
3687    }
3688
3689    #[test]
3690    fn bounded_forms_parse_run_and_answer_in_every_scope() {
3691        assert_bounded_library(rich_quads(), 0);
3692        assert_bounded_library(named_only_quads(), 3);
3693        assert_bounded_library(mtg_shaped_quads(), 0);
3694        assert_bounded_library(annotated_quads(), 0);
3695        let mut mixed = rich_quads();
3696        for i in 0..5 {
3697            mixed.push((
3698                format!("<http://ex/note/{i}>"),
3699                "<http://ex/about>".to_string(),
3700                format!("<http://ex/person/{}>", i % 6),
3701                Some("<http://ex/graph/notes>".to_string()),
3702            ));
3703        }
3704        assert_bounded_library(mixed, 1);
3705    }
3706
3707    /// The gate reads the card's counts, so a graph the size of an
3708    /// OpenCitations part gets bounded bodies whatever its fixture size. Here
3709    /// the rich fixture's card is inflated to 2.9 B statements: every emitted
3710    /// body must be bounded, the profile-answered questions must be gone, and
3711    /// the bounded forms must be the ones that ship.
3712    #[test]
3713    fn a_billion_statement_card_ships_only_bounded_starters() {
3714        let mut card = derive_card(&rich_quads(), 50, 0, CardInput::default());
3715        let scale = 50_000_000u64;
3716        card.triple_count *= scale;
3717        card.quad_count = card.triple_count;
3718        for (_, n) in card.predicates.iter_mut().chain(card.classes.iter_mut()) {
3719            *n *= scale;
3720        }
3721        for l in &mut card.class_links {
3722            l.count *= scale;
3723        }
3724        let queries = generate(&card);
3725        let ids: Vec<&str> = queries.iter().map(|q| q.id.as_str()).collect();
3726        for gone in PROFILE_ANSWERS {
3727            assert!(!ids.contains(gone), "{gone} shipped on a 2.9 B card");
3728        }
3729        for kept in [
3730            "ov-one-row",
3731            "ov-ask-pred",
3732            "id-sample",
3733            "lb-labels",
3734            "top-reach",
3735            "ty-class-shape",
3736            "top-dangling",
3737            "cmp-coverage",
3738            "lk-sameas",
3739            "lk-external",
3740            "sp-within",
3741        ] {
3742            assert!(
3743                ids.contains(&kept),
3744                "{kept} missing from the bounded set {ids:?}"
3745            );
3746        }
3747        let shape = queries.iter().find(|q| q.id == "ty-class-shape").unwrap();
3748        assert!(shape.sparql.contains("LIMIT 1000"), "{}", shape.sparql);
3749        assert!(shape.title.contains("1,000"), "{}", shape.title);
3750        // Within the budget the same card writes the exact bodies.
3751        let small = derive_card(&rich_quads(), 50, 0, CardInput::default());
3752        let exact = generate(&small);
3753        let shape = exact.iter().find(|q| q.id == "ty-class-shape").unwrap();
3754        assert!(!shape.sparql.contains("LIMIT 1000"), "{}", shape.sparql);
3755        // The audit agrees with the generator about the inflated card: what it
3756        // ships is current, and nothing is refuted.
3757        card.queries = queries;
3758        for f in audit(&card) {
3759            assert_eq!(f.revision, "current", "{}", f.id);
3760            assert!(
3761                matches!(f.verdict, Verdict::Answers | Verdict::Undecidable),
3762                "{}: {} — {}",
3763                f.id,
3764                f.verdict.as_str(),
3765                f.why
3766            );
3767        }
3768    }
3769
3770    /// The bounded form's own claim is what `claim_of_query` reports — a
3771    /// sample of a graph may honestly hold no dangling IRI even where the
3772    /// whole graph does, and the measured-zero drop must not call that a
3773    /// generator defect.
3774    #[test]
3775    fn the_claim_follows_the_form_that_shipped() {
3776        let card = derive_card(&rich_quads(), 50, 0, CardInput::default());
3777        let exact = generate(&card);
3778        let shape = exact.iter().find(|q| q.id == "ty-class-shape").unwrap();
3779        assert_eq!(
3780            claim_of_query(&shape.id, &shape.sparql),
3781            Claim::CannotBeEmpty
3782        );
3783        let bounded = generate_with_budget(&card, 0);
3784        let external = bounded.iter().find(|q| q.id == "lk-external").unwrap();
3785        assert_eq!(claim_of("lk-external"), Claim::CannotBeEmpty);
3786        assert_eq!(
3787            claim_of_query(&external.id, &external.sparql),
3788            Claim::Undecidable,
3789            "the 1,000-statement sample may hold no external IRI"
3790        );
3791        // An unknown body falls back to the template's own claim.
3792        assert_eq!(
3793            claim_of_query("ty-class-shape", "SELECT ?x WHERE { ?x ?y ?z }"),
3794            Claim::CannotBeEmpty
3795        );
3796        assert_eq!(claim_of_query("not-a-template", "ASK {}"), Claim::Unknown);
3797    }
3798
3799    /// The new `top-dangling` body returns the rows the old one did, and the
3800    /// old one — on published cards — is recognized and reported superseded.
3801    #[test]
3802    fn the_dangling_query_is_rewritten_without_changing_its_rows() {
3803        let (mut card, rete) = card_and_graph(named_only_quads_as_default(), 30, 0);
3804        let new = sparql_of(&card, "top-dangling").unwrap().to_string();
3805        let old = "SELECT ?o WHERE { ?s ?p ?o FILTER(isIRI(?o)) FILTER NOT EXISTS { ?o ?p2 ?o2 } } LIMIT 100";
3806        let rows = |q: &str| -> std::collections::BTreeSet<String> {
3807            match eval_query(&rete, q).unwrap() {
3808                QueryOutput::Select(_, rows) => rows
3809                    .iter()
3810                    .map(|r| r.get("o").cloned().unwrap_or_default())
3811                    .collect(),
3812                other => panic!("{other:?}"),
3813            }
3814        };
3815        assert_eq!(rows(&new), rows(&format!("{PREFIXES}\n{old}")));
3816        assert!(!rows(&new).is_empty());
3817        ship_instead(&mut card, "top-dangling", old);
3818        let f = audit(&card);
3819        let d = f.iter().find(|f| f.id == "top-dangling").unwrap();
3820        assert_eq!(d.revision, "superseded");
3821        assert_eq!(d.verdict, Verdict::Undecidable);
3822    }
3823
3824    /// `named_only_quads`, moved into the default graph: a graph with one IRI
3825    /// every item references and nobody describes.
3826    fn named_only_quads_as_default() -> Vec<(String, String, String, Option<String>)> {
3827        named_only_quads()
3828            .into_iter()
3829            .map(|(s, p, o, _)| (s, p, o, None))
3830            .collect()
3831    }
3832
3833    // ---------------------------------------------------------------------
3834    // D1: the curated `example_queries` are audited too, apart.
3835    // ---------------------------------------------------------------------
3836
3837    #[test]
3838    fn curated_example_queries_are_audited_separately() {
3839        let mut card = derive_card(&rich_quads(), 50, 0, CardInput::default());
3840        card.example_queries = vec![
3841            // Parses, names only predicates the card knows: nothing to decide.
3842            "SELECT ?s WHERE { ?s <http://ex/knows> ?o } LIMIT 5".to_string(),
3843            // Does not parse.
3844            "SELECT ?s WHERE { ?s <http://ex/knows> ".to_string(),
3845            // A predicate the card's complete list does not contain.
3846            "SELECT ?s WHERE { ?s <http://ex/neverUsed> ?o }".to_string(),
3847        ];
3848        let findings = audit_examples(&card);
3849        assert_eq!(findings.len(), 3);
3850        assert_eq!(findings[0].id, "example-1");
3851        assert_eq!(findings[0].verdict, Verdict::Undecidable);
3852        assert_eq!(findings[0].revision, "curated");
3853        assert_eq!(findings[1].verdict, Verdict::Invalid);
3854        assert!(
3855            findings[1].why.contains("does not parse"),
3856            "{}",
3857            findings[1].why
3858        );
3859        assert_eq!(findings[2].verdict, Verdict::Suspect);
3860        assert!(findings[2].why.contains("neverUsed"), "{}", findings[2].why);
3861        // The generated half is untouched by the curated half.
3862        assert_eq!(audit(&card).len(), card.queries.len());
3863
3864        // A truncated predicate list proves nothing absent: undecidable.
3865        card.predicates = (0..CARD_TOP_N)
3866            .map(|i| (format!("<http://ex/p{i}>"), 1))
3867            .collect();
3868        let f = audit_examples(&card);
3869        assert_eq!(f[2].verdict, Verdict::Undecidable, "{}", f[2].why);
3870    }
3871
3872    #[test]
3873    fn a_curated_default_graph_query_on_a_named_only_file_is_empty() {
3874        let mut card = derive_card(&named_only_quads(), 20, 3, CardInput::default());
3875        card.example_queries = vec![
3876            "SELECT ?s WHERE { ?s ?p ?o } LIMIT 1".to_string(),
3877            "SELECT ?s WHERE { GRAPH ?g { ?s ?p ?o } } LIMIT 1".to_string(),
3878        ];
3879        let f = audit_examples(&card);
3880        assert_eq!(f[0].verdict, Verdict::Empty, "{}", f[0].why);
3881        assert_ne!(f[1].verdict, Verdict::Empty, "{}", f[1].why);
3882    }
3883
3884    // ---------------------------------------------------------------------
3885    // The audit: reading a card that is already published.
3886    // ---------------------------------------------------------------------
3887
3888    /// Replace one starter query's body with the one an older revision wrote —
3889    /// how a fixture stands in for a file that is already on the wire.
3890    fn ship_instead(card: &mut DatasetCard, id: &str, body: &str) {
3891        let q = card
3892            .queries
3893            .iter_mut()
3894            .find(|q| q.id == id)
3895            .unwrap_or_else(|| panic!("{id} is not in this card"));
3896        q.sparql = format!("{PREFIXES}\n{body}");
3897    }
3898
3899    fn verdict_of(findings: &[Finding], id: &str) -> Verdict {
3900        findings
3901            .iter()
3902            .find(|f| f.id == id)
3903            .unwrap_or_else(|| panic!("{id} was not audited"))
3904            .verdict
3905    }
3906
3907    fn rows_of(rete: &Rete, sparql: &str) -> usize {
3908        match eval_query(rete, sparql).unwrap() {
3909            QueryOutput::Select(_, rows) => rows.len(),
3910            QueryOutput::Ask(b) => usize::from(b),
3911            other => panic!("unexpected result form: {other:?}"),
3912        }
3913    }
3914
3915    /// A card this generator just wrote must audit clean: nothing refuted,
3916    /// nothing suspected, nothing unrecognized, and every body current. If the
3917    /// audit and the generator ever disagree about a *fresh* card, one of them
3918    /// is wrong — and they are supposed to share the judgement.
3919    #[test]
3920    fn a_freshly_generated_card_audits_clean() {
3921        for (quads, ng) in [(rich_quads(), 0), (named_only_quads(), 3)] {
3922            let card = derive_card(&quads, 50, ng, CardInput::default());
3923            for f in audit(&card) {
3924                assert_eq!(
3925                    f.revision, "current",
3926                    "{}: the generator would not write this body",
3927                    f.id
3928                );
3929                assert!(
3930                    matches!(f.verdict, Verdict::Answers | Verdict::Undecidable),
3931                    "{}: {} — {}",
3932                    f.id,
3933                    f.verdict.as_str(),
3934                    f.why
3935                );
3936                // The only queries a fresh card may leave undecided are the
3937                // ones the table already says it cannot decide, plus `sp-bbox`:
3938                // its un-grouped aggregate always returns its row (so
3939                // `NonEmpty::Aggregate` is true about *rows*), and whether that
3940                // row is bound turns on two predicates meeting on one subject,
3941                // which is the one thing `class_links` cannot witness.
3942                if f.verdict == Verdict::Undecidable {
3943                    assert!(
3944                        may_be_empty(&f.id) || f.id == "sp-bbox",
3945                        "{}: undecidable but the table claims otherwise — {}",
3946                        f.id,
3947                        f.why
3948                    );
3949                }
3950            }
3951        }
3952    }
3953
3954    /// The **mtg** card as published: the label query joins the most populous
3955    /// class to the most-used label predicate. The audit refutes it, and the
3956    /// engine agrees — the verdict is checked against the graph, not just
3957    /// against the card.
3958    #[test]
3959    fn the_audit_refutes_a_label_query_on_an_unlabelled_top_class() {
3960        let quads = mtg_shaped_quads();
3961        let (mut card, rete) = card_and_graph(quads, 60, 0);
3962        let top = card.classes[0].0.clone();
3963        ship_instead(
3964            &mut card,
3965            "lb-labels",
3966            &format!("SELECT ?s ?label WHERE {{ ?s a {top} ; <http://schema.org/name> ?label }} LIMIT 50"),
3967        );
3968        ship_instead(
3969            &mut card,
3970            "cmp-coverage",
3971            &format!(
3972                "SELECT (COUNT(?s) AS ?total) (COUNT(?l) AS ?have) WHERE {{ ?s a {top} \
3973                 OPTIONAL {{ ?s <http://schema.org/name> ?l }} }}"
3974            ),
3975        );
3976        let findings = audit(&card);
3977        assert_eq!(verdict_of(&findings, "lb-labels"), Verdict::Empty);
3978        // The coverage query is an un-grouped aggregate: it returns its row, and
3979        // the row says `have = 0`. A different failure, reported as one.
3980        assert_eq!(verdict_of(&findings, "cmp-coverage"), Verdict::Vacuous);
3981        assert_eq!(rows_of(&rete, sparql_of(&card, "lb-labels").unwrap()), 0);
3982        assert_eq!(rows_of(&rete, sparql_of(&card, "cmp-coverage").unwrap()), 1);
3983    }
3984
3985    /// The **vidy** shape, and the one false positive this audit was caught
3986    /// making: every unit is typed *twice* — `schema:ArchiveComponent` and a
3987    /// native class — so the `class_links` quotient, which files each subject
3988    /// under its last type only, shows no label row for the top class even
3989    /// though the label query returns rows.
3990    ///
3991    /// The audit must NOT claim a refutation here. Both tells are present (the
3992    /// top class has no row of its own; another class has exactly its instance
3993    /// count), and a survey that called this "provably empty" would have sent
3994    /// someone to re-card a file that was fine.
3995    #[test]
3996    fn the_audit_will_not_refute_a_dually_typed_class() {
3997        let name = "<http://schema.org/name>";
3998        let component = "<http://ex/ArchiveComponent>";
3999        let unit = "<http://ex/Unit>";
4000        let mut quads = Vec::new();
4001        for i in 0..8 {
4002            let s = format!("<http://ex/unit/{i}>");
4003            // Both types, the native one last — so the quotient files every
4004            // statement under `Unit` and says nothing about `ArchiveComponent`.
4005            quads.push(q(&s, TYPE, component));
4006            quads.push(q(&s, TYPE, unit));
4007            quads.push(q(&s, name, &format!("\"Unit {i}\"")));
4008        }
4009        let (mut card, rete) = card_and_graph(quads, 30, 0);
4010        assert_eq!(
4011            card.classes[0].0, component,
4012            "fixture: the twin ranks first"
4013        );
4014        assert!(
4015            !card.class_links.iter().any(|l| l.s_class == component),
4016            "fixture: the quotient never mentions the top class"
4017        );
4018        ship_instead(
4019            &mut card,
4020            "lb-labels",
4021            &format!("SELECT ?s ?label WHERE {{ ?s a {component} ; {name} ?label }} LIMIT 50"),
4022        );
4023        assert_eq!(verdict_of(&audit(&card), "lb-labels"), Verdict::Suspect);
4024        assert_eq!(rows_of(&rete, sparql_of(&card, "lb-labels").unwrap()), 8);
4025    }
4026
4027    /// The **geoadmin** card as published: a fixed two-step geometry path on a
4028    /// file that hangs `geo:asWKT` straight off the feature. Nothing to do with
4029    /// typing — the path names a predicate the card's complete list does not
4030    /// contain, so the refutation is exact.
4031    #[test]
4032    fn the_audit_refutes_a_path_through_a_predicate_that_is_not_there() {
4033        let aswkt = "<http://www.opengis.net/ont/geosparql#asWKT>";
4034        let wkt = "http://www.opengis.net/ont/geosparql#wktLiteral";
4035        let mut quads = Vec::new();
4036        for i in 0..5 {
4037            let s = format!("<http://ex/district/{i}>");
4038            quads.push(q(&s, TYPE, "<http://ex/District>"));
4039            quads.push(q(&s, aswkt, &format!("\"POINT({} 4{i})\"^^<{wkt}>", i + 7)));
4040        }
4041        let (mut card, rete) = card_and_graph(quads, 20, 0);
4042        ship_instead(
4043            &mut card,
4044            "sp-within",
4045            "SELECT ?s WHERE { ?s geo:hasGeometry/geo:asWKT ?w FILTER(geof:sfWithin(?w, \
4046             \"POLYGON((-180 -90, 180 -90, 180 90, -180 90, -180 -90))\"^^geo:wktLiteral)) } LIMIT 100",
4047        );
4048        assert_eq!(verdict_of(&audit(&card), "sp-within"), Verdict::Empty);
4049        assert_eq!(rows_of(&rete, sparql_of(&card, "sp-within").unwrap()), 0);
4050    }
4051
4052    /// A `VALUES` list that does not cover the gate admitting the query: the
4053    /// dataset links only with `skos:closeMatch`, the shipped body names the
4054    /// other three. Refutable exactly — `signals.link_predicates` is computed
4055    /// from the full predicate counts, before any cap.
4056    #[test]
4057    fn the_audit_refutes_a_values_list_that_misses_the_only_link_predicate() {
4058        let close = "<http://www.w3.org/2004/02/skos/core#closeMatch>";
4059        let mut quads = vec![q("<http://ex/a>", TYPE, "<http://ex/T>")];
4060        for i in 0..4 {
4061            quads.push(q(
4062                &format!("<http://ex/a{i}>"),
4063                close,
4064                &format!("<http://other/x{i}>"),
4065            ));
4066        }
4067        let (mut card, rete) = card_and_graph(quads, 20, 0);
4068        assert_eq!(card.signals.link_predicates, vec![close.to_string()]);
4069        ship_instead(
4070            &mut card,
4071            "lk-sameas",
4072            "SELECT ?p (COUNT(*) AS ?n) WHERE { ?s ?p ?o VALUES ?p { owl:sameAs \
4073             skos:exactMatch rdfs:seeAlso } } GROUP BY ?p",
4074        );
4075        assert_eq!(verdict_of(&audit(&card), "lk-sameas"), Verdict::Empty);
4076        assert_eq!(rows_of(&rete, sparql_of(&card, "lk-sameas").unwrap()), 0);
4077    }
4078
4079    /// The pre-#172 `top-reach` — a hub IRI and a top predicate, conjoined —
4080    /// is recognized (so it is not reported as an unreadable body) and honestly
4081    /// declared undecidable: nothing in a card ties a *subject* to a
4082    /// *predicate*. On the published catalog 6 of 26 sampled files answered
4083    /// nothing here, which is exactly what "undecidable" has to mean.
4084    #[test]
4085    fn the_audit_recognizes_the_legacy_reach_query_and_declines_to_decide() {
4086        let (mut card, _) = card_and_graph(rich_quads(), 50, 0);
4087        let hub = card.top_hubs[0].0.clone();
4088        let pred = card.predicates[0].0.clone();
4089        ship_instead(
4090            &mut card,
4091            "top-reach",
4092            &format!("SELECT ?y WHERE {{ {hub} {pred}+ ?y }} LIMIT 100"),
4093        );
4094        let f = audit(&card);
4095        let r = f.iter().find(|f| f.id == "top-reach").unwrap();
4096        assert_eq!(r.verdict, Verdict::Undecidable);
4097        assert_eq!(r.revision, "superseded");
4098        assert!(r.conjoined, "two independently-ranked terms in one body");
4099        assert_eq!(r.substitutions.len(), 2);
4100    }
4101
4102    /// A body from no revision at all is reported as such rather than being
4103    /// force-fitted to the nearest pattern — a wrong binding would produce a
4104    /// confident wrong verdict, which is the failure mode a survey cannot have.
4105    #[test]
4106    fn an_unknown_body_is_not_force_fitted() {
4107        let (mut card, _) = card_and_graph(rich_quads(), 50, 0);
4108        ship_instead(
4109            &mut card,
4110            "lb-labels",
4111            "SELECT ?x WHERE { ?x ?y ?z } LIMIT 3",
4112        );
4113        assert_eq!(
4114            verdict_of(&audit(&card), "lb-labels"),
4115            Verdict::Unrecognized
4116        );
4117    }
4118
4119    /// Ids are unique and every capability the table names is reachable — a
4120    /// `Cap` no template requires is dead weight that will drift out of date.
4121    #[test]
4122    fn the_table_is_well_formed() {
4123        let mut ids: Vec<&str> = TEMPLATES.iter().map(|t| t.id).collect();
4124        let n = ids.len();
4125        ids.sort_unstable();
4126        ids.dedup();
4127        assert_eq!(ids.len(), n, "duplicate template id");
4128        for &cap in ALL_CAPS {
4129            assert!(
4130                TEMPLATES.iter().any(|t| t.requires.contains(&cap)
4131                    || t.fallback
4132                        .as_ref()
4133                        .is_some_and(|f| f.requires.contains(&cap))),
4134                "{} is required by no template",
4135                cap.key()
4136            );
4137        }
4138        for t in TEMPLATES {
4139            assert!(
4140                !t.hook_is_exact || t.provably_empty.is_some(),
4141                "{}: hook_is_exact without a hook to be exact about",
4142                t.id
4143            );
4144        }
4145        // A bounded form must be bounded by its own text: every body reads a
4146        // LIMITed slice or asks a lookup-sized question, and carries the
4147        // placeholders it declares.
4148        for t in TEMPLATES {
4149            if let Some(b) = &t.bounded {
4150                assert!(
4151                    t.scan != Scan::Bounded,
4152                    "{}: a self-bounded template needs no bounded form",
4153                    t.id
4154                );
4155                for body in [Some(b.body), b.named_body].into_iter().flatten() {
4156                    assert!(
4157                        body.contains("LIMIT") || body.contains("FILTER EXISTS"),
4158                        "{}: bounded body without a LIMIT or an existence probe:\n{body}",
4159                        t.id
4160                    );
4161                }
4162            }
4163        }
4164        // Every legacy body names a template that still exists (otherwise the
4165        // audit could never reach it) and is genuinely different from the ones
4166        // that template writes now (otherwise it is dead weight).
4167        for l in LEGACY_BODIES {
4168            let t = TEMPLATES
4169                .iter()
4170                .find(|t| t.id == l.id)
4171                .unwrap_or_else(|| panic!("legacy body for unknown template {}", l.id));
4172            assert!(
4173                ![Some(t.body), t.named_body, t.mixed_body]
4174                    .into_iter()
4175                    .flatten()
4176                    .any(|b| b == l.body),
4177                "{}: legacy body is still a current one",
4178                l.id
4179            );
4180        }
4181    }
4182}