Expand description
Stable Dataset Card API — both halves of a card, so that every writer
(rete build --card-file, the browser builder, any language binding, any
third-party tool) produces the same card from the same graph.
- The curated half ([
crate::card_input]) is the write-time rules a hand-authored card document must satisfy: the reserved top level, thethemeIRI requirement, the boundedextrabag. - The derived half ([
crate::card_derive] + [crate::card_queries]) is the profile computed from the graph’s own statements — predicate/class histograms, vocabularies, datatypes, languages, the class-link quotient, hubs, affordance signals and the tiered starter-query library.card_derive::derive_cardis the entry point for an in-memory quad slice;card_derive::derive_card_encodedfor a streaming build that only has a dictionary + id-triples;card_derive::CardTripleSourcefor anything else.
Derivation is opt-in: it walks the graph a second time, so no build spends that pass unless the caller asks for a card.
Structs§
- Card
Input - The curated subset, as supplied by a
--card-fileJSON document (every field optional). CLI flags override whatever the file provides. The identity/ provenance fields (version, creators, publisher, canonical_url, sparql_endpoint, source_date, derived_from, doi, cite_as, keywords, theme) have no CLI flag — they come from the card file only. - Class
Link - One edge of the effective schema graph: a
predicateconnecting subjects of classs_classto objects of classo_class, with the instancecount. Mirrorscrate::schema_summaryrows (sentinels(literal)/(untyped)). - Coherence
- The build-time OWL RL / RDFS coherence verdict, stamped into the card by
rete build --reason. Deterministic and free of free-text detail (only a sortedby_kindhistogram + short tags), so it folds into the file’s content hash without destabilizing it.scoperecords what was checked andrulesthe ruleset version, socoherent: truecan’t be misread as a guarantee from a different scope or rule set. - Creator
- A dataset creator: a person (or team) with an optional ORCID IRI.
- Dataset
Card - The embeddable dataset card. Curated fields come from
rete buildflags or a--card-fileJSON document; the statistics are derived from the data at build time. Absent optional/empty fields are omitted from the JSON. - Example
Query - One vetted starter query, instantiated with the dataset’s real vocabulary.
- Finding
- One published query’s verdict, with the evidence behind it.
- Label
Index Signal - The file’s label-prefix-search affordance: does it carry a LABEL_INDEX section (kind 8), how many labels, and how large is it?
- Observation
- One starter query, actually run.
- Permutations
Signal - The file’s index permutation set: how many orders of
(s, p, o)it stores, and therefore whether its planner can run a sort-merge join. - Publisher
- The publishing organisation, with an optional ROR IRI.
- Quoted
Triples Signal - Whether the file contains quoted triples — statements standing inside other statements — and, if it does, what a consumer gets when they export it.
- Recorded
- The figures a build wrote into the file, beside the ones just measured.
- Signals
- Detected affordances — the index-free hints a newcomer (or the query-library generator) needs to know what the data supports and how to address it. Every field is optional/empty when the corresponding signal is absent, so geo/time query families are emitted only when the data actually has geometry/time.
- Stream
Report - How many statements a streamed card was derived from, and what the per-subject class map cost — for the caller’s progress report.
- Substitution
- A substituted term and where in the card profile it came from.
- Text
Index Signal - The file’s full-text-search affordance: does it carry a TEXT_INDEX section (kind 6), and what does that index cost?
Enums§
- Claim
- What the template behind an emitted query claims about its emptiness — the
NonEmptydeclaration, read back by id. - Tier
- Which tier of the three-tier exploration model can answer a query cheapest:
Card(precomputed in this metadata section, index-free),Summary(the pyramid superedge totals, index-free), orIndex(needs the triple index). - Verdict
- What a published starter query is worth on the file that carries it.
Constants§
- CARD_
DESCRIPTION_ MAX_ BYTES - Maximum size (UTF-8 bytes) of the curated
description. - CARD_
EXTRA_ MAX_ BYTES - Maximum serialized size (compact JSON bytes) of the whole
extraobject. - CARD_
EXTRA_ MAX_ DEPTH - Maximum container-nesting depth inside an
extravalue (seejson_depth): three levels — a list of objects whose fields may be lists of scalars ([{"query": "…", "words": ["a", "b"]}]), no deeper. - CARD_
EXTRA_ MAX_ KEYS - Maximum number of keys in
extra. A key costs ≥ 8 serialized bytes, so 64 typical entries sit comfortably insideCARD_EXTRA_MAX_BYTES; needing more means the bag is being used as a data store — the graph itself is the place for data. - CARD_
EXTRA_ MAX_ KEY_ BYTES - Maximum bytes per
extrakey. Keys are identifiers, not values. - CARD_
TOP_ N - Cap for every top-N list embedded in the card. The metadata section is
fetched on every overview (it is part of the index-free CARD tier), so an
unbounded
class_links(O(classes × predicates × classes)) or predicate list would bloat that fetch on a large schema (CIDOC-CRM/MMM). Capping keeps the card small and bounded;truncatedflags when a list was actually cut. - CURATED_
CARD_ FIELDS - Every field a card file may carry at its top level, in the order they are
declared on the CLI’s
CardInput— which is also the order serde names them in its “expected one of” message, sovalidate_curated_cardcan reproduce that message verbatim. - GEO_
ASWKT - GEO_
HASGEOMETRY - O_
LITERAL - The
ClassLink::o_classsentinel for a literal object (the object of that row is a literal, so it has no class and no outgoing edges).pub: the query generator reads it to tell an entity-to-entity relation — one a path query can walk — from a literal-valued one. - QUOTED_
TRIPLE_ SURFACE_ RDF12 - The
--quoted-triple-syntaxvalue that writes the RDF 1.2 triple term<<( s p o )>>, andrete export’s default. - QUOTED_
TRIPLE_ SURFACE_ RDF_ STAR - The
--quoted-triple-syntaxvalue that writes the RDF-star surface<<s p o>>, which is also the token rete stores. - STARTER_
MAX_ BYTES - The most bytes a starter query may read when the build (or
rete recard) measures it, counted once the file is open — the measured backstop toSTARTER_SCAN_BUDGET’s estimate. A query that asks for more is dropped from the card and the drop is recorded (“did not finish”). The open is not charged: it reads the section directories (~44 MB on a 26 GB file) whatever the query, a cost no starter can change. 64 MiB is ~3× the heaviest exact body on a budget-sized graph and ~40× the heaviest bounded one. - STARTER_
SCAN_ BUDGET - How many statements a starter query’s body may have to read before the
generator swaps it for its
Boundedform (or drops it). - UNKNOWN_
FIELD_ HINT - The pointer appended to an unknown-top-level-key error. A stray key is usually a typo — or a custom field that belongs in the bag — and saying so costs one line and saves a round trip to the docs.
Traits§
- Card
Triple Source - A replayable source of a graph’s default-graph triples for card
derivation.
replaymay be called more than once (the derivation makes two passes); every call MUST yield the same triples in the same order. Terms are passed as borrowed&strvalid only for that call — so the derivation owns (clones) any key it retains, letting one code path serve both an in-memory quad slice and a streaming build’s dictionary + id-triples.
Functions§
- audit
- Audit every starter query a card already ships.
- audit_
examples - Audit the curated
example_queriesa card carries — the half of a card a publisher wrote, whichaudit(the generated half) never looks at. - canonicalize_
json - Rebuild every nested object with its keys inserted in sorted order, so the
bag’s bytes — which fold into the reproducible content hash — never depend
on author key order. With today’s serde_json (no
preserve_order) maps are BTree-backed and this is a no-op in effect; it is insurance that a future feature unification can’t quietly make map order mean insertion order. - check_
description_ len - The size half of
normalize_description, split out so the CLI’s serde-typed path — where--descriptionmay override whatever the file said — can apply it to the final text without re-parsing anything. - claim_
of - The
Claimbehind a shipped query id. - claim_
of_ query - The
Claimbehind a shipped query, read from its body as well as its id — the form to use whenever the SPARQL is at hand, because the exact and the bounded form of one template claim different things. - compose_
curated_ card - Finish a validated curated document into a complete card, by adding the four counts the build just measured and the format version the writer stamps.
- curated_
counts_ card - Derive a full card from the parsed quads plus a few build-time counts — the
in-memory build path. Statistics are over the default graph only
(named-graph statistics are summarized by
quad_count/named_graph_count), matchingrete stats/rete predicates. A card holding curated fields + top-line counts only — for the external (memory-bounded) build, where deriving the profile lists (predicates/classes/ hubs/links) would need unbounded RAM. Every derived list is left empty andtruncatedis unset (the lists are absent, not cut). - derive_
card - derive_
card_ encoded - Derive a card from a built dictionary + default-graph id-triples — the
streaming (low-RAM) build path, where the raw quads were never retained.
Resolves each id-triple back to its terms through the dictionary, yielding the
same
(s, p, o)strings in file order, so the card is byte-identical to the in-memory derivation on the same graph. - derive_
card_ from - The shared card derivation, over any
CardTripleSource. Two passes: the first builds the subject→class map needed to classify both endpoints of every relation (theclass_linksquotient — the same logic ascrate::schema_summary, folded in here to avoid a second full materialization); the second tallies everything else. All counts are over the raw (pre-dedup) default-graph multiset, matching the existing card stats andrete progressive. - derive_
card_ streaming - Derive a full card — profile, signals and starter queries — from a
.retethat already exists, streaming its sections in sorted order with memory that does not grow with the statement count (see the section notes above). - example_
query_ id - The id
rete card-auditgives the curatedexample_queriesentry atindex(0-based):example-1,example-2, … — curated queries carry no id of their own, and a stable positional one is what--onlycan name. - generate
- Generate the tiered starter-query library for a card: emit each template whose
required capabilities are all present, with the body picked for where the data
lives (
GraphScope), placeholders substituted, and the shared PREFIX block prepended. A body whoseScanis overSTARTER_SCAN_BUDGETstatements is replaced by its bounded form, or dropped when it has none. - json_
depth - Container-nesting depth of a JSON value: scalars are 0, an array/object is 1 + its deepest child. (Recursion here is bounded by serde_json’s own 128-level parse limit.)
- load_
card - Read the dataset card embedded in a
.retefile image, orNoneif it has no metadata section. Parses only the header and the metadata range — it never decodes the dictionary, index or pyramid, so a client that just built a file can read back the card it wrote for the price of a slice. - normalize_
description - Accept a
descriptionwritten either way, and bound it — the write-time gate every card-writing path funnels through. - normalize_
extra - Canonicalize and bounds-check the
extrabag — the write-time gate. - normalize_
string_ list - Canonicalize a curated string list (
keywords,theme) — the write-time gate every card-writing command funnels through (likenormalize_extra). - normalize_
themes normalize_string_listplus thethemerequirement: every entry must be an IRI into a controlled vocabulary. This is what keepsthemefrom becoming a second keywords field — a free-text theme carries no more meaning than a keyword, andkeywordsalready holds those; the agreed concept scheme behind the IRI is the whole valuedcat:themeadds.- validate_
curated_ card - Validate and canonicalize a whole curated card document — the JSON a
publisher writes for
--card-file, or types into the playground’s builder.