Struct LoadableDatasetId 

pub struct LoadableDatasetId(/* private fields */);

Dataset identifier guaranteed to be loadable by DatasetLoader.

This is a thin wrapper over DatasetId that enforces loader support via TryFrom<DatasetId> / FromStr.

Implementations

impl LoadableDatasetId

pub fn all() -> Vec<Self>

All datasets that have loading implementations.

pub fn into_inner(self) -> DatasetId

Access the underlying catalog ID.

Methods from Deref<Target = DatasetId>

pub fn download_url(&self) -> &'static str

Get the download URL for this dataset.

pub fn name(&self) -> &'static str

Get the human-readable name.

pub fn description(&self) -> &'static str

Get the description.

pub fn language(&self) -> &'static str

Get the primary language.

pub fn domain(&self) -> &'static str

Get the domain category.

pub fn categories(&self) -> &'static [&'static str]

Get the category tags for this dataset.

Categories are used for discovery/filtering (domain, language family, annotation properties). They are not intended to encode benchmark outcomes.

pub fn entity_types(&self) -> &'static [&'static str]

Get the entity types for this dataset.

pub fn license(&self) -> Option<&'static str>

Get the license for this dataset.

pub fn citation(&self) -> Option<&'static str>

Get the citation for this dataset.

pub fn paper_url(&self) -> Option<&'static str>

Get the paper URL for this dataset.

pub fn notes(&self) -> Option<&'static str>

Get additional notes for this dataset.

pub fn year(&self) -> Option<u16>

Get the publication year for this dataset.

pub fn format(&self) -> Option<&'static str>

Get the data format for this dataset.

pub fn annotation_scheme(&self) -> Option<&'static str>

Get the annotation scheme for this dataset.

pub fn size_hint(&self) -> Option<&'static str>

Get the size hint for this dataset.

pub fn example(&self) -> Option<&'static str>

Get an example snippet from this dataset.

pub fn splits(&self) -> &'static [&'static str]

Get available data splits (train, dev, test, etc.).

pub fn tasks(&self) -> &'static [&'static str]

Get supported NLP tasks (ner, coref, re, el, etc.). Datasets often support multiple tasks via their annotations.

pub fn tasks_typed(&self) -> Vec<Task>

Get supported tasks as typed Task values (best-effort).

This keeps the registry as the single source of truth (strings), while giving downstream code a type-safe view for control flow and compatibility checks.

Unknown/out-of-scope task strings are ignored.

pub fn expected_docs(&self) -> Option<u32>

Get expected document count for validation.

pub fn sha256(&self) -> Option<&'static str>

Get SHA256 hash for integrity verification.

pub fn hf_id(&self) -> Option<&'static str>

Get HuggingFace dataset ID for automated download.

pub fn hf_config(&self) -> Option<&'static str>

Get HuggingFace dataset config/subset name.

pub fn access_status(&self) -> DatasetAccessibility

Get the access status for this dataset.

Indicates how to obtain the dataset: public download, HuggingFace, contact authors, not yet released, etc.

pub fn mirror_url(&self) -> Option<&'static str>

Get mirror URL for this dataset (alternative download location).

pub fn depends_on(&self) -> Option<&'static str>

Get dependency dataset name (e.g., GCDC depends on “Yahoo L6 corpus”).

pub fn is_automatable(&self) -> bool

Check if this dataset can be automatically downloaded.

pub fn requires_license(&self) -> bool

Check if this dataset requires a license agreement (no public URL).

pub fn has_spdx_license(&self) -> bool

Check if this dataset has SPDX-compatible license.

pub fn cache_filename(&self) -> &'static str

Get the cache filename (derived from variant name).

pub fn is_coreference(&self) -> bool

Check if this dataset is for coreference resolution.

pub fn is_biomedical(&self) -> bool

Check if this dataset is biomedical domain.

pub fn is_social_media(&self) -> bool

Check if this dataset is social media domain.

pub fn is_relation_extraction(&self) -> bool

Check if this dataset is for relation extraction.

pub fn is_multilingual(&self) -> bool

Check if this dataset is multilingual.

pub fn is_historical(&self) -> bool

Check if this dataset is for historical texts.

pub fn is_bias_evaluation(&self) -> bool

Check if this dataset is for bias evaluation.

pub fn is_indigenous(&self) -> bool

Check if this dataset is indigenous language.

pub fn is_african_language(&self) -> bool

Check if this dataset is an African language dataset.

pub fn is_literary(&self) -> bool

Check if this dataset is for literary/fiction texts.

pub fn is_nested_ner(&self) -> bool

Check if this dataset has nested NER annotations.

pub fn is_discontinuous_ner(&self) -> bool

Check if this dataset has discontinuous NER annotations.

pub fn is_dialogue(&self) -> bool

Check if this dataset is dialogue/conversational.

pub fn is_ner(&self) -> bool

Check if this dataset is NER (not coreference).

pub fn is_event_coref(&self) -> bool

Check if this dataset is for cross-document event coreference.

pub fn is_ancient(&self) -> bool

Check if this dataset is for ancient/classical languages.

pub fn is_abstract_anaphora(&self) -> bool

Check if this dataset is for abstract anaphora/discourse deixis.

pub fn is_low_resource(&self) -> bool

Check if this dataset is for low-resource/endangered languages.

pub fn is_constructed(&self) -> bool

Check if this dataset is for constructed languages.

pub fn is_arcane_domain(&self) -> bool

Check if this dataset is for arcane/specialized domains.

pub fn is_adversarial(&self) -> bool

Check if this dataset is for adversarial/robustness evaluation.

pub fn is_speech(&self) -> bool

Check if this dataset is for speech/audio NER.

pub fn is_entity_linking(&self) -> bool

Check if this dataset is for entity linking / named entity disambiguation.

pub fn is_long_document(&self) -> bool

Check if this dataset is for long-document processing (>10k tokens).

pub fn is_clinical(&self) -> bool

Check if this dataset is clinical/medical domain.

pub fn tasks_or_inferred(&self) -> Vec<&'static str>

Get tasks with fallback to category-based inference.

Many datasets don’t have explicit tasks defined but have task-like categories (ner, coref, relation_extraction). This method returns explicit tasks if defined, otherwise infers from categories.

Migration Note

This exists for backwards compatibility. New datasets should use the tasks field explicitly. Categories should be domain/property focused (biomedical, multilingual, nested) not task-like.

pub fn supports_ner(&self) -> bool

Check if this is a NER dataset (explicit or inferred).

pub fn supports_coref(&self) -> bool

Check if this is a coreference dataset (explicit or inferred).

pub fn supports_re(&self) -> bool

Check if this is a relation extraction dataset (explicit or inferred).

pub fn requires_hf_token(&self) -> bool

Whether this dataset is known to require HF_TOKEN (or equivalent) to download.

This is used to keep “automatic” workflows (cache warming, randomized matrices) usable by default while still allowing explicit attempts when a token is provided.

pub fn is_automatable_download(&self) -> bool

Whether this dataset is currently automatable by our loader without manual intervention.

This is intentionally conservative: it can be false for datasets that are “public” but do not offer a stable export path we can consume yet.

pub fn default_metadata(&self) -> DatasetMetadata

Get default metadata for this dataset.

Delegates to registry methods.

Trait Implementations

impl Clone for LoadableDatasetId

fn clone(&self) -> LoadableDatasetId

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for LoadableDatasetId

impl Debug for LoadableDatasetId

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Deref for LoadableDatasetId

type Target = DatasetId

The resulting type after dereferencing.

fn deref(&self) -> &Self::Target

Dereferences the value.

impl Eq for LoadableDatasetId

impl From<LoadableDatasetId> for DatasetId

fn from(value: LoadableDatasetId) -> Self

Converts to this type from the input type.

impl FromStr for LoadableDatasetId

type Err = Error

The associated error which can be returned from parsing.

fn from_str(s: &str) -> Result<Self, Self::Err>

Parses a string s to return a value of this type.

impl Hash for LoadableDatasetId

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for LoadableDatasetId

fn eq(&self, other: &LoadableDatasetId) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for LoadableDatasetId

impl TryFrom<DatasetId> for LoadableDatasetId

type Error = Error

The type returned in the event of a conversion error.

fn try_from(value: DatasetId) -> Result<Self, Self::Error>

Performs the conversion.