Struct DcatClient 

pub struct DcatClient { /* private fields */ }

HTTP client for harvesting DCAT-AP portals using the udata REST catalog endpoint.

Implementations

impl DcatClient

pub fn new(base_url_str: &str, language: &str) -> Result<Self, AppError>

Creates a new DCAT client for the specified portal.

Arguments

• base_url_str - The base URL of the portal (e.g., https://data.public.lu)
• language - Preferred language for resolving multilingual fields (e.g., "fr", "en")

Errors

Returns AppError::InvalidPortalUrl if the URL is malformed. Returns AppError::ClientError if the HTTP client cannot be built.

pub fn portal_type(&self) -> &'static str

Returns the portal type identifier.

pub fn base_url(&self) -> &str

Returns the base URL of the portal.

pub async fn list_dataset_ids(&self) -> Result<Vec<String>, AppError>

Returns all dataset identifiers by fetching all datasets and extracting their IDs.

This is not optimal — udata has no lightweight ID-list endpoint — but is acceptable since the harvest pipeline typically uses search_all_datasets.

pub async fn get_dataset(&self, _id: &str) -> Result<DcatDataset, AppError>

Not implemented: the udata single-dataset endpoint returns plain JSON, not JSON-LD, requiring a separate parser. Use search_all_datasets instead.

pub async fn search_modified_since( &self, since: DateTime<Utc>, ) -> Result<Vec<DcatDataset>, AppError>

Searches for datasets modified since the given timestamp.

Uses the modified_since query parameter supported by udata portals. The response is still paginated and follows the same hydra:next pattern.

pub async fn search_all_datasets(&self) -> Result<Vec<DcatDataset>, AppError>

Fetches all datasets from the portal using paginated catalog requests.

pub fn into_new_dataset( data: DcatDataset, portal_url: &str, _url_template: Option<&str>, language: &str, ) -> NewDataset

Converts a DcatDataset into the normalized NewDataset model.

The url_template parameter is ignored for DCAT portals because the JSON-LD @id field already provides the canonical landing page URL.

pub fn paginate_catalog_stream( &self, modified_since: Option<String>, ) -> BoxStream<'_, Result<Vec<DcatDataset>, AppError>>

Streams catalog datasets page-by-page instead of accumulating into a single Vec.

Each yielded item contains the datasets extracted from one catalog page (~100 datasets). Memory is bounded to a single page at a time.

Trait Implementations

impl Clone for DcatClient

fn clone(&self) -> DcatClient

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.