Struct CkanClient 

pub struct CkanClient { /* private fields */ }

HTTP client for interacting with CKAN open data portals.

CKAN (Comprehensive Knowledge Archive Network) is an open-source data management system used by many government open data portals worldwide.

Examples

use ceres_client::CkanClient;

let client = CkanClient::new("https://dati.gov.it")?;
let dataset_ids = client.list_package_ids().await?;
println!("Found {} datasets", dataset_ids.len());

Implementations

impl CkanClient

pub fn new(base_url_str: &str) -> Result<Self, AppError>

Creates a new CKAN client for the specified portal.

Arguments

• base_url_str - The base URL of the CKAN portal (e.g., https://dati.gov.it)

Returns

Returns a configured CkanClient instance.

Errors

Returns AppError::Generic if the URL is invalid or malformed. Returns AppError::ClientError if the HTTP client cannot be built.

pub async fn list_package_ids(&self) -> Result<Vec<String>, AppError>

Fetches the complete list of dataset IDs from the CKAN portal.

This method calls the CKAN package_list API endpoint, which returns all dataset identifiers available in the portal.

Returns

A vector of dataset ID strings.

Errors

Returns AppError::ClientError if the HTTP request fails. Returns AppError::Generic if the CKAN API returns an error.

Performance Note

TODO(performance): Add pagination for large portals Large portals can have 100k+ datasets. CKAN supports limit/offset params. Consider: list_package_ids_paginated(limit: usize, offset: usize) Or streaming: list_package_ids_stream() -> impl Stream<Item = ...>

pub async fn show_package(&self, id: &str) -> Result<CkanDataset, AppError>

Fetches the full details of a specific dataset by ID.

This method calls the CKAN package_show API endpoint to retrieve complete metadata for a single dataset.

Arguments

• id - The unique identifier or name slug of the dataset

Returns

A CkanDataset containing the dataset’s metadata.

pub async fn search_modified_since( &self, since: DateTime<Utc>, ) -> Result<Vec<CkanDataset>, AppError>

Searches for datasets modified since a given timestamp.

Uses CKAN’s package_search API with a metadata_modified filter to fetch only datasets that have been updated since the last sync. This enables incremental harvesting with ~99% fewer API calls in steady state.

Arguments

• since - Only return datasets modified after this timestamp

Returns

A vector of CkanDataset containing all datasets modified since the given time. Unlike list_package_ids() + show_package(), this returns complete dataset objects in a single paginated query.

Errors

Returns AppError::ClientError if the HTTP request fails. Returns AppError::Generic if the CKAN API returns an error or doesn’t support the package_search endpoint (some older CKAN instances).

pub fn into_new_dataset(dataset: CkanDataset, portal_url: &str) -> NewDataset

Converts a CKAN dataset into Ceres’ internal NewDataset model.

This helper method transforms CKAN-specific data structures into the format used by Ceres for database storage.

Arguments

• dataset - The CKAN dataset to convert
• portal_url - The base URL of the CKAN portal

Returns

A NewDataset ready to be inserted into the database.

Examples

use ceres_client::CkanClient;
use ceres_client::ckan::CkanDataset;

let ckan_dataset = CkanDataset {
    id: "abc-123".to_string(),
    name: "air-quality-data".to_string(),
    title: "Air Quality Monitoring".to_string(),
    notes: Some("Data from air quality sensors".to_string()),
    extras: serde_json::Map::new(),
};

let new_dataset = CkanClient::into_new_dataset(
    ckan_dataset,
    "https://dati.gov.it"
);

assert_eq!(new_dataset.original_id, "abc-123");
assert_eq!(new_dataset.url, "https://dati.gov.it/dataset/air-quality-data");
assert_eq!(new_dataset.title, "Air Quality Monitoring");

Trait Implementations

impl Clone for CkanClient

fn clone(&self) -> CkanClient

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl PortalClient for CkanClient

type PortalData = CkanDataset

Type representing raw portal data before transformation.

async fn list_dataset_ids(&self) -> Result<Vec<String>, AppError>

Lists all dataset IDs available on the portal.

async fn get_dataset(&self, id: &str) -> Result<Self::PortalData, AppError>

Fetches detailed metadata for a specific dataset.

fn into_new_dataset(data: Self::PortalData, portal_url: &str) -> NewDataset

Converts portal-specific data into a normalized NewDataset.

async fn search_modified_since( &self, since: DateTime<Utc>, ) -> Result<Vec<Self::PortalData>, AppError>

Searches for datasets modified since the given timestamp.