Struct DatasetRepository 

pub struct DatasetRepository { /* private fields */ }

Repository for dataset persistence in PostgreSQL with pgvector.

Examples

use sqlx::postgres::PgPoolOptions;
use ceres_db::DatasetRepository;

let pool = PgPoolOptions::new()
    .max_connections(5)
    .connect("postgresql://localhost/ceres")
    .await?;

let repo = DatasetRepository::new(pool);

Implementations

impl DatasetRepository

pub fn new(pool: PgPool) -> Self

pub async fn upsert(&self, new_data: &NewDataset) -> Result<Uuid, AppError>

Inserts or updates a dataset. Returns the UUID of the affected row.

TODO(robustness): Return UpsertOutcome to distinguish insert vs update Currently returns only UUID without indicating operation type. Consider: pub enum UpsertOutcome { Created(Uuid), Updated(Uuid) } This enables accurate progress reporting in sync statistics.

pub async fn get_hashes_for_portal( &self, portal_url: &str, ) -> Result<HashMap<String, Option<String>>, AppError>

Returns a map of original_id → content_hash for all datasets from a portal.

TODO(performance): Optimize for large portals (100k+ datasets) Currently loads entire HashMap into memory. Consider: (1) Streaming hash comparison during sync, or (2) Database-side hash check with WHERE clause, or (3) Bloom filter for approximate membership testing

pub async fn update_timestamp_only( &self, portal_url: &str, original_id: &str, ) -> Result<bool, AppError>

Updates only the timestamp for unchanged datasets. Returns true if a row was updated.

pub async fn get(&self, id: Uuid) -> Result<Option<Dataset>, AppError>

Retrieves a dataset by UUID.

pub async fn search( &self, query_vector: Vector, limit: usize, ) -> Result<Vec<SearchResult>, AppError>

Semantic search using cosine similarity. Returns results ordered by similarity.

pub async fn list_all( &self, portal_filter: Option<&str>, limit: Option<usize>, ) -> Result<Vec<Dataset>, AppError>

Lists datasets with optional portal filter and limit.

TODO(config): Make default limit configurable via DEFAULT_EXPORT_LIMIT env var Currently hardcoded to 10000. For large exports, consider streaming instead.

TODO(performance): Implement streaming/pagination for memory efficiency Loading all datasets into memory doesn’t scale. Consider returning impl Stream<Item = Result<Dataset, AppError>> or cursor-based pagination.

pub async fn get_stats(&self) -> Result<DatabaseStats, AppError>

Returns aggregated database statistics.

Trait Implementations

impl Clone for DatasetRepository

fn clone(&self) -> DatasetRepository

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.