rad-feed

A flexible Rust library for processing Radicle collaborative objects (patches and issues) with pluggable storage backends.

Overview

rad-feed processes operations from patches and issues across Radicle repositories and stores them using your choice of storage backend. Instead of being tied to a specific database, the library uses a trait-based approach that allows you to implement any storage system you need.

Features

• Storage Agnostic: Implement the FeedStorage trait to use any storage backend
• Incremental Processing: Only processes new operations since the last run
• Multi-repository Support: Process COB operations across all delegate repositories
• Action Deduplication: Built-in mechanisms to prevent duplicate processing
• Author Resolution: Automatic author information resolution using Radicle aliases

Implementing Custom Storage

The core of the library is the FeedStorage trait. Implement this trait to use any storage system:

use std::collections::HashMap;

use radicle::{cob::ObjectId, prelude::RepoId, git::Oid};

use rad_feed::storage::{FeedStorage, StorageError, StorageStats};
use rad_feed::entry::ActionEntry;

pub struct MyCustomStorage {
    // Your storage implementation
    data: HashMap<String, ActionEntry>,
    tracking: HashMap<String, Oid>,
}

impl FeedStorage for MyCustomStorage {
    type Error = StorageError;

    fn initialize(&mut self) -> Result<(), Self::Error> {
        // Initialize your storage (create tables, connections, etc.)
        Ok(())
    }

    fn get_last_processed_operation(
        &self,
        rid: &RepoId,
        cob_id: &ObjectId,
        cob_type: &str,
    ) -> Result<Option<Oid>, Self::Error> {
        // Return the last processed operation ID for incremental updates
        let key = format!("{}:{}:{}", rid, cob_id, cob_type);
        Ok(self.tracking.get(&key).copied())
    }

    fn update_last_processed_operation(
        &self,
        rid: &RepoId,
        cob_id: &ObjectId,
        cob_type: &str,
        last_operation_id: &Oid,
    ) -> Result<(), Self::Error> {
        // Update the tracking information
        let key = format!("{}:{}:{}", rid, cob_id, cob_type);
        // Note: In a real implementation, you'd want &mut self or interior mutability
        Ok(())
    }

    fn operation_exists(&self, operation_id: &Oid) -> Result<bool, Self::Error> {
        // Check if action already exists (for duplicate prevention)
        Ok(self.data.contains_key(&operation_id.to_string()))
    }

    fn insert_batch(&mut self, entries: &[ActionEntry]) -> Result<(), Self::Error> {
        // Insert new operations into your storage
        for entry in entries {
            self.data.insert(entry.operation_id.to_string(), entry.clone());
        }
        Ok(())
    }

    // Optional methods with default implementations
    fn get_stats(&self) -> Result<StorageStats, Self::Error> {
        let mut stats = StorageStats::default();
        stats.total_actions = self.data.len() as u64;

        for entry in self.data.values() {
            *stats.actions_by_type.entry(entry.kind.clone()).or_insert(0) += 1;
        }

        Ok(stats)
    }
}

Architecture

Core Components

1. FeedStorage Trait (src/storage/mod.rs): The main abstraction for storage backends
2. FeedProcessor (src/feed.rs): Processes repositories and manages the storage lifecycle
3. ActionEntry (src/entry.rs): The core data structure representing COB operations
4. Built-in Storages: SQLite and in-memory implementations

Action Data Structure

Each ActionEntry contains:

• operation_id: Unique identifier of the COB operation (Git OID)
• cob_id: Collaborative Object ID (patch or issue ID)
• rid: Repository ID where the action occurred
• timestamp: When the action was performed
• action: Serialized action data (JSON string)
• author: Author information with DID and optional alias
• typename: Typename of the COB (e.g. "xyz.radicle.patch" or "xyz.radicle.issue")