archivist-core 0.2.0

Platform-neutral core for the FicHub companion bot: FicHub REST API client, search/recommendation/download command logic, intent classification, pagination cache, and the PlatformMessage IR. Shared by every platform adapter (Discord, Telegram, Matrix, Slack, IRC, fediverse, CLI, web).
Documentation

archivist-core

Platform-neutral core for the FicHub companion bot ("the Archivist").

All business logic shared by every platform adapter — Discord, Telegram, Matrix, Slack, IRC, fediverse (Lemmy/Piefed), CLI and web — lives in this crate. It contains zero platform SDKs (no discord-rs, no teloxide, …): adapters depend on this crate, call the do_* functions, and render the returned PlatformMessage intermediate representation (IR) natively.

What's inside

Module Purpose
api Typed FicHub REST API client (FichubClient) — search, export/download, recommendations, bookmarks, feed, requests, roadmap, auth, forum (forum_* methods: categories, cursor-paginated topics, topic detail + after=, create, reply, follow, mark-read, search, moderation, metamod)
model Typed request/response structs matching the FicHub REST API (incl. forum models ForumCategory, ForumTopic, ForumPost, …)
dispatch CoreCtx + every do_* command: do_search, do_ask, do_quote, do_body, do_recs, do_roll, do_download, do_bookmark, do_metadata, do_help, do_kudos, and do_forum_* (categories, topics, topic, create, reply, follow, mark-read, search)
core The platform-neutral PlatformMessage IR: Text / Rich (header + RichItems + action rows) / File / Ephemeral
intent Free-form mention classification (Intent enum) — optional LLM (Ollama) with a heuristic fallback ladder; everything the LLM can produce is whitelist-checked before it runs
cache Redis-backed pagination cache shared with FicHub (session-based paging, response cache keyed by sha256 of the normalized query)
store Platform-scoped token store (/link flow): archivist:token:<platform>:<ext_id>
ratelimit Cross-platform rate limiting via shared Redis buckets
config Environment-driven BotConfig
error BotError — no platform-specific variants
util Supported-site URL detection, word formatting, truncation
lemmy Lemmy/Piefed community monitor (poll-based)
debug LLM-assisted diagnosis of failed searches

How commands flow

platform event ──► adapter ──► dispatch::do_*(&CoreCtx, ...) ──► PlatformMessage ──► adapter renders natively

The adapter owns platform plumbing (events, message handles, attachments, editing) and calls one do_* function. The core never touches the platform, so there is exactly one implementation of every command's behavior, shared by all adapters — and new platforms get every command for free.

CoreCtx bundles the shared pieces: a FichubClient, the PageCache, and an Arc<BotConfig>.

Adding a new platform adapter

  1. Implement BotConfig::default() loading (or reuse BotConfig::default() for local dev).
  2. On a platform event, translate it into a PlatformMessage and return it — that's the whole adapter contract.
use std::sync::Arc;
use archivist_core::api::FichubClient;
use archivist_core::cache::PageCache;
use archivist_core::config::BotConfig;
use archivist_core::core::PlatformMessage;
use archivist_core::dispatch::{do_search, CoreCtx};
use archivist_core::{BotError, Result};

// One adapter function: turn a platform event into a PlatformMessage.
async fn on_message(cfg: &BotConfig, user_id: u64, text: &str) -> Result<PlatformMessage> {
    let client = FichubClient::new(Arc::new(cfg.clone()))?;
    let cache = PageCache::connect(cfg).await?;
    let ctx = CoreCtx::new(client, cache, Arc::new(cfg.clone()));

    match text.trim() {
        "/search" => {
            // do_* functions are the single source of truth for behavior.
            let params = archivist_core::api::SearchParams::default();
            do_search(&ctx, user_id, &params, None).await
        }
        other => Err(BotError::Command(format!("unknown command: {other}"))),
    }
}

Configuration

Everything is read from environment variables at startup (see BotConfig for the full list and defaults):

Env var Default Purpose
FANFIC_ARCHIVIST_BASE_URL http://localhost:8000 FicHub REST API root (no trailing slash)
FANFIC_ARCHIVIST_REDIS_URL redis://localhost:6379 Redis for pagination cache + token store
FANFIC_ARCHIVIST_LLM_ENABLED 0 Optional Ollama intent classification / debug
FANFIC_ARCHIVIST_OLLAMA_URL http://localhost:11434 Ollama base URL
FANFIC_ARCHIVIST_INTENT_MODEL lfm2.5:8b Model for intent classification
FANFIC_ARCHIVIST_PAGE_SIZE 5 Results per rendered page
FANFIC_ARCHIVIST_API_PAGE_SIZE 20 Results fetched from the API per page
FANFIC_ARCHIVIST_ALLOW_UPLOAD 1 Allow direct EPUB upload on /download
FANFIC_ARCHIVIST_MAX_UPLOAD_BYTES 26214400 Max direct-upload bytes (25 MiB)
FANFIC_ARCHIVIST_FREEFORM 1 Enable free-form @mention handling
FANFIC_ARCHIVIST_LINK_TTL 600 /link code TTL in seconds
FANFIC_ARCHIVIST_INTENT_TIMEOUT_SECS 15 Intent-classification timeout
FANFIC_ARCHIVIST_INTENT_CACHE_TTL 3600 Intent-cache TTL in seconds
FANFIC_ARCHIVIST_LEMMY_* — Lemmy monitor (URL, account, community, poll interval)

All LLM features are optional: with FANFIC_ARCHIVIST_LLM_ENABLED=0 (default) the free-form path still handles fanfiction URLs, docs questions, and natural-language /ask fallback without any Ollama dependency.

Testing

cargo test -p archivist-core

Unit tests cover URL detection/normalization, pagination-row rendering, the PlatformMessage IR (describe, RichItem builder), config URL joining, and cache logic. Integration against a live FicHub instance requires FANFIC_ARCHIVIST_BASE_URL + FANFIC_ARCHIVIST_REDIS_URL pointing at real services.

License

AGPL-3.0-or-later. See the LICENSE file in the repository.