1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
//! # llm-bucket
//!
//! This crate provides all primary domain logic, data models, and processing pipelines for use in `llm-bucket` workflows.
//!
//! - **Scope:** Only open-source pipeline, processing, config, and repository sync code is here.
//! *Proprietary upload/integration logic is handled outside this crate.*
//!
//! ## Architecture & Usage
//!
//! - This library is intended for maximum code reuse across the CLI, tests, and (potentially) other tools.
//! - All fundamental types (e.g., configs, processor inputs/outputs) and clean-pipeline functions live here, with clear interfaces for extensibility.
//!
//! ## Major Modules
//!
//! - [`config`]: Typed config models loaded from YAML, including source definitions.
//! - [`download`]: Download logic for source repositories (e.g., Git, Confluence), creating disk snapshots.
//! - [`preprocess`]: Processing/conversion of downloaded repos to uploadable items (PDFs, file flattening, etc).
//! - [`synchronise`]: High-level pipeline for end-to-end sync (download-process-upload/report).
//! - [`contract`]: Interface trait for uploading sources/items (mockable for test).
//! - [`code_to_pdf`]: Minimal stub conversion of code/README to PDF files.
//!
//! ## Example
//! ```rust
//! use llm_bucket::{download, preprocess, synchronise};
//! // (Example code to show intended usage...)
//! ```
//!
//! ## Contribution and Extending
//! When adding a new source or pipeline, declare your data model and process logic in a new submodule, and extend root/aggregate orchestrators in this crate first, not in the CLI shell.
//!