Crate orbok_workers 

orbok-workers

Synchronous pipeline workers for M5/M6: pull queued jobs from the catalog and execute them in dependency order.

Worker chain (per file):

[Scan queues Extract]
  → ExtractionWorker  (extract + cache + record)
  → ChunkAndIndexWorker (chunk + FTS index + chunk_locations)

Failure isolation: one file’s failure never stops the whole run (RFC-004 §16, RFC-005 §13). Workers update the relevant catalog records with the error category.

Re-exports

pub use cleanup_service::CleanupService;

pub use cleanup_service::FullCleanupOutcome;

pub use model_verifier::FileIssue;

pub use model_verifier::FileIssueKind;

pub use model_verifier::VerifyOutcome;

pub use model_verifier::verify_embedding_model;

pub use model_verifier::verify_outcome_summary;

pub use recovery::IntegrityReport;

pub use recovery::RecoveryReport;

pub use recovery::check_catalog_integrity;

pub use recovery::run_startup_recovery;

pub use storage::update_storage_accounting;

Modules

cleanup_service

End-to-end cleanup service (M10, RFC-011 §11): combines catalog-side cleanup (via CleanupExecutor) with cache-side cleanup (via CacheService), driven by a validated CleanupPlan.

model_verifier

Startup model verification (design decision §3, RFC-021).

recovery

Crash recovery (RFC-018): detects and repairs interrupted state left by a previous session that terminated abnormally.

storage

Storage accounting (RFC-011 §9): measures actual orbok storage consumption and updates the storage_accounting table.

Structs

ChunkAndIndexWorker

Chunk-and-index worker.

EmbeddingWorker

Embedding worker for one file.

ExtractionWorker

Extraction worker instance, held for the duration of an index run.

Functions

run_pending

Run all queued jobs until the queue is empty or limit jobs have been processed. Returns the number of jobs that succeeded.