Crate mimir_librarian

Expand description

mimir-librarian — Mimir librarian. Ingests prose memory drafts, sanitises them (separates observations from directives), structures them into canonical Mimir Lisp, and commits them to the canonical log via the in-process mimir_core::Pipeline.

Category 1 of the 2026-04-21 Rolls Royce engineering plan for librarian-governed draft processing.

§Status

Draft-ingestion foundation. The scope-aware draft envelope, filesystem draft store, submit command, explicit file/directory sweep command, rename-based lifecycle transitions, one-shot run lifecycle skeleton, scratch-pipeline pre-emit validation, bounded LLM validation retry with durable commit, and shared workspace write-lock acquisition are wired. The remaining Category 1 work still lands one concern at a time per the build discipline documented in the Rolls Royce plan.

See the crate README.md for the architectural decisions from the 2026-04-21 Category 1 design conversation and the roadmap of follow-up PRs that fill the skeleton in.

§Architecture sketch

┌───────────────┐   ┌───────────────┐   ┌─────────────────────┐
│ Drafts surface│──▶│  LlmInvoker   │──▶│ PreEmitValidator    │
│ (filesystem)  │   │ (claude -p)   │   │ (mimir_core Pipeline│
│               │   │               │   │  clone-on-write)    │
└───────────────┘   └───────────────┘   └─────────┬───────────┘
                                                  │
                                                  ▼
                                        ┌──────────────────────┐
                                        │   mimir_core::Store  │
                                        │      commit_batch    │
                                        └──────────────────────┘

The draft lifecycle, LLM invocation, bounded validation retry, validator box, Store::commit_batch durable commit, supersession-conflict policy, exact duplicate filtering across all four memory types, and configurable same-day valid_at dedup for Semantic / Inferential records are wired. The binary also exposes a polling watch scheduler, and librarian-specific observability is wired for runner and processor paths. Store commits acquire the shared mimir_core::WorkspaceWriteLock, so direct librarian runs and MCP write sessions exclude each other without introducing an MCP dependency. The earlier Python prototype is retired and is no longer shipped in the public tree.

Structs§

ClaudeCliInvoker: Production LlmInvoker that shells out to the claude CLI in non-interactive mode.
DedupPolicy: Deterministic duplicate-detection policy for candidate records.
DeferredDraftProcessor: Placeholder processor used until LLM structuring / validation / commit processing is wired.
Draft: A prose memory draft staged for librarian processing.
DraftId: Content-addressed identifier for a draft. First 8 bytes of the SHA-256 of the prose, rendered as a 16-character lowercase hex string on display.
DraftMetadata: Metadata required by scope-model.md § 4 for every raw draft.
DraftRunItem: One processed draft row for operator-visible run summaries.
DraftRunSummary: Summary of one run.
DraftStore: Filesystem-backed draft surface.
DraftTransition: Result of a successful draft lifecycle transition.
LibrarianConfig: Runtime configuration for a single librarian invocation.
ParticipantVote: Vote plus confidence and rationale.
PreEmitValidator: Validates a candidate canonical Lisp record against a scratch mimir_core::Pipeline before the librarian attempts to commit it to the real log.
QuorumAdapterRequest: Request payload consumed by a future participant adapter.
QuorumEpisode: Requested quorum episode envelope.
QuorumParticipant: Participant identity captured for quorum auditability.
QuorumParticipantOutput: Participant prompt/response captured for a deliberation round.
QuorumResult: Synthesized quorum result envelope.
QuorumStore: File-backed quorum episode/result store.
RawArchiveDraftProcessor: Deterministic processor that archives raw draft text as governed pending-verification evidence without invoking an LLM.
RetryingDraftProcessor: LLM-backed draft processor with bounded validation retry.

Enums§

ConsensusLevel: Degree of agreement in a quorum result.
DecisionStatus: Final quorum decision status.
DraftProcessingDecision: Decision returned by a draft processor.
DraftSource: Where a draft came from. Drives provenance + retention policy (e.g. AutoMemorySweep drafts may be deleted after successful commit; CliSubmit drafts may be retained for operator review).
DraftSourceSurface: Source surface that exposed a draft to Mimir.
DraftState: Lifecycle state of a draft on the filesystem.
LibrarianError: Every externally-observable failure mode of the librarian.
QuorumEpisodeState: Lifecycle state for one quorum episode.
QuorumRound: Deliberation round that can receive participant outputs.
SupersessionConflictPolicy: Policy for deterministic supersession conflicts.
VoteChoice: Individual participant vote.

Constants§

DEFAULT_DEDUP_VALID_AT_WINDOW_SECS: Default duplicate-detection valid_at window, in seconds.
DEFAULT_LLM_TIMEOUT_SECS: Default claude -p invocation timeout, in seconds.
DEFAULT_MAX_RETRIES_PER_RECORD: Default maximum number of per-record retries when a candidate record fails pre-emit validation.
DEFAULT_PROCESSING_STALE_SECS: Default age after which a draft left in processing/ is assumed abandoned and recovered to pending/.
DRAFT_SCHEMA_VERSION: Current on-disk draft schema version.
QUORUM_SCHEMA_VERSION: Current on-disk quorum episode/result schema version.
SYSTEM_PROMPT: Canonical location of the librarian’s system prompt.

Traits§

DraftProcessor: Processor invoked for each claimed draft.
LlmInvoker: Ask the LLM to produce a JSON response for a prose draft.

Functions§

run_once: Run pending draft processing once.