Struct ArchiveStore 

pub struct ArchiveStore { /* private fields */ }

No Debug implementation: do not accidentally log redactor state or content.

Implementations

impl ArchiveStore

pub fn new(base: &Path, scope_key: &str, logical_id: &str) -> Result<Self>

Open a project’s archive for a seal or a forget. Creates the base, context-archives and project directories if they are missing, makes them private again and syncs them, every time. A read must not go through here: see open_existing.

base may be reached through links (a home directory on another disk): it is resolved once, here, and the store lives in the real directory. Below the base no link is followed: context-archives, the project directory, a shelf or a file that is a link is refused.

pub fn open_existing( base: &Path, scope_key: &str, logical_id: &str, ) -> Result<Option<Self>>

Open a project’s archive for a read (fetch, fetch_note, search), if it is there. Nothing is created, no mode is changed and nothing is synced, so this also works where nothing can be written. None when the project has no archive directory: a search of it finds nothing and a fetch is NOT_FOUND. The base is resolved as new resolves it, and a link below the base is an error, as it is for new.

pub fn with_redactor(self, redactor: Arc<ArchiveRedactor>) -> Self

Opt in to screened tool evidence. Install the SAME trusted policy when reopening a store; retrieval never restores material withheld at seal.

pub fn with_abort(self, abort: Arc<AtomicBool>) -> Self

Once abort is set, an operation that has not been admitted yet (one waiting behind another operation in this process, or for another process to release the lock) fails with Interrupted without reading or writing any record. A read that needs no lock looks at the flag once, before it starts. Setup before admission (opening the store, projecting a seal) is not undone. An admitted operation always completes, so callers must still treat an abandoned seal as possibly published.

pub fn seal(&self, messages: &[SharedMessage], note: &str) -> Result<ArchiveRef>

pub fn seal_with_note( &self, messages: &[SharedMessage], note: &str, ) -> Result<(ArchiveRef, String)>

seal that also returns the screened note stored with the segment (the earlier note when this projection was already sealed), in the same locked operation.

pub fn seal_window( &self, messages: &[SharedMessage], note: &str, ) -> Result<SealedWindow>

Seal one window as one or more segments (“parts”). No per-segment budget fails it; what still can is what fails any seal (I/O, a busy lock, an abort, a part that was forgotten) and a note over its limit, which is refused before anything is written.

A window within the budgets is one part: the same segment, id and note as seal_with_note. A larger one is filled from its start: a part ends before the first message that would take it over a budget, and the next part starts there. Every part is an ordinary segment; together they hold every message of the window exactly once, in order. Only a message that is over a budget by itself is stored reduced, and is then a part of its own: it costs the window two parts more, the one that ends before it and itself.

Where a part ends depends on the messages from its start to the one after its end, and on nothing after that, so every part but the last is the same however long the window is. (One corner: a last part that was over the size of a segment only with the note in it was cut shorter, and is not cut when it is no longer the last.) Parts are sealed first to last, each in its own locked operation, and the note goes to the last one. After an error the parts already sealed stay (callers keep the source history, as for seal). A retry on the same window finds them by id, writes only what is missing and returns the same list. A retry on a window that has grown since (a failed turn leaves the user’s next message in it) finds every part but the last one of the earlier attempt; that one, if it was written, stays as a segment no list names, and its messages are stored again in the new last part.

pub fn fetch( &self, id: &str, start: usize, limit: usize, ) -> Result<Vec<SharedMessage>>

pub fn fetch_with_provenance( &self, id: &str, start: usize, limit: usize, ) -> Result<Vec<ArchivedMessage>>

start indexes ELIGIBLE messages; source_index retains original offsets. Oversized limits are rejected rather than clamped; start beyond end is empty.

pub fn fetch_note(&self, id: &str) -> Result<String>

Explicit host-only access to the hidden, screened note. Never include it in search descriptors or inject it as system/developer instructions.

pub fn search( &self, query: &str, limit: usize, ) -> Result<Vec<ArchiveDescriptor>>

Case-insensitive literal substring search of eligible source JSON, newest first. Empty query lists descriptors. Not an exhaustive ranking service: output is bounded by limit, with no hidden-note matches, and the scan by SEARCH_SCAN_FILES / SEARCH_SCAN_BYTES.

pub fn search_scan( &self, query: &str, limit: usize, ) -> Result<(Vec<ArchiveDescriptor>, Option<Unscanned>)>

search that also says what it did not read: None when nothing it was asked for is missing, otherwise an Unscanned. Shelves are read newest to oldest, a shelf’s files by modification time, the legacy slots last.

One place that cannot be read hides only itself. A shelf name that is not a directory that can be opened and listed, a link, a FIFO or a file that is not ours alone where a segment belongs, a legacy slot that cannot be read, and a file that is read and is not a valid record are each passed over, and each is counted in Unscanned::unread: the rows are what the rest of the project holds, and the caller is told that they are not all of it. Only the project directory itself failing to list fails the search.

Shelf files are read without the lock; it is taken, shared and only if there is a lock file, when the search gets as far as reading a legacy slot. Nothing is written either way.

pub fn forget(&self, id: &str) -> Result<Forgotten>

Durable, idempotent deletion of the content of segment id, in every file that holds it. Each is replaced, under the same name, by a marker; the id and digest are never freed for reuse. Unknown ids are NotFound. This is logical deletion, not secure erasure of filesystem snapshots. The only operation that replaces a file.

The store can hold one projection more than once: under each half of its digest, under one id on two shelves (stored again while a shelf could not be read), in a legacy slot, under an id an import kept. So forget goes by content. The record found under id names the projection, and every live record of it becomes a tombstone: under id and under both halves of the digest on every shelf that can be opened, and in every legacy slot that can be read.

When it returns, a tombstone of the projection is on a shelf under one of the two ids its digest gives it, which is where a later seal of the same window looks. If the projection was only stored under another id (a legacy slot’s, an import’s), that tombstone is written new, on the newest shelf. It is written before any other file is replaced, and a copy under one of those two ids is replaced before the others: an error or a crash part-way leaves the content refused to a seal, and forgetting again finishes. An id that is already a tombstone is not an early return for the same reason.

A shelf that cannot be opened, and a legacy slot or a file under one of the names that cannot be read or is not a valid record (and does not say “forgotten” for this projection), may hold a copy that was not reached: a damaged copy can still have the window’s text in it. That is never silent: Forgotten::unread says so, and forgetting again when the place can be read reaches it. A file under id on a newer shelf that is not a valid record is such a place too: the valid record under id on an older shelf is still found and forgotten.

pub fn for_scope(base: &Path, scope: &str) -> Result<Self>

Open the project directory named scope (64 lowercase hex characters, what a record’s scope is), for an import. Creates what is missing and makes it private, as new does. The store is of no chat: it imports, reads and forgets, and a seal through it is refused.

pub fn existing_scope(base: &Path, scope: &str) -> Result<Option<Self>>

for_scope that creates and changes nothing, as open_existing: None when the project has no directory. What a dry run of an import opens.

pub fn scope(&self) -> &str

The project directory’s name: the scope of every record in it.

pub fn import( &self, line: &[u8], rehome: bool, dry_run: bool, seen: &mut ImportSeen, ) -> Result<Result<Imported, ImportRefused>>

Bring one archive record into this project, as it is: line is the JSON object an archive file holds. It is stored under its own id with its own logical, digest, created_ms, messages and note; nothing is recomputed to make it fit. The one field an import may change is scope: with rehome it is set to this project’s, whatever the line says (the scope is not in the digest); without, a record of another project is refused.

The outer error is the machine’s (I/O, the lock held too long, a file under the id that cannot be read as a record, a shelf or slot that cannot be read and may hold the id): stop, and run again later. The inner one is the record’s (ImportRefused): nothing was written for it. What is refused: a line that is not exactly a record; whatever a read of the store refuses, by the store’s own check on the bytes that would be written, with the check’s name; a digest that is not the digest of the content, computed here; a message that is not an object; an id that holds other content; live content that was forgotten in this store; a tombstone of content this store holds live under another id.

An id is looked up as a fetch looks it up (every shelf, then the legacy slots), a digest as a seal does (its two halves on every shelf, then the slots). Importing a line a second time is ImportOutcome::Present and changes nothing. No file is ever replaced, except by the forget of ImportOutcome::Forgotten. A live record with created_ms gets that as its file’s time.

dry_run writes nothing at all (no file, shelf, lock file or mode) and answers what the real run would, given seen, which carries what the earlier lines of the run would have written. For a project that has no directory yet, see Self::import_dry_absent.

pub fn import_dry_absent( scope: &str, line: &[u8], rehome: bool, seen: &mut ImportSeen, ) -> Result<Result<Imported, ImportRefused>>

The dry run of import into the project directory scope when it does not exist (existing_scope is None): an empty store, and nothing is created.