Module ingest 

Resumable, idempotent ingest.

Three properties matter, and each is bought by one mechanism:

• Cheap re-runs — a cursor per source file records the byte offset consumed. A file whose size and mtime are unchanged is not opened at all.
• Idempotence — event ids are content-derived, so a replay produces lines that are already in the store and are dropped before they are written. Re-ingesting yields a byte-identical store.
• Crash safety — a cursor is written only after a file has been read to its last complete line. An interrupted run therefore replays from the last committed offset, and the replay dedupes, so an interrupted-then-resumed run and an uninterrupted one produce the same store.

A partial trailing line (a writer caught mid-append) is never consumed: it is reported as skipped for this run and picked up once it is complete.

Structs

AdapterIngest

What one adapter did, in the shape warden ingest prints.

IngestOptions

Scope for one ingest run. The window narrows which files are opened; it never drops events from a file that is read, because the store is supposed to be complete for every period it covers.

IngestReport

Result of an ingest run.

Functions

load_cursors

Read state/ingest.jsonl. Append-only, so later records for a path win; callers scan from the back.

run

Run every enabled, implemented adapter against the store.