pub struct Ledger { /* private fields */ }Expand description
A per-job checkpoint ledger, backed by a single SQLite file.
The connection is behind a Mutex purely to make Ledger: Sync —
rusqlite::Connection is Send but not Sync (its statement cache uses
unsynchronized interior mutability), and run_job holds a &Ledger
across .await points in a task spawned onto a multi-threaded runtime,
which requires the held reference to be Send, which in turn requires
Ledger: Sync. There is normally only ever one writer (the job that owns
this ledger), so contention is not a real concern; the lock exists to
satisfy the type system’s threading rules, not to arbitrate real
concurrent access.
Implementations§
Source§impl Ledger
impl Ledger
Sourcepub fn open(path: &Path, graph_fingerprint: &str) -> Result<Self, LedgerError>
pub fn open(path: &Path, graph_fingerprint: &str) -> Result<Self, LedgerError>
Open (creating if absent) a job’s ledger at path, ensuring both
tables exist and job_status has its single row.
graph_fingerprint is recorded only when job_status doesn’t exist
yet (a fresh job) — reopening an existing ledger leaves the
originally-recorded fingerprint untouched, since comparing old vs.
new fingerprint is a resume endpoint’s job, not open’s.
Sourcepub fn job_dir(&self) -> &Path
pub fn job_dir(&self) -> &Path
The directory this job’s state lives in — the ledger file’s own parent. Fan-out results are materialized under here.
Sourcepub fn graph_fingerprint(&self) -> Result<String>
pub fn graph_fingerprint(&self) -> Result<String>
The fingerprint recorded when this job was first submitted — compare against a freshly computed one before resuming.
Sourcepub fn get_completed(&self, node_name: &str) -> Result<Option<Value>>
pub fn get_completed(&self, node_name: &str) -> Result<Option<Value>>
The recorded output of node_name, if it completed successfully.
None for a node that never ran, is still pending, or was skipped.
Sourcepub fn is_skipped(&self, node_name: &str) -> Result<bool>
pub fn is_skipped(&self, node_name: &str) -> Result<bool>
Whether node_name was recorded as skipped (e.g. excluded by a
branch decision).
Sourcepub fn write_completed(&self, node_name: &str, output: &Value) -> Result<()>
pub fn write_completed(&self, node_name: &str, output: &Value) -> Result<()>
Record node_name as completed with output, overwriting any prior
checkpoint for that node.
Sourcepub fn write_skipped(&self, node_name: &str) -> Result<()>
pub fn write_skipped(&self, node_name: &str) -> Result<()>
Record node_name as skipped, overwriting any prior checkpoint for
that node.
Sourcepub fn write_item_completed(
&self,
node_name: &str,
item_index: usize,
output: &Value,
input: Option<&Value>,
) -> Result<()>
pub fn write_item_completed( &self, node_name: &str, item_index: usize, output: &Value, input: Option<&Value>, ) -> Result<()>
Record one fan-out item as completed with output.
Sourcepub fn write_item_failed(
&self,
node_name: &str,
item_index: usize,
error: &str,
input: Option<&Value>,
) -> Result<()>
pub fn write_item_failed( &self, node_name: &str, item_index: usize, error: &str, input: Option<&Value>, ) -> Result<()>
Record one fan-out item as having concluded in failure.
Concluded is the operative word. An item still in flight when the
process died must leave no row at all, so that resume re-runs it —
whereas an item whose block genuinely returned Fail is recorded
here and never retried. That distinction is the entire basis of
fan-out resume semantics: it separates “this chunk is bad” from “we
were interrupted”, without needing to ask which happened.
input is stored so the item can be handed back later — see
Ledger::escalations — and so the warehouse can say where a row
came from. Successes carry it too, for the second reason: a warehouse
row has to be traceable on its own, and “the input is still in the
manifest” only helps somebody who has the manifest, the job directory,
and the knowledge that item 4,013 was line 4,014.
Sourcepub fn get_item_completed(
&self,
node_name: &str,
item_index: usize,
) -> Result<Option<Value>>
pub fn get_item_completed( &self, node_name: &str, item_index: usize, ) -> Result<Option<Value>>
One item’s recorded output, if it completed successfully.
Sourcepub fn item_concluded(&self, node_name: &str, item_index: usize) -> Result<bool>
pub fn item_concluded(&self, node_name: &str, item_index: usize) -> Result<bool>
Whether this item already concluded, either way — the resume check. A concluded item is skipped; anything else is (re-)run.
Sourcepub fn concluded_items(
&self,
node_name: &str,
) -> Result<Vec<(usize, Option<Value>, Option<String>)>>
pub fn concluded_items( &self, node_name: &str, ) -> Result<Vec<(usize, Option<Value>, Option<String>)>>
Every concluded item for node_name, in index order, as
(index, output, error) — exactly one of output/error is Some.
Sourcepub fn concluded_rows(&self, node_name: &str) -> Result<Vec<ConcludedRow>>
pub fn concluded_rows(&self, node_name: &str) -> Result<Vec<ConcludedRow>>
Every concluded item of node_name, with everything the warehouse
needs to describe it.
Distinct from Self::concluded_items, which exists to project the
JSONL files and therefore deliberately discards the status once it has
decided output-or-error. The warehouse keeps the status verbatim: a
reader auditing bronze needs escalated and failed to stay
different, because one means “a human was asked” and the other means
“it simply did not work”.
Sourcepub fn write_escalated(
&self,
node_name: &str,
item_index: Option<usize>,
reason: &str,
input: Option<&Value>,
) -> Result<()>
pub fn write_escalated( &self, node_name: &str, item_index: Option<usize>, reason: &str, input: Option<&Value>, ) -> Result<()>
Record that recovery was exhausted for node_name, giving up.
Stored as an ordinary concluded failure with a distinct status rather
than in a table of its own — the composite key already carries node
and item, and an escalation is a kind of concluded failure. Pass
None for a whole node, Some(i) for one fan-out item.
Sourcepub fn open_read_only(path: &Path) -> Result<Self, LedgerError>
pub fn open_read_only(path: &Path) -> Result<Self, LedgerError>
Open an existing ledger read-only, running no schema statements.
Ledger::open is a writer: it runs CREATE TABLE IF NOT EXISTS and
ALTER TABLE so a fresh or older ledger becomes usable. That is right
when opening the ledger you are about to write, and wrong for reading
somebody else’s — DDL takes an exclusive lock, so merely listing
escalations across every job on the machine could lock the ledger of a
job that is currently running and fail it.
This opens with SQLITE_OPEN_READ_ONLY and touches no schema. A ledger
predating the drain columns therefore reads with them absent, which is
handled rather than migrated: such rows come back with no input, which
is exactly what they have.
Sourcepub fn escalations(&self) -> Result<Vec<Escalation>>
pub fn escalations(&self) -> Result<Vec<Escalation>>
What this job gave up on and nobody has handled yet — the queue.
Sourcepub fn all_escalations(&self) -> Result<Vec<Escalation>>
pub fn all_escalations(&self) -> Result<Vec<Escalation>>
Every escalation this job ever recorded, drained or not.
Draining marks rather than deletes, so this is the historical record. What went wrong stays worth knowing after it’s been handled — especially when the retry fails too.
Sourcepub fn mark_drained(
&self,
node_name: &str,
item_index: Option<usize>,
) -> Result<()>
pub fn mark_drained( &self, node_name: &str, item_index: Option<usize>, ) -> Result<()>
Stamp one escalation as exported.
Called only after the manifest is safely on disk: a row claiming it was handled when the write failed is worse than one exported twice.
Sourcepub fn check_or_record_manifest(
&self,
node_name: &str,
digest: &str,
item_count: usize,
) -> Result<Result<(), String>>
pub fn check_or_record_manifest( &self, node_name: &str, digest: &str, item_count: usize, ) -> Result<Result<(), String>>
Record what node_name fanned out over, or verify it is unchanged.
Ok(Err(previous_digest)) means this node previously ran against a
different manifest. Item indices are only meaningful relative to one
specific manifest, so resuming would quietly pair recorded results
with entirely different inputs — no graph-level fingerprint can catch
an edit to the manifest file itself, which is why this exists.
The outer Result is storage failure; the inner one is the verdict.
Sourcepub fn job_status(&self) -> Result<LedgerJobStatus>
pub fn job_status(&self) -> Result<LedgerJobStatus>
The job’s own terminal status. Running until Ledger::finish is
called.
Trait Implementations§
Source§impl Debug for Ledger
Names the job this ledger belongs to without trying to render the SQLite
connection, which is not Debug. Exists so callers can use
Result-combinators like expect_err on Ledger::open.
impl Debug for Ledger
Names the job this ledger belongs to without trying to render the SQLite
connection, which is not Debug. Exists so callers can use
Result-combinators like expect_err on Ledger::open.
Auto Trait Implementations§
impl !Freeze for Ledger
impl RefUnwindSafe for Ledger
impl Send for Ledger
impl Sync for Ledger
impl Unpin for Ledger
impl UnsafeUnpin for Ledger
impl UnwindSafe for Ledger
Blanket Implementations§
impl<T> Allocation for T
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§fn in_current_span(self) -> Instrumented<Self> ⓘ
fn in_current_span(self) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more