Skip to main content

Ledger

Struct Ledger 

Source
pub struct Ledger { /* private fields */ }
Expand description

A per-job checkpoint ledger, backed by a single SQLite file.

The connection is behind a Mutex purely to make Ledger: Syncrusqlite::Connection is Send but not Sync (its statement cache uses unsynchronized interior mutability), and run_job holds a &Ledger across .await points in a task spawned onto a multi-threaded runtime, which requires the held reference to be Send, which in turn requires Ledger: Sync. There is normally only ever one writer (the job that owns this ledger), so contention is not a real concern; the lock exists to satisfy the type system’s threading rules, not to arbitrate real concurrent access.

Implementations§

Source§

impl Ledger

Source

pub fn open(path: &Path, graph_fingerprint: &str) -> Result<Self, LedgerError>

Open (creating if absent) a job’s ledger at path, ensuring both tables exist and job_status has its single row.

graph_fingerprint is recorded only when job_status doesn’t exist yet (a fresh job) — reopening an existing ledger leaves the originally-recorded fingerprint untouched, since comparing old vs. new fingerprint is a resume endpoint’s job, not open’s.

Source

pub fn job_dir(&self) -> &Path

The directory this job’s state lives in — the ledger file’s own parent. Fan-out results are materialized under here.

Source

pub fn graph_fingerprint(&self) -> Result<String>

The fingerprint recorded when this job was first submitted — compare against a freshly computed one before resuming.

Source

pub fn get_completed(&self, node_name: &str) -> Result<Option<Value>>

The recorded output of node_name, if it completed successfully. None for a node that never ran, is still pending, or was skipped.

Source

pub fn is_skipped(&self, node_name: &str) -> Result<bool>

Whether node_name was recorded as skipped (e.g. excluded by a branch decision).

Source

pub fn write_completed(&self, node_name: &str, output: &Value) -> Result<()>

Record node_name as completed with output, overwriting any prior checkpoint for that node.

Source

pub fn write_skipped(&self, node_name: &str) -> Result<()>

Record node_name as skipped, overwriting any prior checkpoint for that node.

Source

pub fn write_item_completed( &self, node_name: &str, item_index: usize, output: &Value, input: Option<&Value>, ) -> Result<()>

Record one fan-out item as completed with output.

Source

pub fn write_item_failed( &self, node_name: &str, item_index: usize, error: &str, input: Option<&Value>, ) -> Result<()>

Record one fan-out item as having concluded in failure.

Concluded is the operative word. An item still in flight when the process died must leave no row at all, so that resume re-runs it — whereas an item whose block genuinely returned Fail is recorded here and never retried. That distinction is the entire basis of fan-out resume semantics: it separates “this chunk is bad” from “we were interrupted”, without needing to ask which happened. input is stored so the item can be handed back later — see Ledger::escalations — and so the warehouse can say where a row came from. Successes carry it too, for the second reason: a warehouse row has to be traceable on its own, and “the input is still in the manifest” only helps somebody who has the manifest, the job directory, and the knowledge that item 4,013 was line 4,014.

Source

pub fn get_item_completed( &self, node_name: &str, item_index: usize, ) -> Result<Option<Value>>

One item’s recorded output, if it completed successfully.

Source

pub fn item_concluded(&self, node_name: &str, item_index: usize) -> Result<bool>

Whether this item already concluded, either way — the resume check. A concluded item is skipped; anything else is (re-)run.

Source

pub fn concluded_items( &self, node_name: &str, ) -> Result<Vec<(usize, Option<Value>, Option<String>)>>

Every concluded item for node_name, in index order, as (index, output, error) — exactly one of output/error is Some.

Source

pub fn concluded_rows(&self, node_name: &str) -> Result<Vec<ConcludedRow>>

Every concluded item of node_name, with everything the warehouse needs to describe it.

Distinct from Self::concluded_items, which exists to project the JSONL files and therefore deliberately discards the status once it has decided output-or-error. The warehouse keeps the status verbatim: a reader auditing bronze needs escalated and failed to stay different, because one means “a human was asked” and the other means “it simply did not work”.

Source

pub fn write_escalated( &self, node_name: &str, item_index: Option<usize>, reason: &str, input: Option<&Value>, ) -> Result<()>

Record that recovery was exhausted for node_name, giving up.

Stored as an ordinary concluded failure with a distinct status rather than in a table of its own — the composite key already carries node and item, and an escalation is a kind of concluded failure. Pass None for a whole node, Some(i) for one fan-out item.

Source

pub fn open_read_only(path: &Path) -> Result<Self, LedgerError>

Open an existing ledger read-only, running no schema statements.

Ledger::open is a writer: it runs CREATE TABLE IF NOT EXISTS and ALTER TABLE so a fresh or older ledger becomes usable. That is right when opening the ledger you are about to write, and wrong for reading somebody else’s — DDL takes an exclusive lock, so merely listing escalations across every job on the machine could lock the ledger of a job that is currently running and fail it.

This opens with SQLITE_OPEN_READ_ONLY and touches no schema. A ledger predating the drain columns therefore reads with them absent, which is handled rather than migrated: such rows come back with no input, which is exactly what they have.

Source

pub fn escalations(&self) -> Result<Vec<Escalation>>

What this job gave up on and nobody has handled yet — the queue.

Source

pub fn all_escalations(&self) -> Result<Vec<Escalation>>

Every escalation this job ever recorded, drained or not.

Draining marks rather than deletes, so this is the historical record. What went wrong stays worth knowing after it’s been handled — especially when the retry fails too.

Source

pub fn mark_drained( &self, node_name: &str, item_index: Option<usize>, ) -> Result<()>

Stamp one escalation as exported.

Called only after the manifest is safely on disk: a row claiming it was handled when the write failed is worse than one exported twice.

Source

pub fn check_or_record_manifest( &self, node_name: &str, digest: &str, item_count: usize, ) -> Result<Result<(), String>>

Record what node_name fanned out over, or verify it is unchanged.

Ok(Err(previous_digest)) means this node previously ran against a different manifest. Item indices are only meaningful relative to one specific manifest, so resuming would quietly pair recorded results with entirely different inputs — no graph-level fingerprint can catch an edit to the manifest file itself, which is why this exists.

The outer Result is storage failure; the inner one is the verdict.

Source

pub fn job_status(&self) -> Result<LedgerJobStatus>

The job’s own terminal status. Running until Ledger::finish is called.

Source

pub fn finish(&self, status: &str) -> Result<()>

Record the job’s terminal status (e.g. "completed", "failed", "cancelled").

Trait Implementations§

Source§

impl Debug for Ledger

Names the job this ledger belongs to without trying to render the SQLite connection, which is not Debug. Exists so callers can use Result-combinators like expect_err on Ledger::open.

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Allocation for T
where T: RefUnwindSafe + Send + Sync,

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T> Instrument for T

Source§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
Source§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self>

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

impl<T> Pointable for T

Source§

const ALIGN: usize

The alignment of pointer.
Source§

type Init = T

The type for initializers.
Source§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
Source§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
Source§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
Source§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
Source§

impl<T> PolicyExt for T
where T: ?Sized,

Source§

fn and<P, B, E>(self, other: P) -> And<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns Action::Follow only if self and other return Action::Follow. Read more
Source§

fn or<P, B, E>(self, other: P) -> Or<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns Action::Follow if either self or other returns Action::Follow. Read more
Source§

impl<T> Read<Exclusive, BecauseExclusive> for T
where T: ?Sized,

Source§

impl<T> Same for T

Source§

type Output = T

Should always be Self
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<T> WithSubscriber for T

Source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more