Skip to main content

Lister

Struct Lister 

Source
pub struct Lister { /* private fields */ }
Available on crate feature std only.
Expand description

An external lister that iterates a directory URL and emits RDF for its entries.

The input URL is passed as one command-line argument, and stdin is connected to the null device. Sorting, numeric offset, and URI cursor bounds are delegated to the external program. options.limit is always enforced locally as a maximum number of stdout lines in every output mode; --limit is also passed unless native limit support is explicitly unsupported. The local cap works without native support and protects against bugs in programs that accept the flag. Captured stdout is framed into lines, then streamed in bounded batches with backpressure rather than buffered until the program exits. Stderr is drained concurrently while reading and retained for failure diagnostics.

Cursor bounds use entry URIs (JSON-LD @id) and are exclusive in the selected sort order. Both bounds may define an interval; they cannot be combined with numeric offset. The runner validates URI syntax but does not resolve IDs or inspect graphs; the child defines stable ordering and missing-ID behavior.

Implementations§

Source§

impl Lister

Source

pub fn new( program: impl AsRef<OsStr>, input: impl AsRef<str>, output: GraphOutput, options: ListerOptions, ) -> Self

Configures a lister for the directory URL input without starting it.

Adds configured --sort, --offset, --before, --after, --limit, and --output options as --name=value arguments, in that order, followed by options.other and the unvalidated URL. output selects stdout handling; stderr is captured for failure diagnostics. The runner also enforces the limit locally, even if the child accepts the flag but fails to honor it.

Native sorting and pagination support defaults to unknown; use with_capabilities to supply known support. This wrapper enforces limits locally but does not discover capabilities or emulate sorting, offset, or URI bounds. Unknown or supported options are forwarded. An unsupported limit flag is omitted; other explicitly unsupported requests are rejected by execute. Sorting precedes offset or cursor bounds, and limit applies last. The program contract counts complete entries, but this wrapper caps serialized lines without parsing entry boundaries. The SDK formats sort keys using SortKeys; the resulting expression must be supported by the selected program’s profile.

Source

pub fn with_capabilities(self, capabilities: ListerCapabilities) -> Self

Supplies native option support for the configured program without spawning it.

Unknown support preserves the default forwarding behavior. Explicitly supported requests are also forwarded. An unsupported --limit is omitted while retaining the local cap. Other explicitly unsupported requests fail at execution, without spawning or consuming an output writer. Unset requests are unaffected by capability metadata. Validation covers the typed sort, offset, before, and after fields, including Some(0) for offset; other remains a verbatim argument list and is not parsed.

The caller may derive this metadata from module manifests; the runner does not load or verify it. There is no sort/offset/cursor emulation yet. Future fallbacks must preserve sort → offset or cursor bounds → limit order: local sorting cannot operate on output truncated by a native limit.

use asimov_runner::{GraphOutput, Lister, ListerCapabilities, ListerOptions, OptionSupport, StreamExt};

let mut lister = Lister::new(
    "asimov-example-lister",
    "https://example.com/collection",
    GraphOutput::Captured,
    ListerOptions::builder().offset(10).limit(25).build(),
).with_capabilities(ListerCapabilities::builder()
    .sort(OptionSupport::Unsupported)
    .offset(OptionSupport::Supported)
    .limit(OptionSupport::Unsupported)
    .build());
let mut batches = lister.execute().await?;
while let Some(batch) = batches.next().await {
    for bytes in batch?.lines() {
        // Process a line, or use the entire batch for postprocessing.
    }
}
Source

pub async fn execute(&mut self) -> ListerResult

Starts a new lister process and returns its live listing stream, or returns an empty stream without spawning when the limit is zero.

Returns after spawning, without waiting for output or process completion. With captured stdout, items contain batches of complete lines retaining terminators when present, including a final unterminated line. Configure thresholds with Self::with_batching; the local line cap applies first. Other output modes yield no payload batches. With a limit, inherited and forwarded stdout is routed through the same line cap; ignored stdout is read and discarded until EOF or the cap. Stderr and individual line sizes are unbounded. Batch accumulation follows crate::BatchOptions. JSONL framing does not establish how many lines or RDF statements belong to one logical listing entry.

After option and capability validation, None leaves output unlimited. Some(0) returns an empty stream without spawning or consuming a writer. With a positive limit, the child is dropped as soon as the last permitted line is read, even if the caller retains the stream. Reaching the cap ends the listing intentionally: no later output or exit error is observed. Forwarded output is flushed on completion. Blank and unterminated final lines each count as one line. A partially filled final batch is emitted immediately at the cap.

§Errors

Returns an I/O InvalidInput error wrapped in ExecutorError::UnexpectedOther for mixed offset/cursor pagination or malformed absolute cursor URIs. Returns ExecutorError::UnsupportedOption before spawning if requested sorting, offset, or a cursor bound is explicitly unsupported, even with a zero limit. Otherwise returns an ExecutorError if spawning fails. Subsequent I/O and exit errors are delivered through the stream, after flushing buffered complete lines.

Source§

impl Lister

Source

pub fn with_batching(self, options: BatchOptions) -> Self

Sets batching thresholds for captured JSONL output. This does not change subprocess arguments, native pipeline edges, or listing limits. The default policy is crate::BatchOptions::default.

Trait Implementations§

Source§

impl Debug for Lister

Source§

fn fmt(&self, __derive_more_f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
Source§

impl Execute<Pin<Box<dyn Stream<Item = Result<JsonlBatch, ExecutorError>> + Send>>> for Lister

Source§

type Error = ExecutorError

The implementation-specific error returned directly by execution. Read more
Source§

fn execute<'life0, 'async_trait>( &'life0 mut self, ) -> Pin<Box<dyn Future<Output = ListerResult> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait,

Executes the operation using its current input and configuration. Read more
Source§

impl GraphProducer for Lister

Source§

impl Lister<Pin<Box<dyn Stream<Item = Result<JsonlBatch, ExecutorError>> + Send>>> for Lister

Source§

impl PipelineProgram for Lister

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T> Instrument for T

Source§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
Source§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, !>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<T> WithSubscriber for T

Source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more