Module scheduler 

Scheduler module for kaish — pipelines, background jobs, and scatter/gather.

This module provides:

• Pipeline execution: Run commands connected by pipes, where stdout of one command flows to stdin of the next.
• Background jobs: Run commands in the background with &, track them, and wait for completion.
• Scatter/Gather: Parallel fan-out and collection for pipelines.

Architecture

┌─────────────────────────────────────────────────────────────┐
│                     PipelineRunner                          │
│  ┌─────────┐   channel   ┌─────────┐   channel   ┌────────┐│
│  │ cmd1    │────────────▶│ cmd2    │────────────▶│ cmd3   ││
│  │ (spawn) │   stdout    │ (spawn) │   stdout    │ (spawn)││
│  └─────────┘             └─────────┘             └────────┘│
└─────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────┐
│                    ScatterGatherRunner                        │
│  ┌────────────┐         ┌─────────────────────────┐        │
│  │ pre_scatter│────────▶│  scatter (fan-out)      │        │
│  └────────────┘         │  ┌───┐ ┌───┐ ┌───┐      │        │
│                         │  │ 1 │ │ 2 │ │ 3 │ ...  │        │
│                         │  └───┘ └───┘ └───┘      │        │
│                         │         │               │        │
│                         │         ▼               │        │
│                         │  gather (collect)       │        │
│                         └─────────────────────────┘        │
│                                    │                       │
│                                    ▼                       │
│                         ┌────────────┐                     │
│                         │ post_gather│                     │
│                         └────────────┘                     │
└─────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────┐
│                      JobManager                             │
│  jobs: HashMap<JobId, Job>                                  │
│  - spawn(pipeline) → JobId                                  │
│  - wait(JobId) → ExecResult                                 │
│  - wait_all() → Vec<ExecResult>                             │
│  - list() → Vec<JobInfo>                                    │
└─────────────────────────────────────────────────────────────┘

Structs

BoundedStream

A bounded stream backed by a ring buffer.

GatherOptions

Options for gather operation.

Job

A background job.

JobId

Unique identifier for a background job.

JobInfo

Information about a job for listing.

JobManager

Manager for background jobs.

PipeReader

Reading end of a pipe stream.

PipeWriter

Writing end of a pipe stream.

PipelineRunner

Runs pipelines by spawning tasks and connecting them via channels.

ScatterGatherRunner

Runs scatter/gather pipelines.

ScatterOptions

Options for scatter operation.

ScatterResult

Result from a single scatter worker.

StderrReceiver

Receiving end of the stderr stream.

StderrStream

Cloneable handle to the kernel’s stderr output stream.

StreamStats

Statistics about a bounded stream.

Enums

JobStatus

Status of a background job.

Constants

DEFAULT_STREAM_MAX_SIZE

Default maximum size for bounded streams (10MB).

PIPE_BUFFER_SIZE

Default pipe buffer capacity (matches Linux kernel pipe default).

Functions

build_tool_args

Build ToolArgs from AST Args, evaluating expressions.

drain_to_stream

Drain an async reader into a bounded stream.

extract_items

Extract items from structured data or text.

is_bool_type

Check if a type is considered boolean.

parse_gather_options

Parse gather options from tool args.

parse_scatter_options

Parse scatter options from tool args.

pipe_stream

Create a bounded pipe stream pair with the given capacity.

pipe_stream_default

Create a pipe stream pair with the default capacity (64KB).

schema_param_lookup

Extract parameter types from a tool schema.

stderr_stream

Create a new stderr stream pair.