Struct Store 

pub struct Store { /* private fields */ }

A BIRD store for reading and writing records.

Implementations

impl Store

pub fn write_attempt(&self, record: &AttemptRecord) -> Result<()>

Write an attempt record to the store (v5 schema).

Call this at invocation start. The outcome should be written when the invocation completes using write_outcome().

Behavior depends on storage mode:

• Parquet: Creates a new Parquet file in the appropriate date partition
• DuckDB: Inserts directly into the local.attempts table

pub fn start_invocation(&self, record: &AttemptRecord) -> Result<AttemptRecord>

Start an invocation by writing an attempt (v5 schema).

This is the v5 equivalent of start_pending_invocation(). Returns the attempt record for later use with complete_invocation().

pub fn attempt_count(&self) -> Result<i64>

Get the count of attempts in the store.

pub fn get_pending_attempts(&self) -> Result<Vec<AttemptRecord>>

Get pending attempts (attempts without outcomes).

In v5 schema, this replaces the pending file mechanism: SELECT * FROM attempts WHERE id NOT IN (SELECT attempt_id FROM outcomes)

impl Store

pub fn compact_partition_with_opts( &self, partition_dir: &Path, opts: &CompactOptions, ) -> Result<CompactStats>

Compact files in a partition directory, grouped by session.

Behavior depends on options:

• consolidate: Merge ALL files into single file per session
• file_threshold: Compact when > N non-compacted files
• recompact_threshold: Re-compact when > N compacted files exist

pub fn compact_partition( &self, partition_dir: &Path, file_threshold: usize, session_filter: Option<&str>, dry_run: bool, ) -> Result<CompactStats>

Compact files in a partition directory (legacy API).

pub fn compact_data_type( &self, data_dir: &Path, file_threshold: usize, session_filter: Option<&str>, dry_run: bool, ) -> Result<CompactStats>

Compact all partitions in a data type directory (attempts, outcomes, outputs, sessions).

pub fn compact_for_session( &self, session_id: &str, file_threshold: usize, dry_run: bool, ) -> Result<CompactStats>

Compact recent data for a specific session (used by shell hooks).

Checks all date partitions in recent data.

pub fn compact_session_today( &self, session_id: &str, file_threshold: usize, dry_run: bool, ) -> Result<CompactStats>

Fast compaction check for today’s partition only (used by shell hooks).

This is the most lightweight check - only looks at today’s date partition.

pub fn compact_recent( &self, file_threshold: usize, dry_run: bool, ) -> Result<CompactStats>

Compact all recent data that exceeds the file threshold (global mode).

pub fn compact_archive( &self, file_threshold: usize, dry_run: bool, ) -> Result<CompactStats>

Compact all archive data that exceeds the file threshold.

pub fn archive_old_data( &self, older_than_days: u32, dry_run: bool, ) -> Result<ArchiveStats>

Migrate old data from recent to archive with consolidation.

Consolidates all files in each date partition into a single parquet file in the archive, then removes the source files.

pub fn auto_compact( &self, opts: &AutoCompactOptions, ) -> Result<(CompactStats, ArchiveStats)>

Run auto-compaction based on options.

pub fn compact_recent_with_opts( &self, opts: &CompactOptions, ) -> Result<CompactStats>

Compact recent data with full options.

pub fn compact_archive_with_opts( &self, opts: &CompactOptions, ) -> Result<CompactStats>

Compact archive data with full options.

pub fn compact_data_type_with_opts( &self, data_dir: &Path, opts: &CompactOptions, ) -> Result<CompactStats>

Compact data type directory with full options.

pub fn compact_for_session_with_opts( &self, session_id: &str, opts: &CompactOptions, ) -> Result<CompactStats>

Compact files for a specific session with full options.

pub fn clean(&self, opts: &CleanOptions) -> Result<CleanStats>

Clean operation: recover orphaned invocations and optionally prune archive.

This:

1. Scans for pending invocations (attempts without matching outcomes)
2. Creates outcome records for orphaned invocations
3. Optionally prunes old archive data

pub fn prune_archive( &self, older_than_days: u32, dry_run: bool, ) -> Result<PruneStats>

Prune old archive data.

Deletes data from the archive tier older than the specified number of days.

impl Store

pub fn load_format_config(&self) -> Result<FormatConfig>

Load format config from BIRD_ROOT/event-formats.toml.

pub fn detect_format(&self, cmd: &str) -> Result<String>

Detect format for a command using DuckDB SQL matching.

Uses SQL LIKE patterns for matching, which prepares for future integration with duck_hunt_match_command_patterns().

pub fn extract_events( &self, invocation_id: &str, format_override: Option<&str>, ) -> Result<usize>

Extract events from an invocation’s output using duck_hunt.

Parses the stdout/stderr of an invocation and stores the extracted events. Uses read_duck_hunt_log() directly on storage refs for efficiency - no content is loaded into Rust memory.

Returns the number of events extracted.

pub fn write_events(&self, records: &[EventRecord]) -> Result<()>

Write event records to the store.

Behavior depends on storage mode:

• Parquet: Creates Parquet files partitioned by date
• DuckDB: Inserts directly into the local.events

pub fn query_events(&self, filters: &EventFilters) -> Result<Vec<EventSummary>>

Query events with optional filters.

pub fn event_count(&self, filters: &EventFilters) -> Result<i64>

Count events matching the given filters.

pub fn delete_events_for_invocation(&self, invocation_id: &str) -> Result<usize>

Delete events for an invocation (for re-extraction).

Behavior depends on storage mode:

• Parquet: Deletes parquet files containing the events
• DuckDB: Deletes rows from local.events

pub fn invocations_without_events( &self, since: Option<NaiveDate>, limit: Option<usize>, ) -> Result<Vec<InvocationSummary>>

Get invocations that have outputs but no events extracted yet.

Useful for backfilling events from existing invocations.

impl Store

pub fn write_invocation(&self, record: &InvocationRecord) -> Result<()>

Write an invocation record to the store (v5 schema).

This writes both an attempt and an outcome record, since the invocation is already complete. For long-running commands, use start_invocation() followed by complete_invocation().

pub fn recent_invocations(&self, limit: usize) -> Result<Vec<InvocationSummary>>

Get recent invocations (last 7 days).

pub fn last_invocation(&self) -> Result<Option<InvocationSummary>>

Get the last invocation (most recent).

pub fn query_invocations_with_limit( &self, query: &Query, default_limit: usize, ) -> Result<Vec<InvocationSummary>>

Query invocations with filters from the query micro-language.

Supports:

• ~N range selector (limit to N results)
• %exit<>0 field filters (exit code, duration, etc.)
• %/pattern/ command regex

Use default_limit to specify the limit when no range is provided:

• 20 for listing commands (shq i)
• 1 for single-item commands (shq o, shq I, shq R)

pub fn query_invocations(&self, query: &Query) -> Result<Vec<InvocationSummary>>

Query invocations with default limit of 20 (for listing).

pub fn invocation_count(&self) -> Result<i64>

Count total invocations in the store.

pub fn find_by_tag(&self, tag: &str) -> Result<Option<String>>

Find an invocation by its tag. Returns the full invocation ID if found.

pub fn set_tag(&self, invocation_id: &str, tag: Option<&str>) -> Result<()>

Set or update the tag on an invocation.

V5 schema: Updates the tag on the attempts table.

pub fn recover_orphaned_invocations( &self, max_age_hours: u32, dry_run: bool, ) -> Result<RecoveryStats>

Recover orphaned invocations (v5 schema).

V5: Scans attempts without outcomes and checks if the runner is still alive. If not alive, writes an orphaned outcome record.

Note: This now looks at machine_id field which stores the runner_id for local invocations.

impl Store

pub fn write_outcome(&self, record: &OutcomeRecord) -> Result<()>

Write an outcome record to the store (v5 schema).

Call this when an invocation completes (success, failure, or crash). The attempt should have been written at invocation start using write_attempt().

Behavior depends on storage mode:

• Parquet: Creates a new Parquet file in the appropriate date partition
• DuckDB: Inserts directly into the local.outcomes table

pub fn complete_invocation( &self, attempt_id: Uuid, exit_code: i32, duration_ms: Option<i64>, date: NaiveDate, ) -> Result<()>

Complete an invocation by writing an outcome (v5 schema).

This is the v5 equivalent of complete_pending_invocation().

pub fn orphan_invocation(&self, attempt_id: Uuid, date: NaiveDate) -> Result<()>

Mark an invocation as orphaned (crashed without cleanup).

pub fn kill_invocation( &self, attempt_id: Uuid, signal: i32, duration_ms: Option<i64>, date: NaiveDate, ) -> Result<()>

Mark an invocation as killed by signal.

pub fn timeout_invocation( &self, attempt_id: Uuid, duration_ms: i64, date: NaiveDate, ) -> Result<()>

Mark an invocation as timed out.

pub fn outcome_count(&self) -> Result<i64>

Get the count of outcomes in the store.

pub fn recover_orphans(&self) -> Result<RecoveryStats>

Recover orphaned invocations (v5 schema).

Finds attempts without outcomes where the runner is no longer alive, and marks them as orphaned. Returns statistics about the operation.

This is safe to run periodically (e.g., during compaction) and is idempotent - it won’t create duplicate outcomes.

impl Store

pub fn store_output( &self, invocation_id: Uuid, stream: &str, content: &[u8], date: NaiveDate, cmd_hint: Option<&str>, ) -> Result<()>

Store output content, routing to inline or blob based on size.

This is the high-level method for storing invocation output. It:

1. Computes the BLAKE3 hash
2. Routes small content to inline (data: URL) or large to blob (file: URL)
3. Handles deduplication for blobs
4. Writes the output record to Parquet

pub fn write_output(&self, record: &OutputRecord) -> Result<()>

Write an output record to the store (low-level).

Behavior depends on storage mode:

• Parquet: Creates a new Parquet file in the appropriate date partition
• DuckDB: Inserts directly into the local.outputs

pub fn get_outputs( &self, invocation_id: &str, stream_filter: Option<&str>, ) -> Result<Vec<OutputInfo>>

Get outputs for an invocation by ID, optionally filtered by stream.

pub fn get_output(&self, invocation_id: &str) -> Result<Option<OutputInfo>>

Get output for an invocation by ID (first match, for backwards compat).

pub fn read_output_content(&self, output: &OutputInfo) -> Result<Vec<u8>>

Read content from storage using DuckDB’s read_blob (handles both data: and file:// URLs).

impl Store

pub fn push( &self, remote: &RemoteConfig, opts: PushOptions, ) -> Result<PushStats>

Push local data to a remote.

Reads from local schema, writes to remote’s tables. Only pushes records that don’t already exist on the remote (by id). When sync_blobs is enabled, also syncs blob files for file remotes.

pub fn pull( &self, remote: &RemoteConfig, opts: PullOptions, ) -> Result<PullStats>

Pull data from a remote into local cached_ schema.

Reads from remote’s tables, writes to cached_<name> schema. Only pulls records that don’t already exist in the cached schema (by id). After pulling, rebuilds the caches union views. When sync_blobs is enabled, also syncs blob files for file remotes.

pub fn rebuild_caches_schema(&self, conn: &Connection) -> Result<()>

Rebuild the caches schema views to union all cached_* schemas.

Uses explicit transaction for DDL safety. The caches.* views reference local cached_* schemas (not attached databases), so they should be safe to persist. V5 schema: unions attempts/outcomes tables and creates invocations view.

impl Store

pub fn write_session(&self, record: &SessionRecord) -> Result<()>

Write a session record to the store.

Behavior depends on storage mode:

• Parquet: Creates a new Parquet file in the appropriate date partition
• DuckDB: Inserts directly into the local.sessions

Sessions are written lazily on first invocation from that session.

pub fn session_exists(&self, session_id: &str) -> Result<bool>

Check if a session exists in the store.

pub fn ensure_session(&self, record: &SessionRecord) -> Result<()>

Ensure a session is registered, creating it if needed.

This is called lazily when an invocation is recorded. If the session doesn’t exist, it creates a new session record.

pub fn session_count(&self) -> Result<i64>

Count total sessions in the store.

impl Store

pub fn open(config: Config) -> Result<Self>

Open an existing BIRD store.

pub fn connection(&self) -> Result<Connection>

Get a DuckDB connection with full features (attachments, ephemeral views).

pub fn connection_with_options( &self, attach_remotes: bool, ) -> Result<Connection>

Get a DuckDB connection with optional remote attachment (legacy API).

pub fn connect(&self, opts: ConnectionOptions) -> Result<Connection>

Get a DuckDB connection with explicit options.

This is the main connection method. Use ConnectionOptions to control:

• Whether remotes are attached
• Whether project database is attached
• Whether ephemeral views are created
• Whether migration should run

Uses retry with exponential backoff to handle concurrent access.

pub fn attach_remote( &self, conn: &Connection, remote: &RemoteConfig, ) -> Result<()>

Manually attach a specific remote.

pub fn detach_remote(&self, conn: &Connection, name: &str) -> Result<()>

Detach a remote.

pub fn test_remote(&self, remote: &RemoteConfig) -> Result<()>

Test connection to a remote. Returns Ok if successful.

pub fn config(&self) -> &Config

Get config reference.

pub fn query(&self, sql: &str) -> Result<QueryResult>

Query the store using SQL.

Returns results as a Vec of rows, where each row is a Vec of string values.

pub fn last_invocation_with_output( &self, ) -> Result<Option<(InvocationSummary, Option<OutputInfo>)>>

Get the last invocation with its output (if any).

pub fn write_batch(&self, batch: &InvocationBatch) -> Result<()>

Write a batch of related records atomically.

This is the preferred way to write an invocation with its outputs, session, and events together. In DuckDB mode, all writes are wrapped in a transaction. In Parquet mode, files are written atomically.

pub fn load_format_hints(&self) -> Result<FormatHints>

Load format hints from the config file.

pub fn save_format_hints(&self, hints: &FormatHints) -> Result<()>

Save format hints to the config file.

pub fn detect_format_for_command(&self, cmd: &str) -> Result<String>

Detect format for a command using format hints.

Priority:

1. User-defined format hints (by priority)
2. Default format from config (or “auto”)

Note: duck_hunt detects formats from content analysis, not command names. Use format hints to map commands to formats, then duck_hunt parses the output.

pub fn list_builtin_formats(&self) -> Result<Vec<BuiltinFormat>>

Get list of duck_hunt built-in formats.

Note: duck_hunt detects formats from content analysis, not command patterns. This lists available format names that can be used with duck_hunt parsing.

pub fn check_format(&self, cmd: &str) -> Result<FormatMatch>

Check which format would be detected for a command. Returns the format name and source (user-defined or default).

Note: duck_hunt detects formats from content, not command names. This only checks user-defined format hints.