Module ports 

Port trait definitions - service abstractions

Defines interfaces that adapters must implement:

• ScrapingService: HTTP fetchers, browser automation
• AIProvider: LLM extraction services
• CachePort: Caching abstractions
• CircuitBreaker: Resilience patterns Port trait definitions - service abstractions

Defines interfaces that adapters must implement. Following hexagonal architecture, these are the “ports” that connect domain logic to external infrastructure.

Modules

agent_source

Agent source port — use an LLM as a pipeline data source Agent source port trait for LLM-as-data-source.

auth

Auth port — runtime token loading, expiry checking, and refresh. AuthPort — runtime token loading, expiry checking, and refresh.

data_source

Database source port — query databases as pipeline data sources Database source port — query databases as pipeline data sources.

document_source

Document source port — read files from the local file system Document source port — read local files as pipeline data sources.

escalation

Tiered request escalation port — decide when to escalate from plain HTTP to browser Tiered request escalation port.

graphql_plugin

GraphQL plugin sub-module GraphQlTargetPlugin port — one implementation per GraphQL API target.

signing

Signing port — attach HMAC, OAuth 1.0a, AWS Sig V4, or Frida RPC signatures to requests. SigningPort — request signing abstraction.

storage

Storage port — persist and retrieve pipeline results Storage port — persist and retrieve pipeline results.

stream_source

Stream source port — consume SSE, WebSocket, or message queue feeds Stream source port trait for event-driven data sources.

wasm_plugin

WASM plugin port — dynamic plugin loading WASM plugin port — dynamic plugin system

webhook

Webhook trigger port — accept inbound HTTP webhooks that start pipelines Webhook trigger port — accept inbound HTTP requests that start pipelines.

work_queue

Work queue port — distributed task execution Work queue port — distributed task execution interface

Structs

GraphQlAuth

Authentication strategy for a GraphQL request.

ProviderCapabilities

Provider capability flags

RateLimitConfig

Rate limit configuration

ServiceInput

Input to a scraping service

ServiceOutput

Output from a scraping service

Enums

CircuitState

Circuit breaker state

GraphQlAuthKind

Discriminant for GraphQlAuth

Traits

AIProvider

AI Provider port for LLM-based extraction

CachePort

Cache port for storing/retrieving data with idempotency key support

CircuitBreaker

Circuit breaker port for resilience

RateLimiter

Rate limiter port

ScrapingService

Primary port: ScrapingService trait