Module adapters 

Adapter implementations - infrastructure concerns

Concrete implementations of port traits:

• HTTP client with anti-bot features
• AI providers (Claude, GPT, Gemini, Ollama)
• Storage backends (file, S3, database)
• Cache backends (memory, Redis, file) Adapter implementations - infrastructure concerns

Concrete implementations of port traits:

• HTTP client with anti-bot features
• AI providers (Claude, GPT, Gemini, Ollama, Copilot)
• Storage backends
• Cache backends

Modules

agent_source

LLM agent source adapter — wraps AIProvider as a pipeline node Agent source adapter — wraps an [AIProvider] as a pipeline data source.

ai

AI provider adapters AI provider adapters

browserbrowser

JavaScript rendering adapter (headless browser via stygian-browser) JavaScript rendering adapter using stygian-browser

cache

Cache adapters (memory, Redis) Cache adapters

csv_source

CSV/TSV data source adapter CSV / TSV [DataSourcePort] and [ScrapingService] adapter.

distributed

Distributed work queue and executor adapters Distributed execution adapters

document

File system / document source adapter File system document source adapter.

graphql

GraphQL API adapter — generic ScrapingService for any GraphQL endpoint GraphQL API adapter — a generic ScrapingService for any spec-compliant GraphQL endpoint.

graphql_plugins

GraphQL target plugin implementations (one file per API target) GraphQL target plugin implementations.

graphql_rate_limit

Request-count rate limiter for GraphQL API targets with pluggable algorithms.

graphql_throttle

Proactive cost-throttle management for GraphQL APIs Proactive GraphQL cost-throttle management.

http

HTTP scraping adapter with anti-bot capabilities HTTP scraping adapter with anti-bot features

mock_ai

Mock AI provider for testing Mock AI provider adapter for testing

multimodal

Multi-modal content extraction (CSV, JSON, XML, images, PDFs) Multi-modal content extraction adapter

noop

No-op service for testing No-op service adapter for testing

openapi

OpenAPI 3.x introspection adapter — resolves operations from an OpenAPI spec and delegates to RestApiAdapter OpenAPI 3.x introspection adapter.

openapi_gen

OpenAPI spec generator from API discovery reports OpenAPI 3.0 spec generator from API discovery reports.

output_format

Output format helpers — CSV, JSONL, JSON Output format helpers — CSV, JSONL, JSON.

resilience

Resilience adapters (circuit breaker, retry) Resilience adapters

rest_api

REST API adapter — JSON APIs with auth, pagination, and data extraction REST API scraping adapter with authentication and pagination support.

rss_feed

RSS/Atom feed source adapter RSS / Atom feed [ScrapingService] adapter

signing

Request signing adapters — Noop passthrough and HTTP sidecar bridge. Covers Frida RPC, AWS Sig V4, OAuth 1.0a, custom HMAC, and device attestation. Request signing adapters.

sitemap

Sitemap / sitemap-index source adapter Sitemap / sitemap-index [ScrapingService] adapter

storage

Storage adapters (file, S3, database) Storage adapters — persist and retrieve pipeline StorageRecords.

stream

Server-Sent Events stream source adapter Server-Sent Events (SSE) stream adapter.

wasm_plugin

WASM plugin adapter (feature = “wasm-plugins”) WASM plugin adapter

websocket

WebSocket stream source adapter WebSocket stream source adapter.