Skip to main content

Crate crawlkit_extensions

Crate crawlkit_extensions 

Source
Expand description

§crawlkit-extensions

crawlkit 官方第三方服务扩展集,提供开箱即用的云爬虫后端实现。

§支持的服务

服务Feature免费额度说明
Jina Readerjina(默认)1000 次/月网页转 Markdown
Firecrawlfirecrawl500 次/月JS 渲染 + 结构化提取
Lightpandacdp免费开源轻量级 CDP 无头浏览器
Obscuracdp免费开源Rust 编写,内置反检测

§快速开始

[dependencies]
crawlkit-extensions = "0.2"
use crawlkit_extensions::jina::JinaClient;

let client = JinaClient::builder()
    .with_token("your-api-token")
    .build();

Re-exports§

pub use jina::JinaClient;
pub use jina::JinaFormat;
pub use firecrawl::FirecrawlClient;
pub use cdp::CdpClient;

Modules§

cdp
CDP (Chrome DevTools Protocol) 客户端
firecrawl
Firecrawl API 客户端
jina
Jina Reader API 客户端