mocra 0.3.0

A distributed, event-driven crawling and data collection framework
# mocra

`mocra` is a Rust crawler runtime library for building module-based crawlers. A module describes what to request, how to parse responses, and how parsed output moves to the next node in a workflow.

The current runtime supports:

- single-node local execution;
- Kafka-backed queue execution;
- Raft/RocksDB-backed coordination and cache state;
- DAG and linear module workflows;
- response caching, middleware, cron scheduling, metrics, health checks, and DLQ operations.

Redis is not supported by the current codebase.

## Install

```toml
[dependencies]
mocra = "0.3.0"
```

Optional features:

```toml
mocra = { version = "0.3.0", features = ["js-v8", "polars"] }
```

The crate currently targets Rust `1.85` and edition `2024`.

## Minimal Runtime

```rust
use std::sync::Arc;

use mocra::common::state::State;
use mocra::engine::Engine;

#[tokio::main]
async fn main() -> mocra::errors::Result<()> {
    let state = Arc::new(State::new("config.toml").await);
    let engine = Engine::new(Arc::clone(&state), None).await?;

    // engine.register_module(MyModule::default_arc()).await;

    engine.start().await;
    Ok(())
}
```

## Documentation

- [Architecture]docs/en/architecture.md
- [Design]docs/en/design.md
- [Development Guide]docs/en/development.md
- [Configuration Reference]docs/en/configuration.md
- [HTTP API]docs/en/api.md
- [Operations]docs/en/operations.md

Chinese documentation:

- [架构文档]docs/zh/architecture.md
- [设计文档]docs/zh/design.md
- [开发文档]docs/zh/development.md
- [配置参考]docs/zh/configuration.md
- [HTTP API]docs/zh/api.md
- [运维文档]docs/zh/operations.md