# ethl
**WIP**
`ethl` is a powerful Ethereum log event ETL (Extract, Transform, Load) tool designed for efficient and reliable log processing. It provides:
- RPC utilities for streaming logs with support for multiple providers, fallback mechanisms, and retries.
- Tools to archive events into an opinionated Arrow + Parquet format for optimized storage and querying.
- Features enabling fast replay of specific events and continuous indexing for real-time use cases.
This tool was built to address the need for high-performance event processing and storage in Ethereum-based applications.
### Getting Started
To get started with `ethl`, add it to your `Cargo.toml` via `cargo install ethl`:
```toml
[dependencies]
ethl = "0.1"
```
Then, include it in your project:
```rust
use ethl;
```
For detailed examples and usage, check the [documentation](https://docs.rs/ethl).
### Development nodes (anvil, hardhat, ganache)
`ethl` treats RPC provider responses as authoritative. Local development nodes
expose two failure modes that can silently corrupt an event cache:
1. **Silent empty responses past tip.** Some nodes return `{"result": []}` for
`eth_getLogs` on block ranges past their actual tip rather than erroring.
`ethl` guards against this by calling `eth_blockNumber` whenever a batch
returns no logs and refusing to advance the cursor when `to_block` exceeds
the reported tip (yields `RpcError::CursorPastTip`). The overhead is one
extra RPC per empty batch; batches that return logs are unaffected.
2. **Synthetic blocks from local mining.** Dev nodes mine new blocks in
response to transactions you send, including simulations. Those blocks
share a number with real chain blocks but contain only your local
transactions — they are well-formed blocks as far as the API is concerned,
so `ethl` cannot distinguish them from fork-proxied real blocks. If you
index against a node that has been used for writes, **treat the cache as
poisoned**: roll `block_height` back to a value from before the node was
first used and rebuild from a real RPC.