Crate essence 

Modules

api

cache

Smart caching layer with content, robots.txt, and redirect caching

config

crawler

engines

error

format

mcp

MCP (Model Context Protocol) server implementation for Essence.

rate_limit

search

types

utils

validation