Expand description
Priority URL queue with depth and scope filtering.
Binary heap-based priority queue with deduplication, domain tracking, and configurable scope control (allowed/blocked domains and paths).
Structsยง
- Priority
- Priority score for a URL entry (lower = higher priority).
- Queue
Entry - A URL entry in the crawl queue.
- Scope
Config - Scope control for URL filtering.
- UrlQueue
- Priority URL queue with deduplication and scope control.