Expand description
Data structures for representing HTTP requests in spider-lib.
This module defines the Request struct, which is a central component
for constructing and managing outgoing HTTP requests within the
spider-lib framework. It encapsulates all necessary details of an
HTTP request, including:
- The target URL and HTTP method.
- Request headers and an optional request body (supporting JSON, form data, or raw bytes).
- Metadata for tracking retry attempts or other custom information.
Additionally, the module provides methods for building requests, incrementing retry counters, and generating unique fingerprints for request deduplication and caching.