Struct DownloadedDocument 

pub struct DownloadedDocument {
    pub url: String,
    pub mime_type: Cow<'static, str>,
    pub content: Vec<u8>,
    pub size: usize,
    pub filename: Option<Box<str>>,
    pub content_hash: Box<str>,
    pub headers: HashMap<Box<str>, Box<str>>,
}

A downloaded non-HTML document (PDF, DOCX, image, code file, etc.).

When the crawler encounters non-HTML content and download_documents is enabled, it downloads the raw bytes and populates this struct instead of skipping the resource.

Fields

url: String

The URL the document was fetched from.

mime_type: Cow<'static, str>

The MIME type from the Content-Type header.

content: Vec<u8>

Raw document bytes. Skipped during JSON serialization.

size: usize

Size of the document in bytes.

filename: Option<Box<str>>

Filename extracted from Content-Disposition or URL path.

content_hash: Box<str>

SHA-256 hex digest of the content.

headers: HashMap<Box<str>, Box<str>>

Selected response headers.

Trait Implementations

impl Clone for DownloadedDocument

fn clone(&self) -> DownloadedDocument

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for DownloadedDocument

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for DownloadedDocument

fn default() -> DownloadedDocument

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for DownloadedDocument

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for DownloadedDocument

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.