pub struct PagesiftMeta {Show 14 fields
pub content_hash: Option<String>,
pub raw_hash: Option<String>,
pub text_hash: Option<String>,
pub redirect_chain: Vec<String>,
pub etag: Option<String>,
pub last_modified: Option<String>,
pub not_modified: bool,
pub engine: Option<String>,
pub browser_engine: Option<String>,
pub warnings: Vec<String>,
pub timings: BTreeMap<String, f64>,
pub duplicate_of: Option<String>,
pub relevance: Option<f32>,
pub query_filter: Option<QueryFilterStats>,
}Expand description
pagesift 在 metadata.pagesift 里附加的字段。
Fields§
§content_hash: Option<String>正文 Markdown、原始 HTML、规范化纯文本的 BLAKE3 哈希(小写十六进制)。
raw_hash: Option<String>§text_hash: Option<String>§redirect_chain: Vec<String>§etag: Option<String>§last_modified: Option<String>§not_modified: bool§engine: Option<String>http 或 browser:这一页实际用哪种方式抓到的。
browser_engine: Option<String>浏览器渲染时实际用的内核:lightpanda、moli 或 chrome。
warnings: Vec<String>§timings: BTreeMap<String, f64>各阶段耗时,毫秒。
duplicate_of: Option<String>整站爬取里,这一页的正文和之前抓到的某一页完全相同(同一内容的另一个 URL):那一页的 URL。
relevance: Option<f32>带查询的爬取里,这一页和查询的相关度(0 到 1)。
query_filter: Option<QueryFilterStats>queryFilter 的效果:保留了几段、原来几段,过滤前后的 token 估算。
Trait Implementations§
Source§impl Clone for PagesiftMeta
impl Clone for PagesiftMeta
Source§impl Debug for PagesiftMeta
impl Debug for PagesiftMeta
Source§impl Default for PagesiftMeta
impl Default for PagesiftMeta
Source§impl<'de> Deserialize<'de> for PagesiftMeta
impl<'de> Deserialize<'de> for PagesiftMeta
Source§fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
Deserialize this value from the given Serde deserializer. Read more
Source§impl PartialEq for PagesiftMeta
impl PartialEq for PagesiftMeta
Source§impl Serialize for PagesiftMeta
impl Serialize for PagesiftMeta
impl StructuralPartialEq for PagesiftMeta
Auto Trait Implementations§
impl Freeze for PagesiftMeta
impl RefUnwindSafe for PagesiftMeta
impl Send for PagesiftMeta
impl Sync for PagesiftMeta
impl Unpin for PagesiftMeta
impl UnsafeUnpin for PagesiftMeta
impl UnwindSafe for PagesiftMeta
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more