spider-lib-0.4.0
- spider-lib 0.4.0
- Docs.rs crate page
- MIT
- Links
- Homepage
- Repository
- crates.io
- Source
- Owners
- mzyui
- Dependencies
- anyhow ^1.0 normal
  async-trait ^0.1 normal
  bincode ^1.3 normal optional
  bytes ^1.0 normal
  csv ^1.1 normal optional
  dashmap ^5.4 normal
  dirs ^5.0 normal optional
  futures-util ^0.3 normal
  governor ^0.3 normal
  hex ^0.4 normal
  http ^1.4.0 normal
  kanal ^0.1.1 normal
  linkify ^0.10.0 normal
  moka ^0.12 normal
  num_cpus ^1.16 normal
  psl ^2.1.183 normal
  rand ^0.8 normal
  reqwest ^0.12 normal
  rmp-serde ^1.3.1 normal
  robotstxt ^0.3.0 normal optional
  rusqlite ^0.31 normal optional
  scraper ^0.19 normal
  seahash ^4.1.0 normal
  serde ^1.0 normal
  serde_json ^1.0 normal
  serde_with ^3.8.3 normal
  sha2 ^0.10 normal
  spider-macro ^0.1.0 normal
  thiserror ^1.0 normal
  tokio ^1.0 normal
  tracing ^0.1 normal
  tracing-subscriber ^0.3 normal
  ua_generator ^0.5.46 normal
  url ^2.0 normal
- Versions
- 79.12% of the crate is documented
Go to latest version
Platform
- x86_64-unknown-linux-gnu
Feature flags

docs.rs

Rust

Skip to main content

normalize_origin

spider_lib0.4.0

In spider_lib::utils

spider_lib::utils

Function normalize_origin

pub fn normalize_origin(request: &Request) -> String

Expand description

Normalizes the origin of a request’s URL.