Skip to main content

Module session

Module session 

Source
Expand description

Named HTTP session management for the crawler.

The SessionManager lets a spider register multiple HTTP backends under string identifiers. Each Request carries an optional sid (session ID) that tells the engine which backend to use for that particular fetch. This is how you mix plain HTTP fetchers with cookie-persisting sessions, or route certain domains through different proxy configurations.

A session is either a stateless Fetcher or a stateful FetcherSession (which automatically carries cookies and custom headers across requests). The spider configures its sessions in Spider::configure_sessions.

use scrapling_spider::session::{Session, SessionManager};
use scrapling_fetch::Fetcher;

let mut manager = SessionManager::new();
manager.add("default", Session::Fetcher(Fetcher::new()), true).unwrap();

Structs§

SessionManager
Manages named sessions and dispatches fetch requests to the appropriate one.

Enums§

Session
A session backend that can be either a stateless fetcher or a stateful session.