Skip to main content

Module discovery

pagesift_engine

Module discovery 

Source
Expand description

Read-only document navigation from HTTP Link fields. Candidates still pass through the normal crawler scope, robots and budget checks.

Functionsยง

header_links