Crate codoseo_diff 

Change detection between two crawls of the same site.

Pages are matched by the hash of their URL, recomputed here rather than trusted from the stored url_hash, and only pages on the snapshot’s origin count. When the origin itself changed, the earlier URLs are moved onto the new origin first, so a move shows up as one SiteMoved and not as every page removed and added.

Functions

diff

Compares two crawls and returns the changes, sorted by severity, then kind, then URL. Changes to pages in key_pages (and to the origin page) are one severity step higher.

key_pages

The pages whose changes matter more: the origin page, the 20 pages with the most inlinks (ties broken by URL) and the starred ones.

robots_fingerprint

Hash of the rules in a robots.txt body. Comments, blank lines and Sitemap: lines are dropped and field names are lower-cased, so edits that don’t change what bots may do don’t change the fingerprint. Values keep their case, since paths are case-sensitive.