Module traffic 

Traffic insights (18.15.0): who visits — people, AI agents, bots — worked out by rws from each request, for the admin UI’s Traffic page.

This first part is the classifier, classify: from the client’s address, User-Agent and path it names what kind of client this is, using

• the IP ranges Google, Bing, Apple, DuckDuckGo, OpenAI, Perplexity and Common Crawl publish for their crawlers (a match is verified; a User-Agent that claims one of them from elsewhere is spoofed);
• ai.robots.txt’s AI agents and crawler-user-agents’ bot patterns (both MIT, bundled, regenerated by ci/gen_bot_lists.py);
• requests only scanners make (/.env, /.git/config, /wp-login.php…).

A browser User-Agent that matches none of them is a Kind::Browser: whether it’s a person is decided later, by whether that client loads the page’s assets as a browser does.

Modules

apps

Native apps and browser extensions (18.35): their requests are the site’s users, not bots, but they never load a page’s stylesheets, so the browser test can’t find them. They’re recognised here instead:

config

[traffic] settings (RWS_CONFIG_TRAFFIC_*).

device

Browser, operating system and device class of a person’s browser, from its User-Agent — and from Chromium’s default client hints (Sec-CH-UA-Platform, Sec-CH-UA-Mobile), which win when present because Chromium freezes those parts of its User-Agent. Families only; versions aren’t kept.

geo

Country of an IP address, from a table compiled into the binary (the geo feature): DB-IP’s “IP to Country Lite” (CC BY 4.0), converted by ci/gen_geo.py into geo.bin — range starts and a country each, deflated (~700 KB). It’s decoded on first use into sorted arrays (~5 MB) and looked up by binary search. [traffic] geo_db loads a newer file of the same format instead.

hosting

Cloud and server hosting networks, from a table compiled into the binary (the geo feature): DB-IP’s “IP to ASN Lite” (CC BY 4.0) reduced by ci/gen_hosting.py to the ranges of hosting providers, in geo::Table’s format with a provider code where that has a country (~60 KB). A browser User-Agent from one of these is a program — a link scanner, a preview, a headless crawler — not a person, so the Traffic page counts it as a bot (Kind::Hosting). Consumer VPN and CDN egress (Cloudflare WARP, iCloud Private Relay, VPN services) is left out: people browse from it.

owner

The site’s owner (18.35): their visits are counted apart, so the Traffic page shows everyone else. A client is the owner when

record

Counting traffic: every request (fed from the access-log path, so all protocols and whether or not lines are written) lands in hourly counts per host and group, daily unique visitors, and daily top lists.

script

The optional first-party script ([traffic] script = true, 18.31.0): rws serves /_rws/t.js itself, and the script reports to /_rws/t on the same origin what the server can’t see — time on page, scroll depth, screen size, single-page-app route changes, clicks on links to other sites (event outbound, {to: host}, 18.36.1) and custom events (rws.track("signup", {plan: "pro"})). Nothing is stored in the browser; the visitor is the same daily hash as for every request.

storemodel-postgres

Raw page views and script reports in PostgreSQL (opt-in, 18.31.0): [traffic] database_url_env names the environment variable holding a postgres:// URL. A background thread writes in batches; a full queue drops rows (counted) rather than slowing requests. With it, the admin API answers any date range and filter combination from SQL (GET /traffic/query), past retention_days.

summary

The Traffic page’s data: totals for a period and the one before it, a series (hourly up to 48 hours, daily beyond), and the top lists.

Structs

Class

A classified client.

TrafficScriptLayer

Answers /_rws/t.js and /_rws/t when [traffic] script is on (it reads the setting when built); otherwise, and for every other path, passes the request on.

Enums

Group

The three groups the Traffic page shows (and this server’s own calls).

Kind

What a client is, more precisely.

Functions

classify

What the client at ip, sending user_agent, asking for path, is.

configure_sitesNon-WebAssembly

Count the sites toml’s routes match ([[route]] match.host) on their own and list them first (record::set_configured_hosts).

enable

Start counting (the binary does this when the admin API is on).

enabled

Whether requests are being counted.

save

Save the counts now (at shutdown).

start

Start counting, with the saved counts loaded and saved again every five minutes (when persist gives a file). The rws binary calls this when the admin API is on and [traffic] enabled isn’t false.