Expand description
Counting traffic: every request (fed from the access-log path, so all protocols and whether or not lines are written) lands in hourly counts per host and group, daily unique visitors, and daily top lists.
Bounded by construction: MAX_HOSTS hosts (more go to other),
hourly buckets for retention_days, top lists of TOP entries a day
(the rest added to (other)), MAX_PENDING undecided browsers per
shard, a classification cache of MAX_CACHE per shard. Unique
visitors are counted with hash(secret, day, host, ip, user-agent) —
the secret is made at start and never stored, nor are the hashes — so
nobody can be followed from one day to the next and no IP or User-Agent
is kept.
A browser isn’t a person until it loads a stylesheet, script or font as
a browser rendering the page does: its requests wait in a pending record
and are counted as a person’s then, or as “browser, no assets” (a bot)
after UNDECIDED without one — or at once after 32 requests without
one.
Constants§
- BEHAVIOUR
- The per-day lists the script feeds (
[traffic] script): seconds and page views with a report, keyedhost/path; scroll depth, screen sizes, events and event properties, keyedhost\tvalue; goal conversions (sessions that reached the goal), keyedhost\tgoal. - DIMS
- The per-day lists of people’s audience and sources, keyed
host\tvalue: countries, browsers, OSes and device classes count visitors; channels and UTM values count visits (page views arriving from outside the site). - DIM_
SAVED - Entries of each audience/source list kept on disk.
- DIM_TOP
- Entries a day’s audience/source list keeps (all hosts together).
- LIVE_
WINDOW - How far back the live view looks.
- MAX_
CACHE - Classifications cached per shard.
- MAX_
ENGAGED_ SECS - The longest time on one page counted (a tab left open).
- MAX_
HOSTS - Hosts counted separately; others are
other. - MAX_
OPEN_ SESSIONS - Sessions open at once, per shard; past it the quietest is closed.
- MAX_
PENDING - Browsers awaiting their first asset, per shard.
- MAX_
RECENT - People’s recent page views kept for it, per shard.
- MAX_
REPORTS - Reports a session may make; the rest are ignored.
- SESSION_
GAP - A person’s page views more than this apart are separate sessions (as Google Analytics counts them).
- TOP
- Entries a daily top list keeps.
- TOP_
SAVED - Entries of each top list kept on disk and reported.
- UNDECIDED
- A browser that loads no asset this long after its last request isn’t a person.