Problem:
Most analytics still count people and pageviews. Teams cannot see which agents visited, what they read, or whether the business appears in answer engines. A spike looks like a good week. A drop looks like a channel died. In both cases the traffic may have been training crawlers, shopping agents, or a swarm on a form, and the dashboard had no name for any of it.
Cloudflare and others have already said the obvious out loud: automated traffic has overtaken people on large parts of the network. The essay Most internet traffic is no longer human is the public argument. The private version is a founder staring at GA and making product decisions from a human funnel that is no longer the majority of requests.
Without agent-aware measurement, AEO is guesswork and bot detection is superstition. You cannot tell whether a rewrite made the service page more readable to a model, or whether a challenge removed the junk and also removed a crawler you wanted. Defense and discovery both need a log that distinguishes classes of client.
Approach:
Carbonology measures agent traffic alongside discovery and defense: which agents hit the site, which pages they consume, and where the business shows up in answers. We start from the logs and headers you already have — CDN, host, application — and from a short list of questions the team actually needs answered. Vanity bot charts are easy. Decisions are harder.
The useful picture is usually three layers. Acquisition: which known agents and crawlers request the site, and how often. Consumption: which URLs they fetch, and whether those are the pages you wrote for them. Presence: whether you appear in the answer engines you care about for the queries that matter, checked by hand at first rather than by a fake rank-tracker.
We will not invent a precision the logs cannot support. Many clients send a generic user-agent. Some hide behind shared clouds. The work is to classify what can be classified, to keep an unknown bucket honest, and to tie the numbers back to the pages and the forms. Contact us if you want that picture before you spend another quarter arguing from a people-only chart.
Deliverables:
- A measurement plan: the questions, the sources (CDN, app, search consoles, manual answer checks), and what we will not claim.
- Agent and crawler classification on the existing traffic, with a documented unknown bucket.
- URL-level views of what automated clients actually read, compared to the pages you consider canonical.
- A lightweight presence check for the answer engines and prompts you name — qualitative first, repeated on a schedule you can afford.
- Instrumentation notes so the next deploy does not wipe the only useful headers.
- A readout the rest of the team can use next to AEO and defense work, not a dashboard that only we understand.
Who it's for:
Teams doing AEO who need to know whether agents can fetch the new pages. Teams doing bot defense who need to know what they blocked. Operators who suspect the analytics story is about people who are no longer most of the traffic.
If you have no site traffic yet, measurement can wait. If you have traffic and no names for it, this is the service.
Related:
Questions:
No. It sits beside the people funnel. You still want to know what humans did. You also want to know what agents did, because those requests now move the site and, increasingly, the market.
Not as a stable number, and anyone who sells you one is overselling. We can check named prompts on a schedule and record whether you were cited, paraphrased, or absent. That is a presence log, not a rank.
It helps. Application logs can be enough for a first pass. The more of the request we can see, the less we have to guess. We will work with what you can share.
Send the site, the questions you cannot answer today, and whether you have CDN or host logs. Contact is enough.