8 min
Zach Chmael
In This Article
AI agents increasingly shape which companies buyers consider. See the five conditions brands need to meet and the evidence system behind them.
Updated
TL;DR
🤖 GPTBot made 569 million fetches in one month of Vercel's data, while Claude made 370 million. Machines are already major readers of the web.
🚪 GPTBot spent 34.82% of its fetches on 404s. A brand cannot enter an answer if its evidence cannot be reached.
🔎 In one analysis of 500+ SearchGPT citations across 100 queries, more than 87% matched Bing's top organic results. Retrieval remains a gate.
📎 A vendor study of 548,534 retrieved pages across 15,000 prompts reported that only about 15% were cited. Being found is not the same as being used.
🧭 Brands need 5 separate conditions: readiness, recognition, attribution, consideration, and category fit. No single AI visibility score can tell you whether all 5 are healthy.
The Agentic Web Is Rewriting How Buyers Choose
A buyer used to meet your company through a search result, an ad, a referral, or a sales conversation. An AI system can now meet your company first, assemble the evidence, compare you with alternatives, and present a shortlist before the buyer visits your website.
That is the agentic web in practical terms.
It is not simply more people using chatbots. It is a change in who performs the early work of discovery and evaluation.
The shift does not make brand, content, or websites irrelevant. It raises the standard.
Your company has to be reachable by machines, understandable as an entity, supported by current evidence, relevant to the buyer's task, and recognizable beside the right alternatives. A polished homepage cannot compensate for a missing proof point. Perfect crawler access cannot compensate for bad fit.
Cloudflare made this distinction unusually clear in its August 2026 AEO launch. It separated Agent Readiness from AI visibility and reported mentions, citations, competitive share, and industry fit as different observations. That separation is more useful than another promise to "rank in ChatGPT," because each condition points to a different business problem.
What changes when an agent researches before the buyer does?
The first comparison can happen without a browser session on your site. A person states a need, constraints, budget, or category. An AI system expands the task, retrieves sources, compares candidates, and compresses the result into an answer.
The old path often looked like this:
search → click → browse → compare → contact
The emerging path can look like this:
ask → retrieve → evaluate → shortlist → verify → contact or act
The difference is not cosmetic. In the old path, a buyer might tolerate vague copy long enough to explore the site.
In the new path, a system may need to extract a specific answer immediately: who the product serves, what it does, which constraints it satisfies, how it differs, and what proof supports the claim.
This does not mean every purchase is delegated to an autonomous agent. Most important decisions will still involve people, trust, politics, and judgment.
The immediate change is earlier and simpler: machines increasingly mediate what evidence a person sees and which companies enter consideration.
That makes "visibility" too broad to guide action.
A company can be mentioned but not cited, cited but not recommended, recommended for the wrong audience, or retrieved as a source without being treated as a candidate. Those outcomes are not interchangeable.
Why does the agentic web matter now?
The volume of automated reading is already material. Vercel and MERJ observed 4.5 billion Googlebot fetches in one month, alongside 569 million from GPTBot, 370 million from Claude, and 24.4 million from PerplexityBot.
Cloudflare measured GPTBot growing from 2.2% to 7.7% of AI-and-search crawler traffic from May 2024 to May 2025, with a reported 305% increase in raw requests. Among AI-only crawlers in May 2025, its analysis assigned 30% to GPTBot, 21% to ClaudeBot, and 19% to Meta-ExternalAgent.
Those are infrastructure observations, not proof of buyer attention. A request does not prove indexing, retrieval, citation, recommendation, a human visit, or revenue. But the requests show that machines are already collecting the material from which later answers may be built.
Source: Cloudflare.
Cloudflare's broader 2026 report classified 52% of crawler requests by purpose as AI training in June 2026, compared with 22% in spring 2025. Mixed-use crawlers represented more than 36% of the classified activity. Cloudflare also reports that its network sits in front of more than 20% of the web, which makes the dataset large and still not a census of the internet.
The sensible conclusion is not "optimize everything for bots." It is that machine participation has become large enough to require an explicit operating policy and a better evidence system.
What five conditions help a brand thrive on the agentic web?
A brand needs to pass five different gates. Each gate can fail while the others appear healthy.
Condition | The question | What failure looks like |
|---|---|---|
Readiness | Can an agent reach and parse the evidence? | blocked crawlers, blank rendered text, dead URLs, missing index coverage |
Recognition | Does the system know and name the company? | the category answer omits the brand entirely |
Attribution | Does it cite the company's owned or third-party evidence? | the brand is named with no current source, or its page is used without visible credit |
Consideration | Is it compared or recommended for the buyer's task? | the brand supplies information but never enters the candidate set |
Category fit | Is it understood beside the correct alternatives? | wrong category, wrong audience, or irrelevant competitor set |
Readiness comes first, but it is not the finish line. In Vercel's data, GPTBot fetched JavaScript files in 11.50% of requests, and Claude did so in 23.84%, yet neither executed that JavaScript.
GPTBot also spent 14.36% of fetches following redirects. If a critical answer exists only after browser-side rendering or behind a broken path, the rest of the system never gets a fair test.
Source: Vercel.
Recognition asks whether the brand exists in the system's market model. A company may have technically sound pages and still be absent from answers about its category. That can reflect weak public evidence, inconsistent identity, thin third-party presence, a narrow prompt set, or legitimate bad fit. Absence is an observation, not a diagnosis.
Attribution asks whether the system can point to evidence. A brand can be mentioned from model memory without a current citation. A page can also support an answer while the brand behind it remains unnamed.
Both matter, but they mean different things. One suggests awareness without verifiable support. The other suggests authority without buyer-visible recognition.
Consideration is the commercial threshold. A company can be a cited information source and still never appear as a potential vendor. The buyer's task, constraints, proof requirements, and comparison set decide whether the brand advances.
Category fit asks whether the system understands the company correctly. A high mention rate in the wrong category is not success. The useful question is whether the model places the company beside the right alternatives for the right buyer job.
Cloudflare calls a related measure Industry Fit. Trovance treats that inference as a hypothesis a human should confirm, not a machine-generated fact to accept automatically.
How should a company prepare without writing bot bait?
The best agent-readable content usually looks like better evidence for humans. It answers the question directly, names the entity clearly, supports claims, states limitations, and gives the reader a way to verify what is true.
A practical preparation sequence is:
Choose consequential buyer questions. Test the questions that influence discovery, comparison, validation, and selection. A random list of prompts creates activity, not market understanding.
Preserve the actual answers and sources. Record the model, date, mode, prompt, citations, mentions, competitor set, and failures. A screenshot without conditions is a souvenir.
Check access before rewriting. Confirm that critical pages return useful text, resolve correctly, appear in the relevant index, and expose current evidence.
Diagnose the gap. Decide whether the problem is identity, fit, proof, comparison clarity, third-party credibility, retrieval, or a technical failure.
Choose the smallest justified intervention. The correct response may be a proof page, comparison, technical fix, third-party validation, factual correction, monitoring decision, or no action.
Publish with provenance. Keep claims tied to sources, dates, owners, and limits. Agents need evidence they can retrieve; humans need evidence they can trust.
Verify and re-observe. Confirm that the external change landed, then rerun a comparable question after an honest observation window.
Research on generative-engine optimization supports caution. The KDD 2024 GEO benchmark used 10,000 queries across nine datasets and 25 domains, with five samples and five seeds. Quotations and statistics improved the reported visibility metrics in that setup, but the effects varied by domain and condition.
The appendix also corrected one popular talking point. Keyword stuffing scored 19.8 against a 19.8 baseline on the paper's position-adjusted word-count metric when uncertainty was reported. That supports "no measurable benefit," not a universal penalty. The later C-SEO Bench found many optimization methods weak or negative in its tested recommendation settings and found larger effects from retrieval-side relevance.
The durable practice is not a bag of ranking tricks. It is better retrieval, clearer evidence, honest comparison, and repeatable measurement.
What should marketers measure instead of one visibility score?
Measure the transitions that affect a real decision. A single score may summarize activity, but it cannot tell you which gate failed or what to do next.
For each important buyer-question family, preserve:
execution coverage: which engines, modes, and prompts actually ran;
readiness: fetch, rendering, indexing, and error state where observable;
recognition: named, unnamed, or absent;
attribution: cited owned source, cited third party, or no observable citation;
consideration: compared, recommended, shortlisted, rejected, or unknown;
category fit: correct peer set, questionable peer set, or human-disputed inference;
downstream evidence: qualified visits, conversations, pipeline, and revenue with separate attribution rules.
One SearchGPT analysis found that 87%+ of citations matched Bing's top organic results, while 56% matched Google, across just 100 queries. The result is directionally useful and too small to become a universal ranking rule.
A separate AirOps analysis reported that ChatGPT cited about 15% of 548,534 retrieved pages across 15,000 prompts. That is a vendor dataset, but the gate it illustrates is sound.
Retrieved does not mean cited. Cited does not mean recommended. Recommended does not mean selected. Selected does not mean revenue.
At Trovance, we keep those states separate because collapsing them creates confident answers to the wrong question. We also preserve unknowns. If a system does not expose whether it retrieved a page, the field should say unknown rather than convert a hidden process into a guessed zero.
How does Trovance turn agentic-web evidence into action?
Trovance observes how AI systems explain, cite, compare, and recommend a company, diagnoses the evidence gap behind the result, and helps the team produce and publish the proof-backed asset that should exist.
That starts with the raw material: the buyer question, answer, citations, competitors, engine context, and variance. It then asks whether the company is inaccessible, unrecognized, mentioned without evidence, cited but not considered, compared in the wrong category, or correctly excluded because the fit is poor.
A score alone cannot make that decision. The diagnosis determines the next action. Trovance is designed to connect the observation to a governed intervention, preserve the claims and sources behind the work, verify that the external change landed, and later compare what changed without pretending correlation is causation.
We do not believe every gap needs another article. Sometimes the right response is a clearer product fact, stronger third-party proof, a technical retrieval fix, a comparison page, a monitoring decision, or no action at all. The goal is accurate market understanding and better evidence when the fit is real.
Cloudflare's AEO launch is a useful sign that readiness, mentions, citations, and category perception are becoming normal business measurements. Trovance focuses on the work after that observation: why the condition matters, what should be done, what evidence permits the action, and whether the result can be verified.
See how AI systems currently explain your company and which evidence may be missing.
FAQs
What is the agentic web?
The agentic web is the part of the internet where AI systems do more than return links. They retrieve information, synthesize answers, compare options, recommend candidates, and may eventually complete bounded actions for people. Human judgment still matters, but more discovery and evaluation can happen before a buyer visits a company's website.
How is the agentic web different from AI search?
AI search focuses on answering a question from retrieved sources. The agentic web extends that pattern into multi-step work: clarifying a task, applying constraints, comparing candidates, checking evidence, forming a shortlist, and acting through tools. AI search is one input to that process rather than the entire buyer journey.
Does my company need a separate website for AI agents?
Usually no. Start by making the existing site accessible, clear, current, and evidence-backed. Important claims should appear in semantic text, with stable URLs and supporting sources. Machine-readable versions, APIs, or agent protocols may help certain tasks, but duplicating the site can create factual drift and governance problems.
What content helps a brand on the agentic web?
Useful content answers specific buyer questions and supports the answer with checkable evidence. Product specifications, comparisons, methodology, pricing context, case evidence, FAQs, and third-party validation often matter because they help a system determine fit. Structure helps retrieval, but unsupported claims remain unsupported after they are neatly formatted.
Can AEO or GEO control AI recommendations?
No. Companies can improve access, identity clarity, evidence quality, and relevance, then measure how answers change. They cannot control every model, retrieval index, prompt, competitor, or buyer constraint. Any promise that a brand will always be cited, recommended, shortlisted, or converted ignores variables the vendor does not control.
Should we optimize for citations or recommendations?
Track both, but decide based on the buyer job. Citation shows that a source contributed evidence. Recommendation shows that the company entered consideration for a task.
A brand may want authority, recognition, qualified shortlists, or all three. The required evidence and intervention differ for each outcome.
What is the first agentic-web action a small team should take?
Choose three to five consequential buyer questions and run them repeatedly across the AI systems your customers use. Preserve the answers, citations, competitors, and failures. Then classify each result by readiness, recognition, attribution, consideration, and category fit before producing any new content or technical work.
Related Resources
Ready to see the evidence behind how AI systems describe your company? Start with Trovance.
