ai governance
free, metered, or blocked
ai agents read your website so buyers do not have to, and what they can read decides what they say about you. that leaves one decision to make, page by page: free, metered, or blocked. for nearly everything, the answer is free.
picture a head of marketing at a mid-size drugmaker who needs a new agency. they do what most people do in 2026: they open an ai agent and type "who should a company like ours be talking to for a patient-facing digital build." 30 seconds later they have a shortlist of 5. they have not opened a single agency website. one of the 5 has a lovely site, with the case studies in a pdf and the pricing in an image. the agent could not read either. that agency is not on the list, and never learns why.
the agent, the ai tool the buyer typed into (chatgpt, claude, gemini, perplexity), built that answer by reading the agencies' websites on the buyer's behalf and keeping what it could make sense of. a pricing table saved as a picture contributes nothing, because the machine cannot read the picture. a feature list buried in a pdf might make it in, or might not. marketing copy that never states a plain fact gives the machine nothing to quote. the agent still answers the buyer's question. it answers from whichever vendor was easiest to read.
how much reading the machines do
cloudflare is a company that sits between a large share of the world's websites and their visitors, passing the traffic through, so it can count who is doing the reading. in summer 2025 it did: for every 1 visitor anthropic's agent sent to a website, anthropic's machines had read roughly 71,000 of that website's pages1. hardly any of that reading ends in a visit. the answer the buyer sees is built from it.
cloudflare's toll
in july 2025 cloudflare gave the websites it fronts a switch. when a machine asks for a page, the site can hand it over free, charge the machine a price the site sets, or refuse2. a person reading the same page pays nothing, exactly as before. because cloudflare fronts so much of the web, that switch is now within reach of most companies, including yours.
we think the meter gets the web wrong, and we would not turn it on for any page that sells you. the person asking the agent is still a person looking for an answer, and charging their reader because it is a machine charges them for asking.
the pages are the pitch
for most business-to-business companies, the pages a machine wants to read are the ones you most want read: pricing, documentation, comparisons, proof. if buyers get their answers from agents, then being quoted correctly by agents is marketing. put those pages behind a meter and you have cut off the distribution you built them for. the narrow case for charging is data you built and would sell anyway: a coverage database, a priced catalogue, a research archive. for most companies that pile is empty, and that is fine.
in a regulated industry the stakes are highest where the reader is not a buyer at all. a patient who has just been handed a diagnosis does not open 10 tabs. they ask an agent, at 11 at night, frightened, and read what it says. every accurate page that sits behind a meter is a page the agent answers around, and the answer it gives instead comes from whoever left theirs open. your patient-facing content exists to reach people, so it wants every channel reading it, agents included. your medical and legal reviewers will want to have seen a claim before an agent repeats it to a patient, and they are right to: the review rules follow the content, whoever the reader is.
what to do about it
the first check takes an afternoon. ask 3 or 4 agents what a buyer would ask about your company, and read the answers next to your own website. note what they got wrong, and which page the error traces back to. a pricing image or a pdf usually turns up in the first 10 minutes.
then sort your content into the 3 piles and write the decision down. free, where being read widely is the point, which is nearly everything. metered, only for data you built and would sell anyway. blocked, where it is private. the sorting and the review behind it take longer than the afternoon, and they are the same work your reviewers already do for every other channel. put one person's name on it.
sources
- 1cloudflare radar, "the crawl before the fall… of referrals" (july 1, 2025). crawl-to-refer ratios for the week of june 19 to 26, 2025; anthropic 70,900:1, openai roughly 1,700:1. cloudflare's own caveats: the ratios aggregate training crawl with fetches made to answer a user's question, and referrals from the agents' native apps carry no referer header, so the ratios "may overstate". cloudflare both measures this traffic and sells the meter described below; we use the figures because the network position that makes the meter possible is the same one that makes the count credible.
- 2cloudflare, "introducing pay per crawl" (private beta, july 1, 2025). the http 402 mechanism; the site sets a flat per-request price and chooses allow, charge, or block per crawler. the "free, metered, or blocked" of this essay's title is that control set, applied to a company's own content. in july 2026 cloudflare extended the same rail beyond pages to data, apis and tools ("announcing the monetization gateway", <https://blog.cloudflare.com/monetization-gateway/>).