Cloudflare’s latest data provides one of the clearest snapshots yet of how AI companies are using the web and how little they’re giving back.
The company, which powers about 20% of the internet, tracks how its AI bots crawl websites and how often those platforms send users back through referrals. The resulting “crawl to browse” ratio is a simple but easy-to-understand metric: the amount of values extracted compared to the values returned.
The numbers for early April 2026 are grim. Anthropic is by far the worst, with a ratio of 8,800 to 1. This means that their bots crawl a web page 8,800 times for every referral submitted.
OpenAI follows with 993 to 1. Microsoft, Google, and DuckDuckGo look much more balanced in comparison.
Anthropic’s position is particularly notable given its reputation for being “ethical.” Its reputation has made it a preferred choice among some users who want to support more responsible AI development. This data highlights another aspect of ethics: how companies interact with the broader web ecosystem that provides information for the output of AI models.
Historically, the Internet operated on tacit transactions. Websites allowed search engines to crawl and index their content for free, and in return received traffic that could be monetized. Generative AI breaks that deal. Chatbots will increasingly provide direct answers, reducing the need for users to click through to the original source.
As a result, the system extracts more value than it returns, and in some cases increases costs for site owners due to increased bot activity.
Anthropic has previously questioned Cloudflare’s methodology, pointing to an increase in referral traffic due to new features. Still, broader trends are hard to ignore. Anthropic did not respond to a request for comment.
If the web’s economic engine relies on traffic and referrals, these ratios raise fundamental questions. What will be the motivation for sharing verified information online in the future?
Cloudflare is venturing into a new marketplace for web content. It is unclear whether such efforts will be successful. After all, what’s better than using people’s content for free?
Sign up for BI’s Tech Memo newsletter here. Please contact us by email. abarr@businessinsider.com.