Cloudflare and GoDaddy are working together to address a growing problem at the heart of the internet: the disruption of the web’s traditional business models in the age of AI.
The companies’ new partnership aims to address this imbalance.
GoDaddy integrates Cloudflare’s AI crawl controls into its web hosting sites, giving site owners control over how AI bots interact with their content. You may grant access to AI bot crawlers, block them, or charge access fees. These bots are run by giant AI companies and scour the web, siphoning content for free to power chatbot answers and other AI model outputs.
For decades, websites have relied on simple systems. Search engines index content, users click links, and publishers make money through advertising, subscriptions, and sales. However, this model is breaking down as AI response engines increasingly deliver information directly to users without sending it back to the original source.
This shift is creating tension between content creators and AI companies. Automated crawlers run by Anthropic, OpenAI, Google, and other AI companies currently scan vast swaths of the web and produce output, but they return far less traffic than traditional search engines. For many publishers and small businesses, this means less revenue even though their content is widely used.
Cloudflare manages and secures approximately 20% of the web’s traffic. GoDaddy, on the other hand, is the world’s largest domain name registrar, helping millions of people name and manage their websites. Together, these companies have the power to change the way business is done online.
The companies also support new standards such as agent name services and web bot authentication to verify the identity of AI agents and make their operations more transparent.
The broader goal is to create an AI-driven permission-based system for the web. Instead of bots scraping content at will, creators have greater control, visibility, and a way to participate in the value created by AI systems.
This builds on Cloudflare’s previous moves to block AI crawlers by default and introduce tools that allow publishers to monetize access to their data.
The stakes are high. If AI systems continue to extract value without sending traffic back, the incentive to create original content can disappear.
Cloudflare and GoDaddy are betting that new infrastructure focused on identity, control, and compensation will help rebuild sustainable economic models for the next phase of the internet.
Sign up for BI’s Tech Memo newsletter here. Please contact us by email. abarr@businessinsider.com.