Online discussion platform Reddit is making the most of generative AI: its deals with OpenAI and Google allow it to use user content to train its AI, bringing in millions of dollars each year, and now it plans to protect that revenue stream by banning AI data aggregators without content licensing agreements.
Reddit plans to update its own web standards to stop automated data collection: By modifying robots.txt (aka the “Robots Exclusion Protocol”), the platform will limit the number of requests a single entity can make.
Apparently, there are a few companies that are allowed to collect large amounts of data from Reddit: OpenAI and Google, both of which have annual licenses to access all of Reddit's data. No monetary figure was mentioned in the announcement of the deal with OpenAI, but it is known that Google sends $60 million a year to be allowed to use the platform's data.
Further closures
This is another move by Reddit to secure its revenue stream. The company previously had a surprisingly successful IPO, boosted by the announcement of the aforementioned deals with OpenAI and Google. Last year, the company had already hinted at further optimizing profits by charging for its API, which led to the splintering of various third-party Reddit apps, who would otherwise have had to pay millions of dollars for API calls.
Now Reddit is trying to further protect its new business model. Only well-capitalized AI companies will continue to have access to Reddit's data. At least, that's the intention. For example, controversial AI company Perplexity is said to be circumventing robots.txt to collect data, according to a Wired investigation. Data scraping can lead to skyrocketing costs to access websites. So it's no wonder Reddit wants to stop these practices. For example, CEO Steve Huffman revealed that the company's API was paid for over a year ago, before the company went public and long before it went public.
Read also: OpenAI signs new deals with media companies for content usage
