Generative artificial intelligence tools like ChatGPT have spread like wildfire, largely because of their superior capabilities, but also because they are free or almost free to use.
But just because a service doesn’t charge users doesn’t mean it costs nothing. In practice, building and maintaining sophisticated large-scale language models is costly. AI companies will eventually need to recoup their investment somehow.
Marketplace’s Megan McCarty Carino spoke with Will Oremus, a tech news analyst writer for The Washington Post, about the high cost of AI chatbots. Oremus recently took a closer look at how the economics of AI development can affect the evolution of the technology.
Below is an edited transcript of their conversation.
Will Oremus: So there is a huge initial cost when training a model, but in the long run that cost is really just the cost of running the model, i.e. every time you use the chatbot or call the chatbot. It looks small compared to the computing cost of . of these models. And it only runs on certain high-end chips called [graphics processing units]. A set of GPUs capable of running AI applications can cost tens of thousands of dollars. Not just chatbots. So Microsoft has introduced his GPT-type software into everything from Microsoft Excel, Word, PowerPoint, Bing, Skype, so the spread of these massive language models across all kinds of applications , the amount of computing will only increase more and more. price.
Megan McCarty Carino: Well, in the case of ChatGPT, it is said to be one of the fastest-adopted technologies in history. in a few months, [it] It has 100 million users and people use it for all sorts of silly things, probably not even considering the cost of having ChatGPT write a poem about beef stroganoff or something like that.
Oremus: right. And the companies that make these models advise us not to think about cost. They don’t want to talk about it. So I tried OpenAI, I tried Microsoft, I tried Google. I tried Anthropic, which creates a large language model called Claude. No one ever talked about the cost of running these models. It’s a very sensitive subject for them. [OpenAI CEO] Sam Altman said late last year that computing costs for a single conversation with ChatGPT are in the single-digit cents. If it’s just me, it doesn’t seem like much. So you are talking to ChatGPT. Free for me, but OpenAI costs a few cents. Well, OpenAI reportedly reached 100 million users as soon as he was in February. Probably far beyond that now. And then a few cents will start to add up for every conversation between 100 million users, and you’ll start the conversation with real money.
McCarty Carino: right. I am really shocked by your report. Because a lot of the conversations about this technology and the trajectory it takes, I felt that these models will continue to get better and better and will be used more widely in all areas. Given these different use cases, this calculation may not really take into account the actual costs of such development and deployment, if at all possible.
Oremus: That is correct. So the fact that these companies are losing money every time they use AI models is not just about companies. This is for everyone who wants to rely on this technology, and for all that sectors of the economy will rely heavily on AI and there will be an AI revolution that will replace all these forms of human labor. For forecasting, it is a problem. Well, right now, the AI revolution is heavily subsidized by companies vying for market share. Ultimately, like any business, you have to make a profit. So I have to raise the price, [or] They will likely find cheaper, lighter models that are less functional. Alternatively, ads will appear on free products that we are more likely to see. This is not what Sam Altman said he wants to do with his ChatGPT. However, when asked about it at a recent Senate hearing, he did not rule out advertising on ChatGPT because he could not afford to keep losing so much money.
McCarty Carino: And, of course, advertising was the primary model for subsidizing all Internet services available to consumers for free, such as search and social media. I mean, how well does that model work here?
Oremus: Most analysts I spoke to believe that advertising alone is not enough, especially for the higher-end versions of this AI technology that professional applications actually require. Now, the likes of ChatGPT may continue to offer ad-supported free versions. However, a source I spoke with said they don’t think ChatGPT, which is available for free, is as good as the model offered to businesses. And that’s because you can’t afford to put the best and biggest models into a free product in the long run. So one of the interesting things from the consumer side is that usually when a new technology like ChatGPT comes out, people use it and the revenue is reinvested as companies gain market share. to expect. The product keeps getting better. But if it loses, if the free product loses, it’s not actually investing resources in making ChatGPT better and better, but making it cheaper and cheaper so that it doesn’t lose a lot of money. We are investing our resources into And it turns out that GPT-4, a new model in OpenAI’s large-scale language model, is actually significantly better at not fabricating falsehoods than “hallucinating” in industry terms. It’s pretty good at that. But the free version of ChatGPT doesn’t let you do that. I’m not going to get their best product for free.
McCarty Carino: How will all of this affect the AI business? For example, who can play in the sandbox?
Oremus: As a matter of fact, there are only a limited number of companies that can do that. So if you’re a startup and you say, “I’m going to compete with OpenAI and Google and try to build another great language model and AI chatbot,” investors will ask how you do it. . Can you afford to lose as much money as these giants for as long as possible to compete? It means that we are competing for model tweaks. So they’re looking at ways to take his LaMDA and PaLM models from GPT and Google and refine them for specific applications. But the underlying technology will remain one of the giants.
McCarty Carino: And if there is a strong incentive to recover these costs, what does this kind of high cost mean for open source, more open AI models?
Oremus: Now, the silver lining here is that for most of the history of developing these large-scale language models, this has really been an academic endeavor. Rather than developing products that people use, these tech giants are competing to publish papers, set new benchmarks, and build the most powerful models that can be said to have pushed the boundaries. I was. of what is possible. Because it was really all about publishing, we didn’t have to worry about a billion people using the product, so we didn’t even have to think about the cost. One is the push towards an open source model. So if you can get an open-source model and bootstrap it, you might be able to train it on the output of GPT-4 or train it on the output of Google’s model. You can get something that’s not as good, but you can get something that’s probably half or two thirds better for a much cheaper price. So we’re starting to see a move to make these models cheaper and lighter. That includes promoting the use of open source.
McCarty Carino: So what are your thoughts on how big a slowdown this could be to the most optimistic predictions about AI expansion?
Oremus: A lot depends on how well the companies find ways to downsize these models without sacrificing too much quality. I think the biggest impact will be the fact that they have to focus on that now instead of completely focusing on dealing with the problem of misinformation, right? Let’s deal with the problem that it makes up things about people and sometimes undermines information about people. Sure they’re focused on those things, but they need to focus more on how to make it cheaper. So the biggest short-term impact is that we don’t get the kind of quality advancements that we’d expect as companies spend their energy on making these things more affordable instead. I really think it is.
The high demand is one reason why the powerful graphics processing units required to train and run AI chatbots are so expensive.
And it’s basically designed by one company, Nvidia, and manufactured by one manufacturer, Taiwan Semiconductor Manufacturing Co.
In May, we spoke with historian and author Chris Miller about how the AI boom could lead to another shortage of chips.
OpenAI CEO Sam Altman also confirmed that when he recently appeared in Congress. He said he’s really happy that people are using his ChatGPT less because they “don’t have enough GPUs.”
Of course, Nvidia, which sells these GPUs, is doing well. The company’s stock has soared 163% this year alone and is now worth almost $1 trillion.
Will Oremus and I discussed how these AI systems not only incur high economic costs, but also environmental costs.
We explored this a little more closely with Sasha Luccioni, head of climate change at AI company Hugging Face. She points out that the process of training an early version of ChatGPT produced roughly the same amount of carbon dioxide as a gasoline car driven over a million miles.
T.
