Harvey burns more AI tokens. Trillions more.
CEO Winston Weinberg said on Tuesday’s Sourcery with Molly O’Shea podcast that the legal AI startup used 1 trillion tokens in January. Harvey’s AI products are used by law firms and corporate legal teams to help draft, review, and analyze documents.
Since then, the startup’s monthly token usage has increased approximately 12 times.
Weinberg said the company is on pace to use 12 trillion to 13 trillion tokens a month. A Harvey spokesperson confirmed to Business Insider that this estimate refers to May.
“The way it’s used is going crazy,” Weinberg said.
A token is a chunk of text that an AI model processes when a user enters a prompt and receives an answer. It has also become a key metric for measuring AI usage and claims. In general, the more complex the AI task or the more powerful the AI model processing the request, the more tokens will be written.
Harvey’s CEO’s comments come as several tech companies are paying close attention to the cost of using AI. Business Insider recently reported that some startups are retreating from “tokenmaxxing” after receiving high AI bills.
To name a few examples of token revaluation, Coinbase is routing some prompts to cheaper models and Uber executives are questioning whether spending on AI coding tools is leading to enough measurable product benefits.
Weinberg said the next challenge for companies is determining the appropriate level of AI usage for each job.
“We don’t want Frontier Intelligence to do every single task. It’s too expensive,” Weinberg said in a statement to Business Insider. For example, he writes, “A management change review guarantees that, but a first-pass document summary does not.”
In his podcast, Weinberg likened the upcoming debate over AI spending to how law firms justify what they charge their clients. He called it the “billable hour problem.”
“When you get an invoice from a law firm, it lists what people did in six-minute increments, and then it lists the hourly rate,” Weinberg said. “Why did they do that? It’s because they’re trying to show ROI.”
As companies increase spending on tokens, he said, there will be increasing pressure to explain what that spending actually accomplishes.
“This is kind of the main question that I think the whole world is going to face, which is, ‘We just spent $1 billion on a token, where is the ROI?’