Scale AI's valuation on Tuesday came after it revealed it had raised $1 billion in venture capital in a late-stage funding round led by venture capital firm Accel and backed by industry giants Nvidia, Amazon, and Meta. It soared to nearly $14 billion. a bit.
Nvidia has made a fortune selling the hardware that OpenAI, Anthropic, Meta, and others rely on, but Scale's claim to fame is providing the data needed to actually train these models. Also, as previously discussed, modern models require large amounts of data. According to Meta, 15 trillion tokens (words and punctuation fragments that make up prose and speech) were used just to train a relatively small model like Llama 3.
Founded in 2016 by Alexandr Wang, Scale describes itself as a “data foundry” that has been involved in powering “nearly every major AI model” out there. This includes his direct work with OpenAI on GPT-2 and InstructGPT, as well as several programs run by the US Department of Defense.
In addition to providing model builders with large amounts of meticulously labeled data, Scale also provides services to help partners fine-tune existing datasets.
It's no exaggeration to say that sourcing enough data to build ever more capable models has proven to be a problem. The issue is at the center of a number of lawsuits brought by artists, newspapers, photographers, and writers, alleging that OpenAI and others violate creators' copyrights by using their works to train machine learning models.
And as model builders continue to push the limits of what is possible with transformer models, it seems like the problem isn't going to get any easier.
“The laws of scaling imply that the need for data grows exponentially as models grow, raising the important question of whether we will ever run out of data,” Wang told the company on Tuesday. said in a statement.
Scales argues that if there is any hope of artificial general intelligence becoming a reality, we will need a lot of data, and it will need to be of high enough quality to actually contribute to more capable models.
Finally, Scale argues for measurement and evaluation systems to determine whether these models are reliable enough for large-scale adoption.
Scale isn't the only startup to see its valuation soar in the wake of the generative AI boom. Earlier this month, GPU-backed Bitburn CoreWeave raised $1.1 billion in new funding, boosting its valuation to $19 billion. Just a few weeks later, the data center operator revealed it had secured $7.5 billion in debt funding from Blackstone, BlackRock and others to equip more of its data centers with GPUs.
Meanwhile, less established AI startups are successfully raising seed funding. In parallel with Scale's large funding round, French AFI startup H (formerly Holistic AI) on Tuesday secured $220 million to accelerate the development of its multi-agent-based models for generative AI apps. ®