ChatGPT's developer, OpenAI, is working on a new approach to artificial intelligence models in a project codenamed “Strawberry,” according to a person familiar with the matter. Reuters.
Details of the project have not been previously reported, but it comes as Microsoft-backed startups race to demonstrate that the types of models they offer can provide advanced inference capabilities.
A team inside OpenAI is working on Strawberry, a copy of an internal OpenAI document recently revealed. Reuters In May. Reuters While the exact date of the document detailing how Open AI plans to use Strawberry in its research could not be confirmed, a source said of the plans: Reuters As an ongoing project, the news agency was unable to confirm whether Strawberry will be available to the public anytime soon.
According to people familiar with the matter, how Strawberry works is a closely guarded secret even within OpenAI.
According to the sources, the document describes a project to use the Strawberry model to enable the company's AI not only to generate answers to queries, but also to plan ahead so that it can navigate the internet autonomously and reliably to carry out what OpenAI calls “deep research.”
That's something that AI models have not been able to achieve so far, according to interviews with more than a dozen AI researchers.
Asked about Strawberry and the details reported in the article, an OpenAI spokesperson said in a statement: “We want our AI models to see and understand the world in the same way that humans do. Continuous research into new AI capabilities is common practice in the industry, and there is a shared belief that these systems will improve in their reasoning capabilities over time.”
The spokesman did not directly respond to questions about Strawberry.
The Strawberry Project was formerly known as Q*. Reuters This initiative, which was reported last year, was already seen as a groundbreaking achievement within the company.
Two sources said they saw what they were told by OpenAI staff earlier this year was the Q*Demo, which is said to be able to answer difficult scientific and mathematical problems that cannot be solved by current commercially available models.
OpenAI has internally tested an AI that scored above 90% on the MATH dataset, which is the benchmark for math championship problems, according to a second source briefed on the matter. Reuters It was not possible to determine if this was a “Strawberry” project.
On Tuesday, OpenAI said in an all-hands meeting that it showed a demo of a research project that it claims has new human-like reasoning skills. BloombergAn OpenAI spokesperson confirmed the meeting took place but declined to provide details. Reuters It was not possible to determine if the project being demoed was Strawberry.
OpenAI hopes that the innovation will dramatically improve the inference capabilities of AI models, according to the people, adding that Strawberry employs a special method of processing AI models that have been pre-trained on large datasets.
researcher Reuters Those interviewed said that reasoning is the key to AI achieving human or even superhuman levels of intelligence.
Large-scale language models can already summarize dense text and construct elegant prose much faster than humans, but the technology often falls short when it comes to common-sense problems that humans seem to intuitively solve, like recognizing logical errors or playing tic-tac-toe. When the models encounter these problems, they often “hallucinate” false information.
Interview with an AI researcher Reuters It is generally agreed that reasoning in the context of AI involves forming models that enable AI to make plans, reflect the functioning of the physical world, and reliably solve difficult, multi-step problems.
Improving the inference capabilities of AI models is seen as key to enabling them to do everything from making important scientific discoveries to planning and building new software applications.
OpenAI CEO Sam Altman said earlier this year that “the most important area of advancement” in AI will be in reasoning capabilities.
Other companies such as Google, Meta, and Microsoft are also experimenting with different techniques to improve the inference of AI models, as are most academic labs that conduct AI research. However, researchers are divided on whether large language models (LLMs) can incorporate ideas and long-term plans into predictions. For example, Yann LeCun, who works at Meta and is one of the pioneers of modern AI, frequently states that LLMs are not capable of human-like reasoning.
The AI Challenge
Strawberry is a key component of OpenAI's plan to overcome these challenges, according to people familiar with the matter. Reuters We explained what Strawberry accomplishes, but we didn't explain how.
The company has privately told developers and outside parties in recent months that it is on the verge of releasing technology with far more advanced inference capabilities, according to four people who were briefed on the company and declined to be identified because they were not authorized to discuss private matters.
One of the sources said Strawberry involves a special technique called “post-training” of OpenAI's generative AI models, which involves adapting the base model to hone its performance in specific ways after it has been “trained” on large amounts of generalized data.
The post-training stage of model development includes methods such as “fine-tuning,” a process used in nearly all language models today, which can take many forms, including humans giving the model feedback based on its responses, or feeding the model examples of good and bad answers.
Strawberry is similar to a technique called “self-taught reasoner,” or STaR, developed at Stanford University in 2022, one of the sources familiar with the matter said. STaR allows AI models to “bootstrap” themselves to higher levels of intelligence by iteratively creating their own training data, and could theoretically be used to help language models surpass human-level intelligence, said Noah Goodman, a Stanford professor who is one of its developers. Reuters.
“I think that's exciting and frightening at the same time… If things continue to move in that direction, we as humanity have some serious things to think about,” said Goodman, who is not affiliated with OpenAI and is not familiar with Strawberry.
The document says that one of the features OpenAI is looking for from Strawberry is the ability to perform long-term tasks (LHT), which refers to complex tasks that require the model to plan ahead and execute a series of actions over an extended period of time, the first source explained.
To do this, the company creates, trains and evaluates models based on what it calls “deep research” datasets, according to internal OpenAI documents. Reuters It was not possible to determine what was included in that dataset or how long an extended period of time might mean.
According to the documents and one of the sources, OpenAI specifically wants its models to use these capabilities to conduct research by autonomously browsing the web, with the assistance of a “CUA,” or computer-assisted agent, that can act on the findings. OpenAI also plans to test its ability to do the work of software and machine learning engineers.
