OpenAI forms new team to bring ‘hyper-intelligent’ AI under control

Image credit: Bryce Durbin/TechCrunch

OpenAI is forming a new team led by Ilya Sutskever, one of the company’s co-founders and chief scientist, to develop ways to operate and control “hyper-intelligent” AI systems.

In a blog post published today, Sutskever and Jan Leike, lead of the coordination team at OpenAI, predict that AI with super-human intelligence could emerge within a decade. . This AI isn’t necessarily well-intentioned, assuming it actually does end up, Sutskever and Reich say, and research is needed on how to control and limit it.

“At present, there are no solutions for piloting or controlling potentially superintelligent AI and preventing AI misbehavior,” they write. “Current techniques for tuning AI, such as reinforcement learning from human feedback, rely on a human’s ability to oversee her AI. It cannot be monitored with certainty.”

To push the needle forward in the field of “superintelligence alignment,” OpenAI is forming a new Superalignment team led by both Sutskever and Leike, which will contribute 20% of the computing the company has secured to date. will be able to access The team, which includes scientists and engineers from OpenAI’s former coordinating arm, as well as researchers from other organizations within the company, aims to solve the core technical challenge of controlling hyperintelligent AI over the next four years. increase.

how? By building what Sutskever and Reike describe as “human-level automated alignment researchers.” The high-level goal is to train his AI system using human feedback, train an AI to help evaluate other her AI systems, and eventually build an AI that can perform alignment surveys. is to (“Tune research” here refers to ensuring that the AI ​​system achieves the desired outcome or stays off the rails.)

OpenAI’s hypothesis is that AI can advance alignment studies faster and better than humans.

“As progress is made in this regard, AI systems will increasingly take over alignment tasks, and may eventually be able to devise, implement, research and develop alignment techniques that are better than they are today,” Reich said. and colleagues John Shulman and Jeffrey Wu hypothesized in their paper. previous blog post. “They will work with humans to enable their successors to work better with humans . There will be an increasing focus on reviewing published alignment studies.”

Of course, there is no foolproof method. And Leike, Schulman, and Wu, in a post where he acknowledges OpenAI’s many limitations. Using AI in assessments can exacerbate discrepancies, biases, or vulnerabilities in that AI, they say. And it may turn out that the hardest part of the alignment problem has nothing to do with engineering.

But Sutskever and Raike think it’s worth a try.

“Coordination of superintelligence is fundamentally a machine learning problem, and we believe that good machine learning experts are essential to solving this problem, even if they are not yet working on it. ,” they wrote. “We plan to share the results of this effort broadly and believe that contributing to the integrity and safety of non-OpenAI models is an important part of our work.”