(thinkhubstudio/Shutterstock)
What if one day artificial intelligence surpasses human intelligence? This “superintelligence” is what OpenAI is likely to hope for in the next decade, and the company aims to align it with humanity’s best interests. formed a new team focused on
“How can we make AI systems that are so much smarter than humans follow human intentions?” in an OpenAI blog post with Superalignment, co-led by the post’s authors Ilya Sutskever and Jan Leike. It states the question announcing the formation of a new team called.
The company said it is focusing on mitigating hyper-intelligent AI systems rather than artificial general intelligence (AGI) to “emphasize a much higher level of competence.” Sutskever and Reike say there is no current way to control superintelligent AI, and existing tuning strategies, such as reinforcement learning from human feedback, cannot be applied to systems that exceed the capabilities of humans themselves. there is
The company says it is assembling a team of top machine learning researchers and engineers to tackle the superintelligence coordination problem. “Our main basic research stake is the new Super Alignment team, but getting this right is so important to achieving our mission that many teams are looking forward to it.” from development to scale-up to deployment,” the authors write.
(Source: Open AI)
Sutskever is the co-founder and principal scientist of OpenAI. Leike leads his OpenAI alignment team, and his approach to alignment research includes training AI systems using human feedback, training AI systems to assist in human evaluation, and training AI systems to perform alignment studies. His focus was on the three pillars of systems training. Mr Reich said: Tweet Most of the previous alignment teams have joined the new Super Alignment Team.
OpenAI also plans to devote 20% of its previously secured compute to this pursuit over the next four years.somewhere else Tweet“Twenty percent of compute is no small amount,” Reich said, adding that he was “impressed by how aggressively OpenAI is willing to allocate resources at this scale.”
“This is the largest investment ever made in alignment, and probably more than all humankind has ever spent on alignment research,” Reike wrote.
The Super Alignment Team has an ambitious goal of solving the core technical challenges of Super Intelligence Alignment in four years. This blog post highlights how the team’s work revolves around making current models such as ChatGPT more secure, understanding and mitigating AI risks such as misuse, economic disruption, disinformation, stigma and discrimination, and addiction and overdependence. It outlines how it will be deployed.
The authors also say that sociotechnical issues, that is, those related to human-machine collaboration, will also be an area of focus. OpenAI says it is actively working with multidisciplinary experts to “ensure that our technical solutions consider broader human and societal concerns.”
The team outlined their first goal. It is to build a near-human level automated alignment researcher. “Then we can use massive amounts of computing to scale up our efforts and iteratively tune our superintelligence.”
To do this, researchers will need to develop scalable training methods, validate the resulting models, and stress test the entire tuning pipeline, the team writes. Stress testing involves providing training signals on tasks that are difficult for humans to evaluate so that AI systems can be used to evaluate other AI systems. It also includes automating the search and interpretation of problematic behavior.
“Finally, we can test the entire pipeline by intentionally training an untuned model and confirming that our technique detects the worst kind of maladjustment (adversarial testing).” writes the authors.
“How do you know when you’re failing or not progressing fast enough?” Eliezer Yudkowski Asked Commenting on the news, Mr. Yudkowski is a controversial AI researcher known for his view that the AI coordination problem cannot be solved.
“There’s empirical data coming in, and we’re going to look at that,” Reike replied. “We can measure progress locally on different parts of our research roadmap (e.g. scalable monitoring). will be closely monitored.”
Related
