OpenAI said it is devoting one-fifth of its computing resources to developing machine learning techniques to thwart “cheating” in hyperintelligent systems.
Founded in 2015, the San Francisco AI startup has always had the goal of developing artificial general intelligence safely. This technology doesn’t exist yet. Experts are divided on what exactly that will look like and when it will happen.
Nonetheless, OpenAI has carved out 20 percent of its processing power to somehow prevent future generations of machines from endangering humanity, creating a new AI platform led by co-founder and principal scientist Ilya Sutskever. I’m going to start a department. This is a subject that OpenAI has covered before.
“Superintelligence could become the most impactful technology mankind has ever invented, and could help solve many of the world’s most important problems,” said the aspiring savior of mankind this week. Opined.
“However, the vast power of superintelligence can also be very dangerous, leading to the incapacitation and even extinction of humanity.”
OpenAI believes that this decade will see the development of computer systems that surpass human intelligence and overwhelm humanity. [Before or after fusion? Or quantum computing? – Ed.].
“Managing these risks will require, among other things, new institutions to solve the problem of governance and superintelligence coordination, ensuring that AI systems that are far smarter than humans follow human intentions.” What should we do about it?” added the industry.
Speaking of OpenAI…
- The Microsoft-funded startup has made the GPT-4 API publicly available to paying developers.
- Emily Bender, CompSci professor and machine learning expert, has written an essay on the real threat posed by AI models and the superhuman AI horrors that certain fields are driving.
There are already ways to calibrate, or at least try to calibrate, models to human values. These techniques may include what is called Reinforcement Learning from Human Feedback (RLHF). With this approach, you’re essentially monitoring the machine and shaping it to behave more like a human.
RLHF has helped make systems like ChatGPT less likely to generate harmful language, but it can still be biased and difficult to scale. Providing feedback on a model’s output usually requires hiring a large number of people at modest wages, but this comes with its own set of problems.
Developers, it is argued, cannot rely on a few to police technology that affects many. OpenAI’s alignment team is trying to solve this problem by building a “near-human level automated alignment researcher.” OpenAI wants to build AI systems that can align other machines to human values without explicitly relying on humans, instead of humans.
It seems to us that artificial intelligence will train artificial intelligence to be closer to non-artificial intelligence. It’s a bit like chicken and egg.
If AI drives humanity to extinction, it’s our fault
read more
Such systems can, for example, search for problematic behavior and provide feedback or take other steps to correct it. To test the performance of its system, OpenAI said it can intentionally train an untuned model to see how well the tuned AI removes bad behavior. The new team set a goal of solving a linear problem in four years.
“While this is an incredibly ambitious goal and there is no guarantee of success, we are optimistic that a focused and coordinated effort can solve this problem. There are many new ideas, more indicators to help us progress, and today’s models can be used to study many of these issues empirically,” the researchers concluded.
“Solving the problem involves providing evidence and arguments that convince the machine learning and safety community that the problem has been solved. , we hope our findings will allow us and the community to plan accordingly.”
We will start building bunkers. ®
