Ilya Satskeva says we need a new learning paradigm and are already pursuing one

The SSI founder and former OpenAI principal scientist sees AI development at a tipping point. Fundamental research is once again needed instead of increasingly large-scale models. Models, like humans, need to learn more efficiently. He has ideas about how this could be possible, but says we now live in a world where we can’t talk freely about such things.

In a wide-ranging interview with Dwarkesh Patel, Ilya Sutskever, co-founder of Safe Superintelligence Inc. (SSI), describes the fundamental changes in AI development. While 2020-2025 was the “era of scaling”, where progress was achieved primarily through increased computing and data, the industry is now returning to the “era of research”.

“The scale sucked all the air out of the room,” Sutskever said. The recipe of pumping more data and computation into neural networks worked so reliably that every company did the same. However, pre-training data is finite. Scaling is still happening, and simply increasing resources no longer guarantees a qualitative leap, especially in reinforcement learning (RL).

According to Sutskever, we are back to a point similar to pre-2020, where we need new ideas and paradigms, not just large clusters.

AI models suffer from “jaggies”

The central problem with the current model, Sutskever said, is its inconsistency, or “jaggedness.” Models may perform well on difficult benchmarks, but often fail on basic tasks. He cites “vibe coding” as an example. The model recognizes a bug and when fixing it introduces a new bug, but the next attempt to fix it only reinstates the old bug.

Sutskever suspects that reinforcement learning (RL) training makes the model “a little too single-minded.” Unlike pre-training, which simply uses “all data”, you have to choose the RL. This leads researchers to optimize their models for specific benchmarks, often unintentionally (“reward hacking”), compromising their ability to generalize in the real world.

Human emotions as biological “value functions”

To reach the next level of intelligence, AI systems will need to learn how to generalize as efficiently as humans. Teens learn to drive in about 10 hours, which is just a fraction of the data the AI ​​needs.

Sutskever theorizes that human emotions play an important role here, as a kind of robust “value function.” These biologically anchored assessments help humans make decisions and learn from experience long before external outcomes (such as classical RL) are available.

“Perhaps this suggests that human value functions are regulated by emotions in some important way that is hard-coded by evolution,” Sutskever says.

AGI is the wrong goal – superintelligence will be created at work

Sutskever also fundamentally questions the established term AGI. The success of pre-training has created a false expectation that AI should be able to do everything out of the box (“general purpose AI”). But this is over the mark. “Humans are not AGI,” Sutskever says. Humans lack a huge amount of prior knowledge. Instead, it relies on continuous learning.

His vision of superintelligence is therefore more like a highly gifted student than an omniscient database. “I’m raising a 15-year-old who is very intelligent and wants to go,” Sutskever explains. This system knows almost nothing at first, but it can be learned in any profession. Actual abilities appear only after release. Rather than being generated from a training cluster as a finished product, a model must go through a “trial and error” stage in real-world deployment to fully develop its functionality.

Mr. Sutskever will not speak freely about his ideas.

When asked how training would need to change conceptually to achieve this human learning efficiency, Sutskever remained tight-lipped. He has “many opinions” on the issue, but the days of open exchange are over.

“Unfortunately, we live in a world where not all machine learning ideas are freely discussed, and this is one of them,” Sutskever explains.

The mere existence of man is evidence to him that there must be a way. But he acknowledges there may be potential obstacles. That means human neurons may “do more calculations than we think.” If this plays an important role, implementation may be more difficult.

In any case, Sutskever hints that this points to the existence of certain “machine learning principles” that he has a theory about. However, “Due to the circumstances, it is difficult to discuss the matter in detail.”

SSI wants to opt out of commercial rat race

With his new company, SSI, Sutskever is pursuing an approach that aims to deliberately set itself apart from current market trends. The company, which has raised $3 billion, has no immediate plans for a product release.

Sutskever argues that although SSI has less computing power than big tech companies, it can use that computing more efficiently for pure research because its resources are not tied up in commercial products or running large inference clusters.

This plan is a “direct attack” on superintelligence. They want to avoid the “rat race” of bringing products to market and instead research systems in secret until they are safe and mature. However, he acknowledged that a gradual release may be necessary to demonstrate the power of AI to society and begin the regulatory process.

Adjustment to intelligent life

Regarding the safety (coordination) of future superintelligence, Sutskever proposes new goals. That is, AI should be tailored to take into account “intelligent life.” Since AI itself is sentient, it is more natural to instill empathy for other sentient beings than to program it with abstract human values.

Sutskever also predicts that as AI systems become significantly more powerful, the safety strategies of major AI laboratories will become consolidated. As risks become more tangible, current competition will be replaced by pragmatic cooperation. The first signs of this are early collaborations between competitors such as OpenAI and Anthropic.