Why the IAEA model is not optimal for regulating artificial intelligence

OpenAI, the company behind Chat GPT and a clear advocate for strong regulation in the field of artificial intelligence, recently suggested that AI regulation may need an International Atomic Energy Agency-like model. Did.

At first glance, the IAEA may look like a reasonable model for AI regulation. The IAEA safeguards system has evolved over time to provide confidence that safeguarded materials will not be diverted to weapons-related end uses and that safeguarded nuclear facilities cannot be used for weapons development. However, the nuclear governance model is not really suitable for regulating artificial general intelligence.

Some might argue that both nuclear weapons and artificial intelligence have the potential to destroy the world, but for AI the face and course of this catastrophe is less clear than for nuclear technology. While many have focused on the idea that AI could somehow develop or hijack military capabilities, including nuclear weapons (such as Skynet), there is still no credible path for AI to destroy the world. Not established. Work to identify such pathways is ongoing and should continue.

Nonetheless, given that the urgency of dealing with pressing global threats has driven the evolution of non-proliferation and safeguards, lessons from the nuclear sector suggest that countries should collectively take steps to address it. Before we can act, we need agreement on the credibility and definition of the global challenge. .

Evolution of Nuclear Regulation. The IAEA’s current mission—to support the use of nuclear technology for peaceful purposes, to develop safety and security standards, and to ensure that states adhere to their commitments not to build nuclear weapons—will evolve. It took decades. Early efforts to control the atom through international controls (especially the United Nations Atomic Energy Commission) failed, and repeated crises resulted in the ad hoc development of international nuclear governance. This ad-hoc model of nuclear governance reflects the failure of national cooperation in the 1940s and the launch of the United States’ Nuclear for Peace initiative in the 1950s when, as a result of the Cold War, no relevant effective control system existed. can be traced directly to Geopolitics. The result was a system that needed to be patched repeatedly as gaps and obstacles were exposed through real proliferation incident cases.

This cooperation-gap-proliferation-lessons-cooperation (etc.) cycle is a perennial hallmark of the nuclear age and one that must be avoided to effectively regulate AI.

Clearly there are many important differences between the field of nuclear research and the field of AI. Among them is the fact that nuclear matter and technology have a physical form, whereas artificial intelligence is digital and largely intangible in nature. There is another fundamental difference that makes the problem of collective action even more difficult to address. In the nuclear field, the main actors are usually governments. This was certainly true in the early days of the nuclear age, when early efforts were made to devise control systems. Only governments could develop nuclear weapons, and the role of the commercial sector was limited throughout most of the atomic age.

AI can have a transformative impact on governments, the military, and the information sector. However, in AI, it is clear that private companies currently outperform governments in most areas of machine intelligence, and may continue to do so. This private sector focus creates a strong private sector lobbying force that was largely absent in the early nuclear age. Efforts to build corporate partnerships to address collective action issues in other areas, such as the environmental field and the so-called tragedy of the commons, have met with limited success.

It is also not clear whether AI can be protected in the same way as nuclear material. Nuclear safeguards are primarily focused on securing physical nuclear material (that is, fissile material). You can’t make nuclear weapons without nuclear material. This is the starting point for the IAEA to receive declarations of nuclear material from States and then seek verification through material accounting, inspections, trade analysis and open source information to confirm the accuracy and completeness of State compliance.

In the AI ​​space, the key elements of production are the training data, the computing power to train the AI ​​model, the trained model (usually a separate file containing the training weights (parameters) and other algorithmic data). computer files you have), and services provided. While the United States has moved to restrict exports of computing power to China, especially given that AI is intangible, any of the elements used to develop AI technology could be as dangerous as nuclear material. It is not clear whether it is “safely protectable”. -physical) character. This is not to say that export controls or other measures should not be used to limit the export of his military-related AI capabilities. In fact, due to the nature of export controls, many specific uses of military-related AI may already be restricted.

But in reality, there is nothing to stop anyone trying to train a model, be it a state, a company, or even a non-state actor. This possibility only increases as models and model weights are leaked or released on an open source basis and high-performance computing capabilities become more prevalent. This requires very high computing power, so while it may be possible for the time being to limit and monitor the ability to train large-scale language models with high performance, the practicality of such an approach is uncertain. decreases over time.

The IAEA’s Peaceful Uses and Safeguards mandate emerged first and is generally considered to be the IAEA’s primary mandate. However, it is also worth considering the IAEA’s missions related to safety and security that have expanded and emerged over time. Here, the IAEA acts as a standards coordinating body and independent reviewer. The IAEA conducts the International Personal Protection Advisory Service (IPPAS)) The mission will assess the regulatory readiness of countries preparing to embark on nuclear programs. It is perhaps this common standard setting and independent third-party evaluation that is most relevant when considering lessons for the AI ​​field. This model of governance is not unique to the IAEA. In fact, this is such a popular model of international governance that the question arises as to why the IAEA is a suitable role model, rather than the Financial Action Task Force, which implements similar mechanisms in the financial sector.

Identify existential threats that AI may pose. We can expect AI leaders to be proactive in identifying and mitigating the risks posed by technology. Given the significant downside potential of both nuclear technology and AI, it is right to examine the nuclear sector to identify lessons for the AI ​​sector. However, the lessons learned from the nuclear sector review are not all (or perhaps generally) positive. The challenge is that the specific threat path of AI, or how exactly it can pose an existential threat, is not clear, making it a sharp distinction between AI and nuclear weapons.

Even with a common view of risk, it took decades to build an effective control system for nuclear power. If AI poses a threat to humanity’s future, we cannot afford to follow a similar approach to devising controls over AI. This reasoning reaches one clear conclusion. Identifying the pathways through which AI could threaten humanity requires a thorough focus. Only once specific pathways have been identified can a control approach be planned to identify the role of organizations such as the International Atomic Energy Agency.