Last month, researchers demonstrated that generative artificial intelligence (AI) systems could be used to autonomously design, plan, and run scientific experiments without supervision. The system allowed instructions for synthesizing chemicals to be written and executed in remote laboratories where chemicals could be synthesized without human involvement.
Separately, last year, researchers demonstrated the AI model’s ability to discover novel compounds even more toxic than VX, a nerve agent widely considered the most toxic compound ever discovered. reported. One of the researchers said in an interview that for him “the concern was how easy it would be to do it”, adding that “probably if I had a good weekend of work”, less money, more training and a little less money. He said that anyone with the know-how could adjust it. A model that produces worrying results.
According to the National Institutes of Health, research is “dual-use” if “methodologies, materials, or results could be used to cause harm.” Gain-of-function studies on human pathogens are a good example of exploitation. AI research is no different.
Concerns about misuse of AI research are not limited to the synthesis of dangerous illegal compounds such as VX. AI has already been deployed to defend against network intrusions into critical infrastructure and is also being used, albeit less effectively, to develop new techniques for compromising networks. However, as cybersecurity AI research advances, it is not political to expect the current equilibrium in favor of defensive over offensive AI tools to continue. There are also technical reasons for this. AI tools designed for network intrusions are likely to operate independently of command and control and are largely immune to current detection techniques. This equilibrium is also unstable for more mundane reasons. Billions of dollars in incentives to develop offensive AI systems.
The potential for exploitation is even greater. AI is already an integral part of civil and state surveillance regimes. Private companies use this to keep people out of their property. State actors use this to monitor, monitor and control their citizens in ways human agents alone cannot. Authoritarianism would be even more effective if it relied on the ability to detect adherence to rules. It’s completely uncontroversial to say that more effective authoritarianism is a bad result of AI.
Why is AI research so often abused? The AI lab itself suggests the answer. A stated goal of many leading AI labs is to achieve, and ideally exceed, human-level performance on a wide range of tasks. Repeated successes in achieving this goal can lead to Artificial General Intelligence (AGI). But whatever the future of AGI, intelligence is the quintessential dual-use capability, and machine intelligence is the quintessential dual-use technology.
The fact that AI research is at such pervasive risk of misuse creates a serious unresolved tension between norms of transparency and the need to mitigate the risk of harm. Transparency in scientific research is definitely valuable. Transparency drives discovery, connects disciplines, disseminates information, improves reproducibility, and rewards accuracy.
Therefore, it would be natural to assume that all AI research should be completely transparent in all respects under all circumstances. This is wrong.
Researchers have long recognized the need for broader responsibility and greater transparency to reduce the risk of harm. Other dual-use technologies, especially those used in weapons and biomedical applications like AI technologies, have existing limitations on transparency. To his credit, AI researchers are beginning to understand the dual-use nature of AI. But despite the move to responsibly withhold certain research artifacts (code, training procedures, model weights), the norms surrounding publication do not adequately reflect the trajectory of AI research. The trajectory of AI research is declining toward increasingly robust capabilities for an ever-wider range of tasks. calculate.
Researchers aren’t the only ones who can reduce the risk of research misuse. But they have a special influence on this issue. They can choose which parts of their research to publish.
Based on the premise that AI researchers are making a good faith effort to fulfill their responsibilities, we can extrapolate their existing image of those responsibilities from actual current practices. However, as we have pointed out, current practice clearly requires sufficient research transparency to allow other researchers to use the work to derive formulas for lethal compounds. reporting (or providing relatively unrestricted access to research output). This degree of moral vigilance is far too lax by any reasonable standard.
A stricter standard is much more attractive. On this moral question, we should borrow a line of thinking from the law. There are three well-known concepts in law: defaults, criteria for reassessing defaults, and procedures for doing so. For example, by default people don’t commit crimes. The criteria for re-evaluation is being charged with the crime. And the re-evaluation procedure is a criminal trial. Borrowing: By default, AI research can be abused (and therefore should not be published). The criteria for re-evaluation is the decision to be transparent about the research. The procedure is… well, whatthat’s right?
It doesn’t have to be the only right choice, but like any procedure related to the criminal justice system, the procedure should be widely credible. The easiest way to ensure that a procedure is trustworthy is to state exactly what it is and have a trusted party perform it. One way he operates this is by hiring independent reviewers (“red teams”) to assess (and certify) whether research meets the moral standards associated with publication.
For example, consider the work that led to today’s large-scale language models, the paper that put a “T” on GPT (Generative Pretrained Transformers). The work reported in this paper has enabled other scientists, including OpenAI, to train highly capable large-scale language models at scale and speed, on huge datasets. In other words, it turns out that the research could have been widely exploited in the sense that the technology enabled by this study allowed researchers to build systems that were susceptible to widespread exploitation. Therefore, today’s default should be against publication of the paper, for the reasons just suggested. Again, this is because, as I suggested, all AI research is subject to abuse and should always be banned from publication by default. This does not mean that the paper should or should not have been published. Rather, the question of whether to do so means that (reliable) procedures must be followed to ensure that it cannot be exploited.
But what about the reasons in favor of transparency? How do they weigh the reasons for stopping the publication of AI research? There are two broad categories of relevant reasons. practical and epistemological. For example, there are (practical) reasons to publish, including flexibility for investors, attracting top talent, or otherwise promoting research. I doubt anyone thinks that practical reasons like this carry enough weight to justify publishing potentially harmful results. The relative weight of epistemological considerations is more complicated. The point here is not that transparency has no epistemological benefits, such as allowing other researchers to learn from their work. Rather importantly, these epistemological gains are currently not properly weighed against the moral costs associated with predictable future abuse.
Researchers think a lot. Their laboratories could therefore consider reducing the mental burden of individual researchers by introducing automated processes that follow reliable procedures such as the competence and basic research just sketched. It is relatively inexpensive to do so. Even better, it cheaply shows that the lab takes its responsibilities seriously.
I expect to see commercial entities investing heavily in AI research pursue such a line in the coming months. Increased secrecy will distrust some, especially those interested in democratic oversight. But just as secrecy about bioweapons research is incompatible with democratic governance, opaque reporting of research is not incompatible with democratic governance. And researchers may welcome it because it reduces risk.
As the coronavirus crisis shows, we need science now more than ever.
This bulletin elevates the expert’s voice above the noise. But as an independent, not-for-profit media organization, our operations depend on the support of readers like you. Help us continue to provide quality journalism that holds leaders accountable. It is important that you support our work at all levels. In return, we promise that our reporting will be understandable, impactful, prudent, solution-oriented and fair-minded. Together, we can make a difference.
