Anthropic urges industry adjustment to ‘pause’ AI development

KELVIN CHAN and MATT O’BRIEN, AP Business Writers

Anthropic is proposing that the world’s top artificial intelligence companies come up with a collaborative way to pause the development of advanced AI systems, warning that technology is advancing so quickly that humans risk losing control.

The company behind the Claude chatbot said in a blog post Thursday that as cutting-edge AI performs tasks faster and faster, “having the option to slow or temporarily pause” its development “would be good for the world.”

Anthropic said its in-house research institute will work with others to investigate the issue and “take action” to help create a system for reliable slowdowns or suspensions, but was less specific.

Humanity rival OpenAI argued for a different approach in a report published Wednesday, saying that “democratic governments, rather than private companies acting alone, must ultimately decide the rules, safeguards, and accountability mechanisms.”

“Our view is that decisions about the pace of AI innovation should not be left to any particular institute, company, or special interest group,” the report said.

Anthropic said in its post that its AI models are getting faster and rapidly increasing the speed at which they can perform software tasks like coding on their own. Based on current trends, with sufficient computing power, AI systems could potentially design and develop their own successor systems, in what is called “recursive self-improvement.”

Anthropic said self-building AI would be a major technological milestone that would benefit science, medicine and other fields, but “it could also increase the risk that humans will lose control of AI systems.”

Some tech industry insiders have long warned about this scenario.

Anthropic’s post comes after another alert was issued this week by a team of researchers at the University of Toronto, which showed how AI tools can be used to create new types of AI “worms” and how they can adapt their hacking strategies as they spread from device to device and take over vast computing networks.

“I think it’s really important for people to understand that it’s not just the biggest and most powerful language models that pose security concerns,” lead researcher Nicolas Papenot said in an interview.

Anthropic contributors Jack Clark, the company’s co-founder, and Marina Favaro, the institute’s director, said they would use the hiatus to allow “social structure and coordination research” to catch up with advances in AI. Alignment is an industry abbreviation for ensuring that technology is aligned with human values ​​and intentions.

The proposed adjustment would allow advanced AI laboratories to verify that global rivals are actually stopping or slowing down their work, and that “malicious actors cannot take advantage of the adjusted slowdowns to advance covertly.”

The company said a globally coordinated mechanism was needed. Without that, a slowdown in AI development could catch up the “least cautious” players and increase pressure on companies and governments to make tough choices about AI safety.

Anthropic’s post comes as the company and ChatGPT maker OpenAI race to sell their shares on the stock market in an IPO that could value Anthropic close to $1 trillion.

Papernot notified Canadian cybersecurity authorities before releasing the report. The report shows how researchers developed the worm in the lab using “open source” AI tools that software developers can access and modify at low cost.