Janet Yellen warns that AI in finance poses 'significant risks'

CNN
—

Treasury Secretary Janet Yellen is expected to warn bankers and tech executives on Thursday that while artificial intelligence could bring huge benefits to the financial system, the technology could also pose new dangers, according to excerpts of the speech first obtained by CNN.

Yellen's speech at the financial stability conference was her most extensive remarks yet on AI.

The moves come as investors rush to capitalize on the AI ​​boom, tech giants embark on an AI arms race, and regulators worry about what might happen.

“The enormous opportunities and significant risks associated with financial firms' use of AI have propelled the issue to the top of financial regulators' agendas,” Yellen is due to say in speeches to conferences at the U.S. Treasury Department on Thursday and the Brookings Institution on Friday.

On the opportunities front, Yellen is expected to note how AI is already being used to help investors with forecasting and portfolio management, and by banks to fight fraud and assist with customer service, according to prepared remarks.

Going forward, Yellen is expected to say that the “rapid evolution” of AI could pave the way for making financial services cheaper and more accessible, specifically pointing to advances in natural language processing, image recognition and generative AI.

AI chatbots like OpenAI's ChatGPT and Google's Gemini are wowing users with their abilities, with the latest AI tools being able to conjure up images and spit out song lyrics almost instantly, and some can even generate cinematic-quality videos.

A Treasury Department official told CNN that Yellen herself has experimented with AI chatbots.

“I think all of us here recognize that financial institutions' use of AI comes with risks as well as opportunities,” Yellen said in prepared remarks.

A Treasury Department official told CNN the meeting is expected to include regulators, tech executives, asset managers, insurers, academics, civil society groups, and small and large banks.

Last December, a team of top US regulators led by Ms Yellen first warned that AI posed a risk to the financial system, and the Financial Stability Oversight Council, a task force of regulators formed after the 2008 financial crisis, formally designated AI an “emerging vulnerability.”

In her speech, Yellen is expected to detail what could happen in financial markets.

For example, Yellen is likely to point out that the “complexity and opacity” of AI models could pose problems.

The problem is that many AI models operate as “black boxes,” meaning their inner workings are not understandable to outsiders.

If Wall Street firms rely on mysterious AI models, regulators will have a hard time understanding how secure those systems actually are.

Yellen is also expected to address “inadequate risk management frameworks” regarding AI risks and “the interactions that arise when many market participants rely on the same data and models.”

In other words, when many investors rely on tools to produce the same results, market positions become crowded and can exacerbate market movements, either up or down.

Similarly, Yellen is likely to say there is a “concentration” risk associated with the fact that there are only a few companies providing AI models, meaning if one goes down, many companies on Wall Street could be affected.

Another concern is that AI models can produce biased results, which can be a big problem in the real world, especially when it comes to making financial decisions like who gets a loan.

“Insufficient or inaccurate data can perpetuate or introduce new biases in financial decision-making,” Yellen is expected to say, according to prepared remarks.

One big challenge facing AI that Yellen didn't mention in her speech excerpts is so-called hallucinations: AI models have a history of falsifying facts, often in convincing ways.

These hallucinations have also gotten lawyers into trouble in the past for citing non-existent legal precedents.

Yellen is expected to emphasize that regulators will continue to monitor the impact of AI on financial stability.

Given how quickly AI is evolving, Yellen said regulators and industry could use scenario analysis to better understand “future vulnerabilities” and what they can do to “build resilience.”

A Treasury official told CNN that the FSOC is working to understand what threats AI could pose to the financial system, including by stepping up efforts to oversee how AI is already being used.

Of course, U.S. government officials themselves are also relying on AI to do their jobs.

For example, the IRS announced last September that it had begun deploying AI to detect tax evaders.

And in February, the Treasury Department revealed it had quietly deployed AI to catch bad actors stealing from taxpayers.

The Treasury plans to make greater use of AI in the future.

“We have been working with the public and private sectors to use AI to detect some of the biggest risks we face, from money laundering to terrorist financing to sanctions evasion,” Yellen is expected to say.