As Mythos raises stakes, security experts gather in DC to discuss how to secure AI systems

Welcome to Eye on AI with AI reporter Sharon Goldman. In this episode: Top Republicans urge party to avoid $300 million AI lobby… AI model fraud is scary good… Anthropic’s new AI model sets off global alarm.

This week, as Anthropic Mythos generated a new wave of headlines highlighting both its advanced capabilities and the vulnerability of such systems to abuse, I headed into a conference room outside Washington, DC. It brought together a multidisciplinary group of AI security experts, standard setters, and policy experts to understand what AI security actually looks like.

Outside of the industry, acronyms like SANS, NIST, CoSAI, and OWASP may not mean much. Inside Security helps set the rules that organizations around the world follow. But now we’re having a hard time upholding these rules.

I was invited to join the discussion as organizations race to bring AI to everything, including the handing over of sensitive data and critical workflows. This is despite the fact that those same systems are becoming more attractive targets for adversaries.

The session was led by Rob van der Veer, chief AI officer at the software platform Software Improvement Group and founder of the security community OWASP’s AI Exchange. Systems like Mythos speed the discovery of vulnerabilities and shift the balance toward attackers, he said.

“These studies show that weaknesses in AI systems can be discovered faster and at scale, often before developers even notice,” he said. “This shifts the balance towards the attacker and reduces the margin for error.” So far, concerns about Mythos have focused primarily on how good it and similar models are at finding so-called “zero-day” vulnerabilities in traditional software, but they could also find vulnerabilities in the AI models and systems that companies are deploying across their organizations.

The problem is that most organizations are unprepared to address most of the AI security concerns that have already been identified and those that will emerge in the future. There is a growing need for practical guidance on how to identify and respond to AI-specific threats. However, the field remains fragmented, with overlapping frameworks, competing recommendations, and little agreement on where to start.

How to ensure the safety of AI systems has not yet been established.

Even the basics have not been established yet. What does it mean to measure whether an AI system is secure? How should it differ between use cases, infrastructure, third-party tools and underlying models? Should guidance focus on capabilities or outcomes?

Gary McGraw, co-founder of the Berryville Institute of Machine Learning, pointed to a core gap. Benchmarks today tend to measure how well an AI system can perform a security task, rather than how secure the system itself is. Companies should keep this distinction in mind when evaluating tools and defenses.

Back in 2019, McGraw warned that securing machine learning systems would be “one of the defining factors in the cybersecurity battle over the next decade.” That moment has now arrived.

“These conferences are a way to remind people of the basics as we try to define what machine learning security actually is,” he said.

Another significant concern is that there are no finite guardrails that are universally robust against adversarial prompts, said Apostol Vasilev, supervisor of a research team working on AI security at the National Institute of Standards and Technology (NIST), part of the U.S. Department of Commerce. “This means that the security of AI systems is not a static problem, a once-and-done problem,” he said. Unlike many vulnerabilities in traditional software that can be patched, AI security requires a more dynamic approach. This means continually updating guardrails to address known exploits, conducting internal red teams to discover new adversarial prompts, patching defenses before attackers strike, and prioritizing resiliency so enterprises can limit the impact of inevitable exploits and recover quickly.

“The ultimate goal is to reach an equilibrium where it is difficult and costly for attackers to find new exploits,” he added. “But that will only happen if companies invest in adopting and sustaining this dynamic posture.”

Similar to moving to security protection software

Still, many conference attendees were optimistic that the industry would catch up. McGraw pointed out that security has gone through transitions like this before, including the software boom of the mid-’90s. “When software took over the world, we didn’t have to panic,” he said. “I remember when banks realized, ‘Oh my god, we’re a software company.'”

He warned that in moments like these, the stories told by companies like Anthropic and OpenAI can outpace reality. “Guards love a good story about a pile of broken items going up in flames and the fire department coming to the rescue,” he says. “I remain optimistic that we are always making progress toward better security engineering. We can take what we learn and apply it to machine learning.”

That’s why this type of meeting on industry alignment is important, Van der Wiel said. “Aligning standards and guidance across initiatives will reduce fragmentation, improve clarity and give practitioners a consistent path forward,” he explained. “This allows organizations to act quickly without losing control.”

So, here’s more AI news for you.

sharon goldman
sharon.goldman@fortune.com
@SharonGoldman

The fate of AI

The AI boom is single-handedly driving the U.S. import market, increasing the trade deficit by $200 billion, Fed study finds – Tristan Bovet

European AI unicorn run by baker’s son – he learned the basics of business by watching his father make bread rolls – Written by Kamal Ahmed

Cursor’s 25-year-old CEO is a former Google intern who just signed a $60 billion deal with SpaceX. – Written by Marco Quiroz Gutierrez

Meta begins tracking employee screens and keystrokes to train AI tools – Written by Eva Roitberg

Investors continue to punish ServiceNow despite CEO McDermott’s predictions of strong earnings and rapid growth in sales of AI products — by Jeremy Kahn

AI in news

The White House memo focuses on protecting U.S. AI from “distillation” attacks. In a new White House memo, Science and Technology Policy Director Michael Kratsios said the United States has evidence that foreign companies, primarily from China, are conducting an industrial-scale distillation campaign to “steal American AI.” The memo emphasizes maintaining U.S. leadership in AI, protecting critical infrastructure and supply chains from foreign adversaries, and expanding collaboration across government and with allies, while ensuring systems are “safe, secure, and reliable” and aligned with democratic values. It also emphasizes the growing role of the private sector in developing AI, calls for more talent and computing infrastructure, and warns that if we don’t act quickly we could be left behind by rivals, defining AI as a geopolitical competition rather than a mere technological transformation.

AI chipmaker Cerebras has filed to go public after canceling its IPO plans last year. CNBC reported. cerebral system has filed to list on the Nasdaq (ticker: CBRS) and reports a sharp rebound in 2025 revenue of $510 million (up 76% year over year) and net profit of $87.9 million, while signaling a major strategic shift from selling chips to operating as a cloud service, exposing it to more direct competition from hyperscalers. The application also highlights the significant concentration of customers in UAE-related institutions, including: Mohammed bin Zayed University of Artificial Intelligence It accounts for the majority of revenue. At the heart of the company’s growth plans are more than $20 billion in computing contracts. OpenAIis backed by a $1 billion loan and equity warrants, but it relies heavily on Cerebras delivering large-scale infrastructure on time, leaving OpenAI with the option of exiting if it doesn’t.

Top Republican urges party to avoid $300 million AI lobbying effort. of financial times Political backlash against pro-AI money is growing in the US election, as Josh Hawley urges Republicans to reject funding from well-funded AI super PACs backed by figures associated with investors such as OpenAI, Palantir Technologies and Andreessen Horowitz. With war chests reportedly running into the hundreds of millions, critics including Bernie Sanders and Alexandria Ocasio-Cortez have warned that the AI industry’s influence is dampening debate and stifling regulation ahead of the midterm elections. Hawley argued that unchecked AI poses risks ranging from harm to children to financial burden, and blamed bipartisan deference to Big Tech for the bill’s stall. The move comes amid broader tensions in Washington as the White House pushes for national AI supremacy and resists state-level regulation, while lawmakers of both parties push for stronger safeguards.

Five AI models tried to trick me. Some of them were frighteningly good. This is an interesting piece from wired Reporter Will Knight provides a first-hand account of how advanced AI models are becoming incredibly effective at social engineering, and details highly personalized phishing attacks generated entirely by open-source models. DeepSeek-V3. Use the testing platform from Charlemagne InstituteKnight shows how several major models are done, including: claude 3 haiku, GPT-4oetc., can convincingly impersonate the attacker, adapt in real time, and scale the fraud even if they fail or are rejected outright. This article argues that AI’s conversational strengths, such as flattery and context awareness, make it particularly dangerous in phishing and fraud, and that large parts of the attack “kill chain” could be automated. mythology Both defensive expectations and new risks increase.

Pay attention to AI numbers

80%

According to research by Epoch AI, that’s how many U.S. adults who reported using Claude in the past week live in households with annual incomes of $100,000 or more.

In contrast, only 37% of Meta AI users have an annual income of $100,000 or more. Nationally, the survey found that about 50% of U.S. adults fall into this income bracket.

32% of Meta AI users live in households with incomes below $50,000, compared to 7% of Claude users and 24% of U.S. adults. Other large providers have a relatively narrow focus, with 56% to 64% of users in households making $100,000 or more.