More well-known companies are turning to agent-based artificial intelligence (AI) to create software, rather than having human developers code the software by hand. For example, Zendesk CTO Adrian McDermott recently told Computer Weekly that his company now has more than a dozen teams using purely agent-coding techniques.
However, these widely used AI models are trained on vast datasets of varying quality, which can produce compelling but fundamentally flawed responses. Therefore, it is not surprising that code generation models frequently emit insecure code because they emulate human-written software that often does not reflect security best practices.
We sat down with Pieter Danhieux, CEO and co-founder of Secure Code Warrior, to discuss the security risks of AI-generated code, the need for governance, and how the industry can defend against looming automated threats.
Editor’s note: This interview has been edited for clarity and brevity.
How can you ensure the quality of code generated using AI tools?
Peter Danhue: I don’t think AI can currently generate secure, high-quality code. The code is fast to build, but the quality of that code is not quite there yet in my opinion. There are some facts to prove it. Secure Code Warrior will soon publish a paper co-authored with the University of Melbourne. In this paper, we benchmarked the Frontier model’s ability to generate secure code in 16 different languages.
What became clear was that none of them wrote perfect code. There are some big differences. Each model has advantages and disadvantages. Some models are better in Java and worse in C++, and vice versa. This also applies to different versions of the same model. It can be very confusing for developers to choose a model and know whether they are getting secure code or not.
Our plan is to monitor the situation and benchmark all models that are released. Because I want to know when safe code is generated. For now, we have to rely on humans and the methods we already have to generate secure code.
Does this pose any special problems for citizen developers or vibe coding?
Dunhue:That’s a problem for anyone building software, whether they’re vibe coders or professional developers. Some of these models produce secure code in some languages and others don’t, but no one ever says, “Be careful because you’re using Java, and the model you chose isn’t suitable for producing secure code in that language.” That’s one of the things we plan to build into the product.
The only thing that will change with AI is that you will now be paying Anthropic to generate code, find vulnerabilities, and patch them for you.
Pieter Danhieux, Secure Code Warrior
If the code created by citizen developers using AI tools like Claude or Gemini is insecure, there are opportunities to deploy it in a way that is safe and fits into the broader architecture. I hope someone develops technology that can enhance hardware and operating systems like the iPhone App Store or Google Play Store and deploy those apps in containers. But as far as I know, no one currently has that technology for enterprises.
Lately, a lot has been said about Claude Mythos’ ability to detect vulnerabilities in his code. Could you add that functionality to the model that generates the code so you can detect and fix vulnerabilities before releasing your working code?
Dunhue: I don’t have access to Mythos, so I can’t really talk about it. It used to be that you paid people to develop software, you paid people to find holes in the software, you paid people to fix those holes. I paid 3 times and the software should have been secure from the beginning. The only thing that will change with AI is that you will now be paying Anthropic to generate code, find vulnerabilities, and patch them for you. Instead of paying a human, you pay a vendor. However, the model that is generating this code does not generate secure code to begin with.
From what I’ve read, Mythos is good at finding vulnerabilities and also being able to fix them. This means that you will be faced with a barrage of vulnerabilities that will be discovered. Organizations like Commonwealth Bank access that model through partners to quickly patch all systems. But what about other businesses in Australia? Anthropic is currently expanding access to organizations in Australia, but we don’t know anyone who has access.
I really don’t understand why this is limited to certain banks and not others. Because I think the rest of the banks will suffer if that model comes out, and the model has already been leaked and copied. I think the vulnerability apocalypse will begin within the next 3-6 months. I told my product security people, “We need to stop thinking, ‘Let’s patch this vulnerability in 24 or 48 hours.’ We need to build a system that can patch it within minutes.”
So they’re trying to figure out how to use AI to learn about vulnerabilities and patch them in production within minutes? We need to use AI because neither humans nor teams can work at that speed. AI should be used for defense.
It’s great that they have systems like Mythos, but some of this code is used in Cobol, Java, and very old languages, so they need to focus more on responding, defending, and helping developers build and patch these things, even if it’s legacy code.
Last year, we released AI Security Rules for free on GitHub. What do you think?
Dunhue: The idea is to provide some rules that can be applied to weak models, raising the security of the output code to the level of more expensive models. The rules we released last year were an experiment for us, using the data we collected to target specific models that produce specific languages. Certain models require five rules to be applied during Java generation, making the output more secure. Our current product doesn’t include that, but we hope to have it by the end of the year.
The reason for fewer rules is that the more context you provide to the model, the more it will ignore some of that context. If you include 300 rules, 100 of them could be ignored. It is not possible to retrieve all security rules and feed them to the model. You need to be specific. Therefore, we are planning rules to specifically fix the gaps we identify in a particular model.
Some say you should lay off most of your developers, but if you have 1,000 developers using agent AI, why would you want to lay off half of them and slow down development?
Pieter Danhieux, Secure Code Warrior
Then there’s SCW Trust Agent, which was released a few months ago. What role does it play?
Dunhue: Predict that some models will be compromised. At some point, whether it’s a Chinese model or a European model or whatever, you’re going to find out that the model has been compromised for months. Any organization would then ask, “Did we use this model?” Where was it used? What part of the code is affected? How do we know that no one is using it?
This is the functionality we provide with Trust Agent. You can go back in time and see, for example, that DeepSeek was used by these agents or these developers and used in these applications and these lines of code, so you have full traceability.
Additionally, we go a step further by providing visibility into AI tools. Is it AI-assisted? Are they AI native? Are they agents? Are they using Claude Code or Google Antigravity? Which MCP [model context protocol] Are the servers in use? Once the Trust Agent maps all of this, you can implement governance policies that prevent the use of tools and models that are deemed unfit for purpose. Trust Agent provides visibility and governance for software development.
If you look at the past 60 days, there have been two to three major breaches in the AI ecosystem that no one expected. GitHub is one of them, a company owned by Microsoft that takes security seriously. We are seeing many different types of automated attacks compared to a year ago, and until we understand this ecosystem and put proper visibility and controls in place, this trend will accelerate and get worse.
Do you provide short-term advice to those responsible for software development within your organization?
Dunhue: Move fast with AI, but don’t move too fast. This needs to be deployed in a controlled manner and not give everyone the freedom to write code using their favorite vibe coding tool and publish it to production as soon as possible. Unfortunately, I’ve seen many of my fellow CEOs tell their people to just go for it, because they need to reinvent their companies, and to release code as soon as possible to show the market that they’re on board the AI train. That’s dangerous.
Do you think the developer will still be around in a few years?
Dunhue: When using AI in life, 80% of the objectives can be achieved with AI. I’m a pretty technical person. For the remaining 20%, you have to keep debugging, fixing, and rerunning. I think the same is true for software development. You can get to 80% very quickly, but to make sure it works perfectly, developers need to debug and problem-solve and adapt it to the right architecture.
So I think developers are here to stay. Some say we’ll fire most of the developers, but if you have 1,000 developers using agent AI, why would you want to fire half of them and slow down development? I think there are other economic factors to lay off developers rather than replace them with AI. I’m sure developers will continue to exist, but perhaps doing things differently than they have in the past.
