Are small-scale models closing the gap with frontier AI cyber tools?

The Frontier model, which is part of Anthropic’s Project Glasswing and OpenAI’s Daybreak, is behind an access control program and is available only to vetted partners. Access is limited because the model can reason through vast codebases, chain individual vulnerabilities into working exploits, and surface flaws that have survived decades of prior scrutiny.

Small language models built to run with less data and fewer tokens were thought to be unsuitable for that kind of inference, but industry experts are now exploring whether and under what conditions they can close that gap.

It’s easy to find flaws in specific sections of code if the model points to the right places. Reasoning through an entire codebase to find vulnerabilities that no one has named before is another matter entirely, and that’s where smaller models struggle.

According to Philippe Delassou, AI penetration testing lead at Aikido Security, a cheaper model might succeed 30% of the time with a complex identification step, while a Frontier model would have an 80% success rate. As the exploits cascade and move deeper into the compromised system, the odds double with each step. The smaller model completes the entire sequence about 3 percent of the time, and the frontier model completes it nearly half the time, he said. “The more difficult and time-consuming the task, the better the smart model will perform,” he told ISMG. Frontier models can hold more code and prior inferences in working memory simultaneously, and training involves solving multi-step problems.

Dipto Chakravarty, chief technology officer at Black Duck, said that while smaller language models have limitations in solving more difficult tasks, the key to success rate is understanding how often those tasks actually occur. Most of what organizations need is reliable high-volume detection and triage, rather than new exploit chains, he said. To that end, the scaffolding around the model – the systems that govern what code is inspected, how many times the model is tried, and how the results are organized – is as important as the model itself.

“If Anthropic runs Terminal-Bench with a cap of 1 million tokens per task, 5 retries, and 3x compute, an honest lead engineer would guess that half of the observed feature delta belongs to the harness, not the weights,” he told ISMG. In other words, if the benchmark gives the model enough resources and multiple trials, much of the performance improvement will come from those conditions, not from the model getting smarter.

This argument is supported by Microsoft’s findings during an experiment called MDash. The experiment combined a frontier model for complex inference with a smaller extractor model, a compressed version trained to efficiently perform a specific task across a pipeline of over 100 tuned agents. On CyberGym, a benchmark of 1,507 real-world vulnerability reproduction tasks, MDash outperformed both Mythos and GPT-5.5. No single model delivered the results, but the pipeline did.

For high-volume, repeatable tasks that occupy most security teams’ days, such as matching findings to known vulnerability categories or correlating alerts across systems, small-scale models trained on the domain already outperform state-of-the-art models in terms of accuracy. IBM Research’s CyberPal 2.0 is a family of security-focused models that outperforms GPT-4o and o1 in core threat investigation tasks.

George Gerchow, chief security officer at Bedrock Data, said the results the small-scale model reproduces are not what the Frontier model demonstrated. “The bug lived on for decades through all the static analyzers, fuzzers, and pattern matchers we targeted against the same code,” he said of the Mythos vulnerabilities that surfaced. “They didn’t survive because no one saw them. Because finding them requires reasoning across thousands of lines of context to find interactions that no one knew existed. It’s not a search problem. It’s an inference problem, and the upper limit of inference is in the model.”

In most benchmarks, small models are given relevant code directly, he said. A true autonomous scan starts with the entire codebase and needs to find that code first.

Vulnerability reports generated by AI are arriving faster than human reviewers can evaluate them. HackerOne suspended its Internet bug bounty program earlier this year after AI submissions exceeded its ability to triage. The cURL project also ended its bug bounty program for the same reason. Garchow says the frontier model’s reasoning power is the most difficult to replace. That’s because it evaluates not only whether a flaw exists, but whether it’s reachable, exploitable, and worth addressing.

Howie Koh, vice president of innovation at Forescout, said both positions describe different layers of the same problem. Small-scale models handle continuous, cost-effective sweeps, while Frontier AI covers periodic deep analysis when inference depth is required.

“The result is multiple models within a single harness, opening up a market gap for vendors who can optimize outcomes and return on investment rather than offering a harness that works best with their own Frontier models,” he told ISMG. This opportunity essentially belongs to vendors who don’t have a unique model to protect and sell. Because vendors can choose the right tool for each task, rather than defaulting to the most expensive tool.

“To justify the investment to run a frontier model, all it takes is one zero-day discovery that the frontier model emerges and smaller models are missed,” Gerchow said. The release of Fable 5 gives security teams another option depending on what they’re trying to solve and how much they’re willing to miss.