While generative artificial intelligence is beginning to have a growing presence in state and local government operations, other types of AI are already seeing mainstream adoption, particularly in the field of cybersecurity.
Threat detection, incident response, and anomaly identification all require rapid processing of large amounts of data, which is why automation and machine learning have been used since the late 1990s. The public release of OpenAI's ChatGPT in fall 2022 sparked a boom in generative AI, creating an opportunity to explore new uses for AI, but experts told StateScoop that the application of AI in cybersecurity will only enhance what automation and machine learning have been doing for years.
Cybersecurity officials and analysts said the real power of generative AI in cybersecurity is not in threat detection, but in integrating data generated by threat scans and helping cybersecurity professionals learn from them contextually. This ability, they said, allows generative AI to address increasingly sophisticated threats and reduce the need for human intervention.
“From what I've seen, the biggest benefit that AI brings as far as cybersecurity tools is the ability to basically crunch large amounts of data very quickly and contextualize the data,” said Andy Hanks, senior director at the nonprofit Center for Internet Security.
“Whether it's for incident response or threat hunting or whatever tool you use it for or any area of cybersecurity. [generative] “AI really changes the game when it comes to processing large amounts of data very quickly and providing contextual information about it,” said Hanks, a former chief information security officer for the state of Montana.
Before the advent of generative AI, automation enabled cybersecurity professionals to deploy pattern-recognition-based defensive systems, also known as expert systems. Expert systems use automation to mimic human expertise and use data to power task-specific algorithms. Expert systems have been around since the 1970s, but in recent decades in cybersecurity they have been used to scan the vast amounts of data generated by infrastructure scans for known threat signatures.
“It's very rules-based, especially when it comes to networks and that kind of thing. A good analogy is diagnosis in healthcare,” says New Jersey CISO Mike Geraghty. “We take specialists, we input all of their findings and their different roles, and we provide different rules to the next doctor who uses the system, and he makes a diagnosis based on that. It's the same with network security, antivirus, everything.”
Expert systems rely on frequent updates to detect new threats. Hanks said he knows of a ransomware attack that occurred in an unnamed state several years ago, in which threat actors used a known, signature-based virus that was supposed to be detected, but evaded the state's pattern-based system by simply changing a few cosmetic features of the signature. Hanks said it was as if the bad actors had given their malware a fake mustache to fool the detection system.
Joining a Stack
Kansas CISO John Godfrey told StateScoop that with threats evolving so quickly, it's too risky to wait for pattern-based automation techniques to be updated and new threat models built into detection mechanisms. To overcome this, stacking automated threat detection tools as filters has become standard practice. Email threat detection is a good example of how stacked AI techniques are being used in cybersecurity today, he said.
“It might start with a spam filter list, then it might be Bayesian logic or fuzzy math logic,” Godfrey said of the decision-making framework, “and then it might run through a couple of different antivirus engines to see if anything still jumps out, and then finally as it goes through the stack, a conviction or outcome is typically made based on the output.”
This is where generative AI can help: Contextual information such as summary scan results and recommended defensive actions can help cybersecurity professionals better orient themselves and their tools to predict future attacks, as well as customize detection systems that can automatically trigger alerts and response actions.
In New Jersey, Geraghty said generative models are helping rewrite threat detection algorithms. While humans can write such computer code, large code bases often become unwieldy, especially when managing multiple tools in the stack.
“We use AI-powered code generation,” he says, “When we think about ChatGPT and others, we write detection rules. We say, 'How do we detect something?' and we can ask the generative AI tool to help us write the rules for us across the different types of tools that we're using.”
“They're all using AI.”
While the advent of commercial generative AI makes the distant reality depicted in science fiction movies like “The Matrix” seem closer than ever, AI designed to emulate neural processes to complete complex tasks has been around for decades, even before the advent of expert systems. Geraghty said the novelty of generative AI has overshadowed this history, making the term “artificial intelligence” blurry.
“I think in the last two years generative AI has become so popular that everyone thinks of it as AI, but by the time generative AI came along, there's probably been 70 years of all kinds of machine learning models and other AI concepts being used,” he said.
It ultimately comes down to a difference in what AI is designed for, and that difference is important to cybersecurity, he continued. As the name suggests, generative AI is used to create new content — text, images, audio, video, code — in response to specific text prompts. In contrast, automation is designed to streamline complex processes and save time, especially when it comes to tedious tasks.
But apart from code generation, generative AI isn't yet perfect for threat detection, Geraghty said.
Hanks, the director of the Center for Internet Security, said one of his first jobs in cybersecurity was on the IT server team in the 1990s, where one of his roles involved manually identifying anomalies in large data sets such as security logs – a task that AI-powered expert systems were designed to do.
“Back then, your first job on the server team — any new guy — was to look at logs all day,” Hanks says. “You'd literally sit there for eight hours a day scrolling through logs of different servers, looking for issues. After a while, you got good at scrolling, you could scroll pretty fast, and your brain could detect breaks in the patterns.”
Eventually, that role was replaced by AI, and over the past two decades the use of machine learning has become essential to the practice of cybersecurity. From basic pattern recognition, to identifying known threats, to automating routine tasks, fundamental aspects of cybersecurity all rely on AI.
“All of these tools and technologies, like intrusion detection systems and antivirus systems, use AI in some form, but it's not the generative AI that we all know and love these days,” Geraghty said.
Inconsistent models
While generative AI helping to contextualize data is likely one of the most impactful uses in cybersecurity, others are emerging. One new use, Godfrey said, is predictive threat hunting, which further limits the need for human intervention.
One way to do this, Godfrey said, is to use “honeypots,” which are trap servers and databases that are made to look legitimate. Honeypots have been around for decades, but generative AI is making them even better, Godfrey said.
“What I'm starting to see is a shift towards AI that can run threat hunting continuously in the background,” Godfrey said, “and in some cases start to spin up virtual honeypots in very situational instances based on signals from threat actors, encouraging them to engage with this virtual fake infrastructure that's created at the time of interaction, to deflect attacks against your core infrastructure.”
Steven Sims, a researcher at the SANS Institute who was one of the first to successfully create malware using ChatGPT, told StateScoop that stacking generative AI and deep learning models is an increasingly common technique, similar to how different automated tools are stacked for email threat detection.
“I also think that companies [large language model] “Agents, each with a specific specialty, work together to identify threats,” Sims said in an email. “One example is educating them on what threat actors might look like. This involves running penetration tests and red team exercises against target environments, where different agents ingest data and are able to distinguish between legitimate and attack traffic.”
New Jersey has tested and recently released its own large-scale language model that allows state officials to ask questions about cybersecurity incident data, Geraghty said. The model doesn't necessarily generate new detection rules, he said, but it can give officials answers to questions like, “How many ransomware incidents have occurred in the water department in the last six months?”
Jerraghy said the state has no plans to make the tool available to the public, and that the model's inability to return consistent answers shows how much generative AI still has to advance.
“What we've found, as is common with generative AI, is that if you ask the same question over and over again, you'll get different answers. That shouldn't be the case in cybersecurity,” Geraghty said. “The risk assessment that we provide to our customers has to be accurate. There can't be redundant data, hallucinations, anything like that.”
