There’s still evidence that ChatGPT doesn’t kill IT security teams.
Endor Labs researchers tested ChatGPT 3.5 against 1,870 artifacts from the PyPi and NPN open source repositories of code. Identified 34 as infected with malware. But only 13 actually had bad code. His other five artifacts obfuscated the code, but did not reveal any malicious behavior. One artifact, on the other hand, was a proof of concept that downloaded and opened an image via an NPM install hook. As a result, the researcher considered ChatGPT-3.5 to be 19 out of 34 options.
However, 15 of the results were false positives.
The researchers also found that the tested versions were able to reduce their ratings from malicious to benign by using benign function names, by including comments in queries that denote benign functions, or by including string literals. I’ve also discovered that you can be tricked into changing to
Endor Labs researcher Henrik Plate concludes in a blog that model-based multilingual malware review “can complement human review, but not replace it.” .
However, the latest version is ChatGPT-4, and Plate admits to showing different results.
He also acknowledged that preprocessing code snippets, additional efforts in rapid engineering, and future models are expected to improve the company’s test results.
According to researchers, Large Language Models (LLMs) such as GPT-3.5 and GPT-4 help IT staff assess the likelihood of malware. Microsoft already does that with their Security CoPilot application.
Still, here are the researchers’ conclusions: ChatGPT-3.5 is not ready to replace humans.
“One particular problem seems to be the reliance on identifiers and comments to ‘understand’ the behavior of the code,” writes Plate. “While they are a valuable source of code developed by harmless developers, they can also be easily exploited by attackers to evade detection of malicious behavior.
“However, LLM-based assessments should not be used, Instead Can certainly be used as one additional signal and input for manual review. In particular, it is useful for automatically reviewing a large number of malware signals generated by noisy detectors (otherwise at risk of being completely ignored if review capabilities are limited). ”
