AI spam filters suffer from old-fashioned text salting

Machine Learning


safety

Decades-old email trick turns out to still work for some LLM-powered email filters

Have you noticed an increase in spam getting past your corporate email filters lately?According to cybersecurity firm Barracuda, attackers are using a technique known as “text salting,” which hides seemingly innocuous words, in an attempt to confuse some AI-powered email filters.

Email security organizations announced Thursday that they have detected more than 1 million retail-themed phishing attacks using text salting since April. This is not a new technique and has been used for years to fool traditional secure email gateways, but it also has the potential to confuse machine learning and LLM-based security tools, Barracuda said.

Text salting involves peppering a malicious email with random, seemingly innocuous words (sorry) to trick the email scanning system into thinking the message’s contents are okay (sorry again), and then tricking the system into passing the message on to the recipient for consumption (no food jokes here).

But according to Barracuda, if you put a pile of salty text on top of an email, human readers will likely become suspicious. As a result, attackers typically use one or more of three flavor variations to hide additives from human readers rather than automated scanners.

Typical techniques include CSS cropping, which sets the display window small enough that humans cannot see hidden filler text. Text manipulation to move salty copy off the visible screen. and zero font technology that inserts misleading words between suspicious phishing copies that are visible to machines but invisible to humans.

The end result of each of these techniques is that a machine reads a message that is less malicious and gibberish, makes it think the email is benign, and a human sees the message as the attacker intended.

Modern email security systems are primarily adapted to these techniques, with new tools that allow you to remove hidden text to see what should be shown to readers, and to sound the alarm if too many hidden elements have been inserted into an email. But AI still hasn’t caught up, Barracuda said.

“Text salting and related techniques can be used to confuse AI-driven content analysis engines by flooding emails with random terms that prompt AI systems to make incorrect classification decisions, similar to SEG in the early 2000s,” the company said in its report. What a technological leap forward we have made.

Barracuda said LLMs are typically designed to simply process email text and source code, without understanding whether the text is visible or hidden from the user. It can be trained to do so, but that means most tools probably don’t do it by default.

So what can businesses do to stop the flow of salty spam to their employees? Rather than relying solely on keyword detection, Barracuda recommends a multi-layered approach to email security, including checking sender reputation, authentication results, embedded URLs, HTML rendering techniques, and the difference between user-visible and hidden content.

It might not be a bad idea to eliminate AI spam filters. ®



Source link