Human content moderators can outperform AI when it comes to recognizing the material that drives policy, but they are also significantly higher costs.
Marketers who want to prevent ads from surface with toxic slurries are faced with a dilemma.
Researchers from AI Brand Protection performed the mathematics detailed in a prelint paper entitled “AI vs. Human Moderators: Comparative Evaluation of Content Moderation for Brand Safety in A Complay LLMS.”
This paper was accepted in the upcoming Computer Vision (CVAM) of the Advertising and Marketing (CVAM) workshop at the International Conference on Computer Vision in 2025, and presents an analysis of the cost and effectiveness of a multimodal leading language model (MLLM) of brand safety tasks.
Researchers' calculations show that human moderation is the best dul and about 40 times the most cost-effective machine learning labor.
Brand safety means preventing inappropriate content from being associated with a brand and damaging the brand's reputation. It has become a kind of moving target in the wake of the Trump administration's rollback of diversity, equity and inclusion. It's accused of intentionally distributing harmful content, unlike consumer content moderation on social media sites like Meta's Instagram. [PDF] Face related lawsuits.
The ZEFR team stated, “Advertisers define what content categories they want to avoid, ranging from violent or adult-themed materials to controversial political discourse. While general content moderation aims to identify and manage content that uses policy, brand safety is particularly relevant to appealing advertising preferences and advertising placement.
The authors generally state that brand safety efforts include a combination of human reviews of images, audio and text with machine learning-based analytics. The purpose of this study was to investigate whether MLLM could do well at work.
They evaluated six models: GPT-4O, GPT-4O-MINI, GEMINI-1.5-FLASH, GEMINI-2.0-FLASH, GEMINI-2.0-FLASH-LITE, and LLAMA-3.2-11B-VISION. Death, Injury, Military Conflict (DIMC); and Child Content.
Researchers achieved performance in each of three categories: accuracy, recall and F1, a common method of machine learning evaluation. Accuracy refers to the predicted positive classification of content (policy violations) compared to actual positive instances of the dataset. A recall refers to the percentage of actual positive instances that are correctly classified. F1 is the harmonic average of accuracy and recall.
The overall score (accuracy, recall, F1) is as follows, with 1.00 representing 100% accuracy, with no false positives or false negatives.
| Model | accuracy | Reminiscence | F1 |
|---|---|---|---|
| GPT-4O | 0.94 | 0.83 | 0.87 |
| GPT-4O-MINI | 0.92 | 0.85 | 0.88 |
| Gemini-1.5-flash | 0.86 | 0.96 | 0.90 |
| Gemini-2.0-flash | 0.84 | 0.98 | 0.91 |
| Gemini-2.0-Flash-Lite | 0.87 | 0.95 | 0.91 |
| llama-3.2-11b-vision | 0.87 | 0.86 | 0.86 |
| human | 0.98 | 0.97 | 0.98 |
“In the MLLMS, the Gemini model appears as the best overall model, outperforming others in terms of F1 score,” the researchers stated in the paper, adding, interestingly, that compact versions of these models have not deteriorated significantly.
“These results highlight the effectiveness of MLLM in automating content moderation, but precisely highlight the continued advantage of human reviewers, particularly in more complex or nuanced categories that require context and deep understanding,” the paper states.
Researchers also observed that these models often failed due to false associations, lack of contextual understanding, and linguistic differences. One example they cited is a video discussing caffeine addiction in Japanese, which was mistakenly flagged as a drug category violation by all models. The authors attribute this to the flawed connection between the addiction and the term gap in Japanese contextual understanding. In general, they said that these models show poor performance for content outside of English.
In terms of cost, good human moderation looks like luxury. Here's how to compare the models in terms of F1 score and price:
| Model | F1 | Fee |
|---|---|---|
| GPT-4O | 0.87 | $419 |
| GPT-4O-MINI | 0.88 | $25 |
| Gemini-1.5-flash | 0.90 | $28 |
| Gemini-2.0-flash | 0.91 | $56 |
| Gemini-2.0-Flash-Lite | 0.91 | $28 |
| llama-3.2-11b-vision | 0.86 | $459 |
| human | 0.98 | $974 |
“Compact MLLM has shown that it offers a much cheaper alternative compared to its larger counterparts without sacrificing accuracy,” the author concludes. “However, human reviewers have excellent accuracy, especially in complex or subtle categories.”
“Large multimodal language models such as Gemini and GPT can handle the moderation of brand safety videos across text, audio and visuals, but are surprisingly accurate and far less expensive than human reviewers alone, but lacking in cases with many subtle contexts.
The datasets and prompts used are published on GitHub. ®
