A YouTuber in his 30s has been arrested for using artificial intelligence to create and spread a fake video that falsely depicts police using excessive force against civilians. The arrests reflect a stepped-up police response to a growing number of cases in South Korea and abroad where AI-generated content causes false reports and erroneous dispatch of police and fire departments.
On February 2, the Gyeonggi Northern Regional Police Agency announced that it had arrested a YouTuber on charges including distributing false communications based on the Telecommunications Basic Act, and sent him to the prosecutor’s office. Police said the suspect created an AI-generated video that resembled police body camera footage and distributed it online. According to investigators, the suspect is said to have created and uploaded 54 fake videos since October last year.
Police said the YouTuber used tools such as ChatGPT to generate the prompts and an AI video generation program called Sora to create the videos. He mainly uploads short videos on social media platforms, which have a total of around 34 million views.
Videos included police chasing an armed Chinese man and subduing him with a Taser, and officers pushing a female streamer to the ground and arresting her after a confrontation on the street. Another video showed participants at a rally against US President Donald Trump shouting at police officers: “Corrupt police, resign!” The footage is created to closely mimic real police activity and includes AI-generated audio and ambient background noise.
Officials say a key concern is how easily such AI-generated videos impersonating state authorities can be discovered across major platforms. As of February 2, manipulated videos featuring police officers and firefighters remained accessible on major social media sites without being removed.
In one video uploaded last October, police reprimand a young man for hoisting a South Korean flag at a rally, telling him that he needed permission to wave the flag and that since a Chinese delegation was visiting that day, he should hoist a Chinese flag instead. Many viewers mistook the footage for real and posted comments claiming the country was becoming communist.
Other provocative videos showed police spraying water on streamers and fire trucks blocking roads and ramming vehicles to force them aside.
Police have warned that AI-manipulated videos pose risks that go beyond undermining public trust in law enforcement, including the misuse of limited police and emergency response resources. Last year, a family in South Korea called 911 after mistaking an AI-generated video of a homeless person breaking into their home as an actual incident.
Similar cases are emerging overseas as well. In Japan, an AI-generated fake police officer appeared on a video call and asked for money under the guise of an investigation. Controversy erupted in Australia after an AI-generated video spread misinformation about firearm licensing requirements.
Seo Jun-bae, a professor of public administration at the Korea National Police University, said, “It is natural that videos that are fabricated with malicious intent will be punished for crimes such as obstruction of official duties and defamation, which is a legal punishment that is heavier than a violation of the Telecommunications Act.”
Email: cho@donga.com
