Apple has removed an app it says created non-consensual nude images from its App Store. This is a move that shows Apple's willingness to tackle the dangerous app category.
Generative AI's ability to create images based on prompts has made it an incredibly useful tool in photography and design. However, this technology has also been misused to create deepfakes and non-consensual pornography.
Despite the dangers, Apple has stayed surprisingly quiet on the issue. Before the recent move, it hadn't done much to solve potentially serious problems.
In a report by 404 Media, Apple has been informed that a number of AI image generation apps are available on the App Store. Specifically, it was an app that was advertised as being able to create non-consensual nude images.
These apps offered features such as face swapping for adult images. Others were marketed as “undressing” apps that effectively strip clothing from the subjects of innocuous photos.
After being alerted to these apps and associated ads, Apple removed three of them from the App Store. Google also removed the app from the Play Store.
The report's investigation previously raised the issue of Instagram promoting the app through Meta's ad library, and once the ads were flagged, Meta removed them.
late start
While it's good news that Apple has removed the app from the App Store, some issues remain. First, Apple couldn't ban an app as part of the App Store review process, instead having to be alerted to its existence by a third party.
Still, this is a step further than previous attempts by publications to combat apps.
A 2022 report on these apps explains that the apps initially appeared benign on their App Store pages. However, deepfake porn features were advertised on porn sites.
At the time, Apple and Google were alerted to these apps but refused to remove them. In exchange, the companies were ordered to stop advertising on adult sites and the apps were allowed to remain on the App Store.
Despite this order, one of the apps continued to market adult features until 2024, after which it was removed from the Google Play Store.
Apple's decision to finally remove the app in question from the App Store is the latest move by Apple to keep its dealings with AI as normal as possible.
In addition to using AI language model training methods that protect privacy, Apple has also avoided using copyrighted works in illegal ways. Microsoft and OpenAI are being sued; new york times Citing copyright infringement by using its own articles for AI training, Apple instead sought to license the work from major publishers in exchange for millions of dollars.
