apple and google According to cease-and-desist letters sent to companies obtained by WIRED, they were ordered to remove apps that allow people to “undress” or “undress” people and to stop profiting from harmful technology.
On Thursday, San Francisco City Attorney David Chiu sent legal notices to Apple and Google asking them to remove 13 face-swapping apps that can create non-consensual AI-generated nude images from their app stores. The letter says major Silicon Valley companies should stop “aiding and abetting” the sale of explicit deepfake images and “sever” business relationships with app developers.
“Producing non-consensual intimate images is illegal, harmful and completely unacceptable,” Chiu told WIRED. The city attorney, who previously filed suit against 16 popular deepfake websites, said Apple and Google likely “earn millions of dollars in fees” from apps offering nudity and that they need to improve their moderation processes to prevent them from appearing on the store in the first place.
“These companies have a responsibility to ensure that the apps on their platforms do not facilitate sexual abuse,” Chiu said. California law prohibits support for services that create deepfake pornography, according to the city’s legal letter. According to the letter, these apps use in-app payments, of which the technology companies receive a cut. “The fact that some of the world’s largest and most established technology companies are facilitating this has to stop.”
Researchers have repeatedly discovered and reported apps on Apple’s App Store and Google’s Play Store that can use AI to generate sexual images. This includes apps that are rated suitable for use by children. New laws and bans are aimed at addressing the scourge of blatant deepfakes online, but technology and social media companies consistently steer millions of people to harmful technology.
Apple and Google both have developer policies that prohibit pornography, abuse, and harassment on their platforms. They have so far removed dozens of nudify and deepfake apps following reports from researchers and journalists.
Google spokesperson Dan Jackson told WIRED that the company has removed “hundreds” of apps that had the ability to disable policy violations. That includes five Android apps reported by Chiu’s office, and other measures have been taken to restrict access to them.
“Google Play does not allow apps with sexual content, and we continue to take proactive steps to detect and remove apps with harmful content,” Jackson said in a statement. “When violations are reported to us, we investigate and take swift action, including, in the case of these apps, suspending hundreds of violating apps and restricting related search terms such as ‘nudity’ in the Store.”
Apple told WIRED that developers are ultimately responsible for their apps’ content, and that developers who don’t follow the rules will have their apps removed. The company also says users can self-report potentially problematic apps. Apple spokesperson Adam Dema told WIRED in a statement that the company has removed three of the apps accused by the San Francisco City Attorney and is “in the process of suspending developer accounts from our programs.” Four other developers “must also address the risk of removal if they do not address policy violations,” the company said.
“The App Store is designed to be a safe and trusted place for our users, and we have always strictly prohibited apps designed to generate, distribute, or consume pornography,” Dema said in a statement. “‘Nude’ apps violate our app review guidelines, and we have actively rejected many of these apps and removed many others, including when users reported them through our reporting tools.”
