San Francisco city attorney David Chiu has demanded that Apple and Google remove 13 artificial intelligence applications from their stores that generate non-consensual nude imagery. The legal notices, allege the tech giants are profiting from the apps and must stop facilitating sexual abuse by severing ties with these developers.
Legal Demands and Financial Stakes
The city’s move targets a specific class of software designed to “nudify” or “undress” individuals through AI-driven face-swapping technology. In cease-and-desist letters sent to both Apple and Google, the San Francisco city attorney’s office argues that these platforms have effectively become engines for digital sexual abuse. The legal notices claim that by hosting these applications and collecting a percentage of in-app payments, the companies are aiding and abetting
the distribution of explicit deepfake content.
David Chiu, San Francisco city attorney, said: Generating non-consensual intimate images is illegal, harmful, and completely unacceptable. The fact that some of the world’s largest and most established technology companies are facilitating this has to stop.
Chiu, whose office previously took legal action against 16 popular deepfake websites, asserts that Apple and Google have likely made millions of dollars in fees
from these apps. He contends that the companies possess the responsibility to ensure their platforms do not facilitate harm, suggesting that current moderation processes are failing to catch the software before it reaches the public. The city’s legal letters emphasize that California’s laws prohibit supporting services that create deepfake pornography.
Platform Responses and Policy Enforcement
Both tech giants maintain that their developer policies strictly prohibit pornography, abuse, and harassment on their platforms. However, the companies have responded to the specific request from the city attorney’s office with varying degrees of action. Google spokesperson Dan Jackson stated that the company has deleted “hundreds” of apps with nudifying features for policy violations, including the five Android apps flagged by Chiu’s office.
Google Play does not allow apps that contain sexual content, and we continually take proactive steps to detect and remove apps with harmful content,
Jackson said in a statement. He added that the company has taken steps to restrict access to them, including limiting related search terms like “nudify” on the store. When violations are reported, Jackson noted that the company investigates and takes “swift action.”
Apple told WIRED that developers are ultimately responsible for the content of their apps. In a statement to Engadget, Adam Dema, an Apple spokesperson, confirmed the company has removed three of the apps flagged by the city attorney and is in the process of terminating their developer accounts from our program.
Regarding the other four developers, Apple stated they must address policy violations or risk being removed as well.
Dema added that 'Nudification' apps are against our App Review Guidelines and we have proactively rejected many of these apps.
Visibility and Safety Concerns
The push by San Francisco officials highlights a broader, persistent issue with the visibility of deepfake tools. Researchers have repeatedly found apps that allow users to generate sexual imagery using AI—including some apps being rated as suitable for use by children. A report from The Tech Transparency Project (TTP) alleged that nudification apps were frequently making it past Apple and Google’s moderators and were even being actively promoted on the stores. Finding the apps, it claimed, is as simple as searching for terms like “nudify” or “undress.”
The conflict places the companies in a difficult position. While Apple maintains the App Store was designed to be a safe and trusted place for users,
the city attorney’s office is pushing for a more proactive stance to ensure such apps do not appear in their stores in the first place. The tech industry continues to face pressure regarding deepfakes, as Meta’s Oversight Board recently called for stronger protections for ordinary people, and other companies like xAI have faced lawsuits relating to nonconsensual deepfakes.
