San Francisco city attorney demands removal of AI 'nudify' apps from app stores

The San Francisco City Attorney’s Office has formally demanded that Apple and Google remove 13 specific applications from their respective digital storefronts. These applications, which utilize face-swap technology to generate nonconsensual nude imagery, have been the subject of cease-and-desist letters sent to both technology companies this week. The legal action targets software that the city alleges is overwhelmingly used to target women and girls for the creation of explicit content without their consent.

The demand highlights the role of major platform operators in the distribution and monetization of these tools. Reports indicate that both Apple and Google have likely generated millions of dollars in revenue through fees collected from these applications. By hosting these programs, the companies are accused of facilitating the proliferation of nonconsensual imagery while simultaneously profiting from the transactions associated with their use.

The letters serve as a direct challenge to the current moderation policies maintained by the two tech giants. While the specific names of the 13 applications were not detailed in the initial reports, the scope of the demand covers a category of software that has drawn increasing scrutiny from local authorities. The city attorney’s office is seeking to halt the availability of these tools to prevent further harm to individuals targeted by the generated content.

This development places pressure on the companies to reevaluate their app store guidelines regarding generative software. The legal correspondence emphasizes the responsibility of platform owners to police the content and capabilities of the applications they allow on their systems. As of now, the companies have not publicly confirmed their specific next steps in response to the city's demand, leaving the status of the identified applications uncertain pending further action from the tech firms.