The Oversight Board, an independent policy advisory body for Meta, is closely examining how explicit, AI-generated images have been managed on social media platforms Instagram and Facebook. The board has launched investigations centered around incidents in India and the United States, where Meta’s moderation systems initially failed to address the distribution of explicit content depicting public figures.
In these two instances, the platforms eventually removed the content, but not before Meta’s automated review systems had inadequately responded to user reports. The Oversight Board has chosen not to disclose the identities of the targeted individuals to prevent further gender-based harassment.
When users are dissatisfied with Meta’s moderation decisions, they can make appeals directly to the company before potentially involving the Oversight Board. The board plans to release comprehensive reports with findings and recommendations on these cases in the near future.
The Particular Incidents
The first incident involved an explicit AI-generated image of a public figure that was reported by a user on Instagram in India. The image was a part of a series posted by an account that shared AI-generated images of Indian women. Despite the initial report, Meta did not take down the image, and the complaint was automatically closed after 48 hours without further review. It was only after an appeal to the Oversight Board that the company removed the image for violating its community standards on bullying and harassment.
The second incident occurred on Facebook in the United States where an explicit AI-generated image resembling a public figure appeared in a group dedicated to AI creations. Facebook removed the image because it had been previously taken down, and Meta had designated it as inappropriate in its Media Matching Service Bank.
Broader Implications
The Oversight Board selects cases that represent wider issues within Meta’s platform. By examining instances from different regions, the board aims to assess the global efficacy of Meta’s policy and enforcement measures. Co-Chair of the Oversight Board, Helle Thorning-Schmidt, highlighted the importance of evaluating whether Meta’s policies are effectively protecting individuals worldwide.
Challenges with Deep Fake Porn and Online Gender-Based Violence
Generative AI has the capability to produce pornography, which raises ethical concerns and highlights the issue of biased data sets. In places like India, the prevalence of deepfakes, particularly concerning Indian actresses, has sparked significant worry. Deputy IT Minister Rajeev Chandrasekhar in India has criticized tech companies for their lax approach to countering deepfakes and has suggested that platforms may be blocked if they fail to take appropriate action.
Although India is considering introducing specific laws to combat deepfakes, there currently are no firm regulations. Online gender-based violence needs robust legal processes that do not trivialize the issue. Aparajita Bharti, from The Quantum Hub, advocates for limitations on AI models to prevent the generation of harmful content and for implementing default labeling for easier detection.
Meta’s Response and Further Action
Meta has stated it uses a combination of AI and human review to filter sexually suggestive content and asserts that such content is not promoted on areas of its platforms like Instagram Explore or Reels recommendations. The Oversight Board has called for public input on these matters, with a deadline for comments set at the end of April. It is expected that a decision, incorporating these public comments and their investigation, will be posted shortly thereafter.
These cases underscore the ongoing struggle social media platforms face with moderating content. While AI is utilized in both content creation and detection, perpetrators continue to craft ways to evade these systems, posting content that can cause harm and distress.
Meta’s experiments with AI detection tools, including those aimed at identifying deepfakes, are ongoing. The company announced it would start using “Made with AI” badges for content that platforms can identify as AI-generated, using technical indicators or user flags. Nonetheless, addressing the subtleties of AI-generated content and its potential misuse remains an active challenge for major platforms like Meta.
