OpenAI Introduces New Child Safety Tools to Detect CSAM and Prevent Harmful AI Use
OpenAI has announced a new set of child safety tools designed to fight child sexual abuse material, including AI-generated images, as concerns continue to grow over how artificial intelligence platforms can be misused.
The new measures include an image classifier built to identify child sexual abuse material, warning screens and temporary account blocks in ChatGPT, updated guidance for developers, and AI-powered tools for law enforcement teams working on child protection cases.
A major part of the announcement is OpenAI’s new CSAM image classifier. The system is designed to detect visual signs of abuse in both real and AI-generated images, even when the images have not been previously seen or matched in existing databases. Instead of making final enforcement decisions on its own, the tool assigns a risk score that can help human reviewers prioritize and assess potentially abusive material more efficiently.
OpenAI said it developed the classifier in collaboration with child safety experts, including the National Center for Missing and Exploited Children. The company is making the tool available through its API to select organizations at no cost, with the goal of helping trusted partners identify abusive content faster and reduce the burden on review teams.
According to early testing shared by OpenAI, the classifier helped reduce the number of images that needed to be reviewed by 48% while still identifying the same amount of abusive material. The company also said images flagged by the classifier were twice as likely to contain abuse, suggesting the tool may help reviewers focus on the most serious cases more effectively.
OpenAI is also expanding safety measures inside ChatGPT. Users who attempt to create or access content that sexualizes children will now see warning messages. These alerts are intended to interrupt harmful behavior, direct users toward specialized support resources, and make clear that the activity violates OpenAI’s policies.
In some cases, OpenAI will apply a 24-hour account suspension as a warning. The company said repeated violations can lead to permanent account removal. ChatGPT already blocks requests involving child sexual abuse material, and the new warnings are meant to act as an additional deterrent before users continue down harmful paths.
OpenAI said the wording and structure of these warning messages were developed with input from child protection organizations and researchers. The company also noted that an independent evaluation was carried out during a trial of deterrent messages, helping shape how the alerts are presented to users.
Beyond ChatGPT, OpenAI is publishing new guidelines for developers who use its AI tools. These guidelines are focused on helping developers prevent, detect, and report child sexual abuse material across apps and services built with OpenAI technology. In early testing, OpenAI said the guidance helped developers adopt child safety protections and register to report abusive material to the appropriate authorities.
The company is also introducing support for law enforcement teams that handle child safety investigations. These tools include AI-assisted investigative features, dedicated plugins, data analysis support, training materials, and a field guide on how investigators can use AI responsibly. Access to these resources will be managed through program administrators.
The announcement comes at a time when AI companies are facing increasing pressure to strengthen safety protections, especially for younger users. OpenAI has faced criticism and legal scrutiny over alleged harms linked to AI tools, while child safety advocates have warned that generative AI can make it easier to create or spread harmful sexualized content.
Concerns are not limited to one platform. Other AI chatbots and image tools have also been misused to create nonconsensual sexualized imagery, including content involving minors. This has intensified calls for stronger safeguards, better reporting systems, and closer cooperation between AI companies, child protection organizations, and law enforcement.
Over the past year, OpenAI has introduced several safety-focused changes, including a broader safety framework, updated content rules, model adjustments designed to reduce harmful outputs, and a teen-focused version of ChatGPT. The latest child safety tools build on those efforts by targeting one of the most serious risks associated with generative AI: the creation, discovery, or distribution of child sexual abuse material.
With the new classifier, ChatGPT warning screens, developer guidance, and law enforcement support, OpenAI is positioning child protection as a central part of its AI safety strategy. The effectiveness of these tools will depend on how widely they are adopted, how accurately they perform in real-world settings, and how quickly platforms can respond when harmful content is detected.






