What happened

OpenAI has announced a tool designed to flag suspected child sexual abuse material, including images made or altered with artificial intelligence. The company says vetted organisations can access the classifier through its API without charge. It also announced new ChatGPT warnings, guidance for developers and support for specialist investigators.S1S4R1

The classifier looks for visual indicators in previously unseen images, rather than relying only on matches to files already identified as abusive. It gives reviewers scores to help prioritise material. OpenAI says it developed the system with assistance from the National Center for Missing and Exploited Children, which operates the CyberTipline.S3S4R1

Why it matters

OpenAI estimates that its new classifier would flag 48% fewer images than its existing protections while finding roughly the same number of abusive images. It says flagged images were about twice as likely to contain abuse in its evaluation. Those are company testing results, not a demonstrated reduction in the overall volume of child abuse online.S3S4R1

The technical report draws a boundary around what a flag means. A score does not confirm abuse, establish an image’s legal status or verify whether AI produced it. OpenAI recommends trained human review before classifier-based reports are sent to a hotline, and says performance can differ on other platforms.R2

The background

NCMEC recorded more than 400,000 CyberTipline reports connected with generative AI in 2025, compared with 67,000 in 2024. Its definition is broader than newly generated abuse images: some reports involve training data or other AI-related exploitation. It explicitly says not all AI-linked reports involve creating or sharing AI-generated material.R3S1

The distinction affects how the workload is understood. Reports can require assessment even when they do not establish that a new synthetic file was created. OpenAI’s stated aim is to help teams prioritise their work; neither its announcement nor the technical report demonstrates that the new tool has already reduced the national tipline backlog.R1R2R3

What happens next

OpenAI says its new warnings will interrupt attempts to find abusive material, remove the flagged conversation and pause ChatGPT access for 24 hours while directing the user towards specialist support. Repeated violations may bring longer restrictions or account closure. Fox’s account distinguishes this intervention from the company’s existing policy of banning and reporting attempts to generate or upload such material.S3R1

The company also plans training and investigative tools for child-safety law enforcement teams. Their effectiveness, alongside the warnings and classifier, will require further assessment in use. The published model card says the classifier can both miss abusive material and incorrectly flag other images, and recommends evaluation on each organisation’s own representative traffic.S4R1R2