OpenAI’s ChatGPT can reportedly be used to generate graphic erotica and engage in sexually explicit conversations, even if the users are underage.
After a few additional prompts, ChatGPT voluntarily generated sexual stories and further asked about specific role-play scenarios and kinks of users that were registered as minors, that is under the age of 18, according to a report by TechCrunch.
As part of its investigation, the news outlet said it created more than six ChatGPT accounts with birth dates indicating various ages from 13 to 17. It also found that in some cases, ChatGPT even nudged these users to ask for more explicit content.
This comes just days after a similar investigation by The Wall Street Journal found that Meta’s AI chatbot on Facebook and Instagram engaged in sexual role-play with fictional characters in conversations with underage user accounts created by the business daily as part of its testing.
In its response to the TechCrunch report, OpenAI said that it was “actively deploying a fix” to restrict such content. “Protecting younger users is a top priority, and our Model Spec, which guides model behavior, clearly restricts sensitive content like erotica to narrow contexts such as scientific, historical, or news reporting. In this case, a bug allowed responses outside those guidelines, and we are actively deploying a fix to limit these generations,” an OpenAI spokesperson was quoted as saying.
According to OpenAI, its policies do not allow for such content to be shown to under-18 users. But in recent months, the Microsoft-backed AI startup has sought to shed some of the guardrails around ChatGPT and make its responses more permissive.
It appears that this may have led to the default AI model (GPT-4o) powering ChatGPT to more readily generate content that includes depictions of sexual activity, regardless of the user’s age.
Story continues below this ad
While OpenAI has said that it has patched the ChatGPT bug, it underscores ongoing concerns of online safety of teenagers, particularly given the company’s partnerships with various organisations like Common Sense Media to incorporate the chatbot into classroom learning.
OpenAI rolls back GPT-4o update
On Wednesday, April 30, OpenAI said it would be rolling back its latest update to GPT-4o following a string of user complaints about the chatbot’s extreme sycophancy. Posting screenshots on social media, users pointed out how ChatGPT has become overly validating and agreeable with users, even applauding problematic user inputs.
the last couple of GPT-4o updates have made the personality too sycophant-y and annoying (even though there are some very good parts of it), and we are working on fixes asap, some today and some this week.
at some point will share our learnings from this, it’s been interesting.
— Sam Altman (@sama) April 27, 2025
https://platform.twitter.com/widgets.js
“The last couple of GPT-4o updates have made the personality too sycophant-y and annoying (even though there are some very good parts of it), and we are working on fixes asap, some today and some this week,” OpenAI CEO Sam Altman said in a post on X.
“We started rolling back the latest update to GPT-4o last night. It’s now 100% rolled back for free users and we’ll update again when it’s finished for paid users, hopefully later today. We’re working on additional fixes to model personality and will share more in the coming days,” Altman added, without directly mentioning ChatGPT’s responses to underage users that reportedly contained sexually explicit material.
