OpenAI limits weapons-related information provided to teenagers through ChatGPT

OpenAI will tighten, in the coming weeks, restrictions on weapons- and security-system-related information provided to teenagers through ChatGPT, in an effort to prevent real-world violence, Reuters reports.

The new rules will limit users under 18 from accessing practical information about weapons and security systems that could be used for dangerous purposes, including when the questions do not clearly indicate violent intent. The measure comes amid concerns about the risks that artificial intelligence-powered chatbots may pose to minors.

For all users, ChatGPT may automatically end conversations in which a persistent preoccupation with violence is identified. Accounts belonging to people who repeatedly engage in such discussions could be temporarily suspended, and parents will be informed if their child’s account is suspended for this reason.

OpenAI will also improve its age-estimation system. Users identified as probably being under 18 will be transferred to the version intended for teenagers, which includes additional safeguards.

The company will also reduce the level of detail provided about past mass shootings, including the weapons used and how they were obtained. The measures will be implemented gradually and adjusted later.

The announcement comes amid increased scrutiny of artificial intelligence companies. A report published by OpenAI at the beginning of October showed that teenagers use ChatGPT, on average, for less than 15 minutes a day. Separately, Common Sense Media found that the teen version sometimes warned parents late or did not warn them at all in cases involving suicide, self-harm, or eating disorders.