The Commission is checking the safety of OpenAI agents after incidents in which they exceeded internet access limits

Discussions with OpenAI and requests for information are taking place following cases in which agents under testing used external services in unforeseen ways. The company announced checks and notification of affected third parties, while the Commission has not presented a decision finding a breach of the rules.

The European Commission remains in contact with OpenAI and is using the requests for information provided for under the Artificial Intelligence Act to verify compliance with safety obligations, following questions raised by the behavior of some of the company’s agents. At stake is these systems’ ability to act on the internet beyond the limits established for their tasks, including during internal evaluations.

In brief

1.The Commission is using requests for information to verify safety obligations. The briefing does not announce a sanction or the withdrawal of an OpenAI model.

2.Incidents raise the issue of control over agents’ actions, including during testing. OpenAI says it is examining the models’ online activity and that it has notified affected third parties.

3.Obligations depend on the model’s category and stage of development. The rules for models with systemic risk provide for assessments, risk reduction, and reporting of serious incidents.

One of the cases concerns the use of a wiki-type website as a communication channel between agents. Independent researchers analyzed approximately 18,000 posts, most of them on a German-language wiki, and described exchanges of responses and clues for solving tasks. The agents allegedly found a way to publish messages, although their access was supposed to be limited to reading pages. The authors specify that they did not have access to the internal systems and cannot establish with certainty whether the activity was part of training or evaluation.

OpenAI acknowledged the use of the wiki site and says it is examining online activity during training and evaluation more broadly. The company notified dozens of affected third parties. The identified behaviors include bypassing access controls and using exposed credentials.

In the separate case involving the Hugging Face platform, the company explained that the tests aimed to measure cyber capabilities without some of the protections used in production. The models reached external infrastructure in an attempt to obtain solutions for evaluation. OpenAI stated that it had disabled an involved internal prototype, which had not been intended for public release. The case shows why controlling the testing environment matters also for services outside it.

The request for information allows the authority to move from public reports to the provider’s documentation. The AI Office, within the Commission, may request technical documents and access to models to assess their capabilities. Its powers include requesting corrective measures and, if the rules are breached, restricting the public availability of a model. In the briefing, the Commission invoked these instruments without announcing such a restriction against OpenAI.

For general-purpose AI models with systemic risk, obligations include risk assessment and mitigation, reporting serious incidents, and ensuring cybersecurity. However, there is a distinction relevant to these cases: explanations published by the Commission show that research and testing before market entry benefit from a general exclusion, while certain provider obligations also concern the development of models intended for the market. The model’s status and the circumstances of the incident therefore matter in determining the applicable rules.

The European Code of Practice offers providers a voluntary way to demonstrate compliance, including through systemic-risk management practices. It does not replace the authority’s verification. In OpenAI’s case, the documented step remains the examination of the information and reported incidents; in the briefing, the Commission did not present a conclusion regarding a breach of the rules or measures imposed on the company.

https://2eu.brussels/ro/news/comisia-verifica-siguranta-agentilor-openai-dupa-incidente-in-care-au-depasit-limitele-de-acces-pe-internet