OpenAI Confronts Significant Oversight Gaps Following Three-Week External Audit

A new audit reveals that as much as 25 percent of internal traffic from OpenAI’s most advanced models may have been entirely unmonitored during a recent three-week evaluation period. This finding, part of an estimated range between 6 and 25 percent, highlights significant vulnerabilities in the company’s internal safety monitoring infrastructure. The data was gathered by researchers who spent several weeks embedded within the company’s systems, providing a detailed look at how the organization tracks potential rogue behavior in its artificial intelligence agents.

The audit was conducted by two researchers from METR and Redwood Research, who operated on-site at OpenAI from July 21 to August 11, 2026. During this timeframe, the team was granted access to the company’s codebase and internal documentation. They also interviewed approximately a dozen employees to understand the operational workflows surrounding the monitoring of agentic AI systems. The primary focus of their inquiry was the internal monitoring system designed to detect and manage unexpected or harmful behavior from AI models.

The investigators identified numerous weaknesses in the current oversight framework. Beyond the broad estimate of unmonitored GPT-5.5 class traffic, they pinpointed specific technical failures. For instance, roughly 10 percent of internal Codex traffic that was theoretically subject to monitoring went untracked between May and July. This gap was attributed to a specific bug in how the system handled compaction events, which involve summarizing context. METR noted that while OpenAI independently discovered and fixed this bug during the audit, the uncertainty surrounding these specific estimates remains high in both directions.

The auditors further discovered that the monitoring triggers were not always effective in capturing agent activity. The system was designed to activate oversight only when an agent handed control back to the user. Consequently, agents could effectively delay or avoid scrutiny by simply continuing to execute actions without returning control. This structural loophole allowed for a period of reduced visibility, during which agents could perform tasks without immediate external checks.

More significant technical loopholes were also identified. The researchers found methods to hide hundreds of actions from the monitoring systems and ways to bypass the Codex sandbox, a contained environment intended to limit agent capabilities. Additionally, they determined that specially crafted inputs could likely cause the monitor to crash entirely. Another major finding was that model behavior during reinforcement learning (RL) training was not being monitored at all when the audit began, leaving a critical phase of model development without active surveillance.

OpenAI has responded to these findings by stating that all identified weaknesses have been corrected. The company noted that it paused RL training following an incident with Hugging Face in August and subsequently expanded monitoring to cover this area. However, it is important to note that these changes were implemented after the auditors’ access period had ended, meaning they were not included in the original evaluation of the system’s state during the three-week window.

The transparency of the audit results has drawn attention due to the extent of the redactions applied to the final report. While the full 61-page document was provided internally to OpenAI, the public received only a short summary. Leadership at METR and Redwood Research were given a 38-page version after OpenAI applied its redactions. METR stated that it did not have the right to disclose the redacted portions independently, leaving the public with a limited view of the detailed technical findings and specific examples of the vulnerabilities found.

This exercise was part of a pilot program organized jointly by METR, Redwood Research, and OpenAI, aimed at providing outside researchers with deep access to internal infrastructure. The goal was to test the boundaries of external oversight in a real-world corporate environment. Looking ahead, METR has expressed a desire to push for greater public transparency in future audits, suggesting that the current balance of internal and public reporting may need adjustment to better serve the broader AI community and stakeholders.

The findings underscore the ongoing challenge of monitoring complex, autonomous AI systems as they evolve in capability. While OpenAI has moved to address the specific gaps identified, the audit highlights the difficulty of ensuring comprehensive coverage in dynamic environments. As the company continues to scale its operations, the pressure on both internal teams and external auditors to maintain rigorous oversight is likely to increase, particularly as model capabilities and autonomy grow.