OpenAI Monitoring System Flaws Found by METR and Redwood in Pilot Test

Nonprofit evaluators METR and Redwood Research announced on Thursday that they uncovered several "novel vulnerabilities" in OpenAI's internal misalignment monitoring system during a three-week pilot review, including an exploit that would allow AI agents to block "hundreds" of recent actions from being monitored. During the pilot from July 21 to Aug. 11, one employee from each nonprofit received chaperoned on-site access to OpenAI's internal codebase, documentation, and select technical staff using company-issued devices that never left the premises. The evaluators compiled a 61-page internal report for OpenAI and agreed to release only a high-level public summary because the pilot was not structured to provide broad public transparency, while OpenAI told the groups it has since patched the identified flaws affecting its production safeguards. The test evaluated OpenAI's monitoring system as it existed on and before July 21, precedes CEO Sam Altman's September pledge to grant independent evaluators "employee-like access," and comes as OpenAI stated earlier in October that it is actively finalizing contracts with external safety assessors. "This collaboration builds on our longstanding commitment to pioneering new approaches to third-party testing, and we're using what we learned to strengthen our safeguards and shape future collaborations with independent evaluators." — OpenAI Spokesperson