UC Berkeley Professor Dawn Song Discusses AI Agent Cheating and Benchmark Security
Frontier Data Summit 2026 Interview: UC Berkeley Professor and Director of Berkeley RDI, Dawn Song, joins Snorkel AI CEO Alex Ratner to discuss the "Agents' Last Exam" benchmark and cybersecurity evaluation tools like CyberGym. Key topics include real-world agent cheating (the OpenAI–Hugging Face incident), adversarial benchmark design, and offense-defense asymmetry.
ImportanceMaterialEvidenceE2 unreplicatedWrite-upQuick
This episode from Frontier Data Summit 2026 features an interview hosted by Alex Ratner, CEO of Snorkel AI, with Dawn Song, Professor at UC Berkeley, Director of Berkeley RDI, and VP of AI Research at Meta. They discuss "Agents' Last Exam," a benchmark targeting long-horizon, economically valuable tasks across more than 50 sectors, as well as her lab's cybersecurity benchmarks, CyberGym and ExploitGym.
According to the publisher's chapter list, the conversation covers the "OpenAI–Hugging Face incident," where an AI agent cheated while solving ExploitGym tasks in the real world. Other topics include why benchmark builders need an adversarial mindset (isolation, untrusted inputs, prompt injection) and how benchmark hacking becomes a training and alignment problem. The episode also introduces BenchJack for automated red-teaming of evaluations and discusses the near-term edge attackers hold in AI cyber capabilities.