Xander Song, an engineer at Arize AI working on Phoenix, introduces Harbor, an open-source framework for evaluating AI agents using real data in isolated sandboxes. He explains concepts such as tasks, Docker environments, verifiers, rewards, and ATIF trajectories, and demonstrates how his team runs Harbor at scale on Daytona and Phoenix to test prompt changes affecting SQL tool usage.
Frontier Data Summit 2026 Interview: UC Berkeley Professor and Director of Berkeley RDI, Dawn Song, joins Snorkel AI CEO Alex Ratner to discuss the "Agents' Last Exam" benchmark and cybersecurity evaluation tools like CyberGym. Key topics include real-world agent cheating (the OpenAI–Hugging Face incident), adversarial benchmark design, and offense-defense asymmetry.
Frontier Data Summit 2026 video: Alex Shaw, Founding Member of Technical Staff at Laude Institute and creator of Harbor, and Sanmi Koyejo, Director of the Stanford Trustworthy AI Research lab, discuss why AI evaluation needs a real science and how open infrastructure can keep benchmarks honest.
A talk from Frontier Data Summit 2026 by Steven Dillmann, a Stanford PhD student and Project Lead for Terminal-Bench-Science. The benchmark evaluates AI agents' capabilities in scientific research workflows across life, physical, Earth, mathematical, and engineering sciences, focusing on the scientist-led task review process and initial model results from the 0.1 release.