LecturaInvestigaciónRadarMarco de inversión
Iniciar sesión / Registrarse
← Volver a la lectura
NoticiasTema · Harbor基准框架Resumen rápido1 min de lecturaEditorial de DisconfirmAI

Stanford PhD Student Steven Dillmann Introduces Terminal-Bench-Science: A Benchmark for AI Agents in Scientific Research

A talk from Frontier Data Summit 2026 by Steven Dillmann, a Stanford PhD student and Project Lead for Terminal-Bench-Science. The benchmark evaluates AI agents' capabilities in scientific research workflows across life, physical, Earth, mathematical, and engineering sciences, focusing on the scientist-led task review process and initial model results from the 0.1 release.

2026-10-16 23:12 GMT+8

This resource is a talk video from Frontier Data Summit 2026, presented by Steven Dillmann, a Stanford PhD student and Project Lead for Terminal-Bench-Science.

Terminal-Bench-Science aims to evaluate AI agents' performance in real scientific research workflows, covering five major scientific domains: life, physical, Earth, mathematical, and engineering sciences.

The video covers the scientist-led task review process, examples and results from the 0.1 release, leaderboard results across models and domains, and a roadmap for how researchers can contribute tasks.

The project received support through Snorkel AI’s Open Benchmarks Grants. For researchers and developers interested in the actual capability boundaries of AI agents in scientific discovery, this video offers direct reference material on benchmark construction methods and preliminary evaluation results.

Fuentes

  • youtube.com↗
Tema
Iniciar sesión / Registrarse
LecturaInvestigaciónRadarMarco de inversión

Harbor基准框架

Harbor是由Laude Institute联合创始人Alex Shaw与Terminal-Bench团队共同开发的开源AI智能体评估框架。该框架旨在解决当前智能体评测中环境异构、集成复杂的问题,通过统一的适配器架构支持在隔离沙箱中并行运…

3 pasos4 piezasActualizado 2026-11-08 04:19

Cómo llegó hasta aquí

  1. 2026-10-08Frontier Data Summit在旧金山举行,Alex Shaw与Sanmi Koyejo发表关于“AI评估需要真正的科学”的主题演讲,强调开放基础设施对保持基准诚实性的重要性。frontierdatasummit.ai ↗
  2. 2026-10-18斯坦福博士生Steven Dillmann在Frontier Data Summit介绍Terminal-Bench-Science,展示科学家主导的任务审查流程及初步模型结果。youtube.com ↗
  3. 2026-10-27Arize AI工程师Xander Song演示Harbor框架,展示如何在Daytona和Phoenix上大规模运行Harbor以测试提示词变更对SQL工具调用的影响。youtube.com ↗

Lo que aún está pendiente

  • Harbor-Index作为精选的高难度任务集,其28%的最高通过率是否意味着当前最强模型在长周期复杂任务上仍存在显著瓶颈?
  • 随着更多第三方基准接入Harbor适配器,如何确保不同来源的评分逻辑(Verifier)在统一框架下的公平性与一致性?

También en este hilo

  • Arize AI Engineer Xander Song Introduces Harbor: Evaluating AI Agents in Isolated Sandboxes2026-10-27
  • UC Berkeley Professor Dawn Song Discusses AI Agent Cheating and Benchmark Security2026-10-18
Leer el tema completo →
Continuar investigando →