ChipAgents Develops Build-A-Chip-Bench with Nvidia to Evaluate AI Agents on Full Chip Design Workflow
New benchmark shows best AI agents pass hard gates on only 65% of tasks, indicating end-to-end automation remains early-stage.
Semiconductor engineering firm ChipAgents is developing a new benchmark suite called Build-A-Chip-Bench, with contributions from Nvidia to the scoring rubric. The benchmark aims to evaluate AI agents' ability to complete the full chip-design workflow, from workload specification through to GDS layout, using open-source tools and an open-source PDK.
Unlike previous benchmarks that tested isolated steps such as writing RTL code from a specification, Build-A-Chip-Bench requires each design to pass a "hard gate" covering functional correctness against hidden verification IP, routability, hold timing, and DRC (Design Rule Check). Only designs passing all hard checks are scored on Power, Performance, and Area (PPA) constraints.
According to ChipAgents, the best-performing AI agent currently passes the hard gate on only 65% of tasks, achieving an overall score of 16.6%. Frontier models struggle to revisit earlier decisions, protect working behavior while optimizing, and plan around slow tool runs. These results highlight the limitations of current AI in complex engineering tasks, providing important context for investors and engineers tracking AI adoption in the EDA sector.