OpenAI Performance Factory: Independent Agent Rebuttals Improve Root-Cause Accuracy
OpenAI engineers Okay Zed and Cyrus Forbes detail their Performance Factory system, explaining how early agents sometimes stopped at the first plausible explanation to provide incorrect diagnoses, and how introducing an independent agent specifically tasked with rebutting those diagnoses intercepts these errors—achieving a current root-cause accuracy of 70 out of 100 cases, though fixes remain harder to implement than diagnoses.
Okay Zed and Cyrus Forbes, performance engineers at OpenAI, published an engineering blog post in November 2026 explaining how their Performance Factory system uses Codex agents to automatically investigate and fix performance regressions in ChatGPT. The most valuable part of the article is not the system itself, but a workflow design issue it reveals: investigation agents sometimes stop at the first plausible explanation they find and issue incorrect diagnoses, whereas separating investigation and questioning into distinct agent turns can intercept these errors.
The origin of the problem was specific. In earlier versions, agents would sometimes identify a plausible-sounding explanation, mark the cause as found, and later prove to be wrong. The article provides a clear failure case: one agent traced a ChatGPT conversation failure to an image download error and marked the cause as found. However, an independent Codex reviewer noticed that the internal service responsible for locating files had already recovered while image requests were still failing—the diagnosis could not explain the persistent failures. The reviewer reopened the investigation and recorded what remained unestablished.
Their solution is adversarial review: a separate Codex session dedicated to attempting to disprove the proposed diagnosis. The reviewer rechecks assumptions and tests counterfactuals—if a rollout caused the slowdown, did similar requests not exposed to that rollout stay healthy? If the suspected change was rolled back, did the affected metrics recover? This is not simple repetition; it involves challenging the same conclusion from different angles.