ReadingResearchRadarInvestment framework
Sign in / Sign up
← Back to reading
News3 min read·2026-10-15

OpenAI Engineer Rodrigo Roim Shares Practices and Prompts for Cleaning Codebases with GPT-6 Family Models

An engineer from OpenAI's Core Experimentation team, Rodrigo Roim, details the use of GPT-6 Astra and GPT-6.1 Sol within Codex for code maintenance tasks, including dead code removal, stale experiment cleanup, and CI optimization. The article provides specific prompt examples, comparative cost and accuracy data (e.g., GPT-6.1 Sol Ultra achieving 92% of Astra's findings at 14% of the cost), and quantified benefits from over 1,400 merged PRs (such as a 48% reduction in container image size), while emphasizing the critical role of human review and prioritization in automated cleanup.

ImportanceMaterialEvidenceE3 inspectableWrite-upQuick

Verified 2026-10-15 14:54 GMT+8

As codebases grow, maintenance tasks such as removing unused code, cleaning up completed experiments, and fixing flaky tests become increasingly burdensome. Traditionally, engineering teams have relied on complex static analysis systems to detect dead code, but these systems often struggle with edge cases like reflection calls or dynamically enabled paths, leading to accidental deletions. An engineer from OpenAI's Core Experimentation team, Rodrigo Roim, recently shared their practices for using GPT-6 series models in the Codex environment to perform more intelligent, context-aware code maintenance.

The team initially tested GPT-6 Astra on vague dead code cleanup requests. Unlike traditional tools requiring precise instructions, Astra autonomously investigated Slack discussions, internal project trackers, and production performance profiles to gather evidence and justify each deletion. This capability allowed five engineers to merge over 1,400 pull requests and delete more than 800,000 lines of code during two weeks of spare-time collaboration. These cleanups yielded measurable infrastructure benefits, such as reducing a service's container image from 3.31GB to 1.72GB (a 48% decrease) and removing approximately 400 visual snapshot variants from the test suite.

Sign in / Sign up
ReadingResearchRadarInvestment framework

Addressing the harder problem of cleaning up stale experiments, where static analysis fails due to runtime configuration dependencies, Roim designed a prompt with safety guardrails. It required Astra to verify live Statsig configurations, check Git history, and preserve production behaviors acting as kill switches. Astra adhered to these constraints and generated a detailed Markdown report grouping candidates by CPU cost and ownership. In one case, Astra identified a seemingly innocuous line of Python code mapping a feature flag to all others, causing significant CPU fan-out. Although flagged as 'Permanent,' Astra submitted it for human review, which led to its removal and an estimated 0.42% CPU saving in the affected services.

To evaluate cost-efficiency, the team compared GPT-6 Astra and the newer GPT-6.1 Sol on identical subdirectory audit tasks. Data showed that GPT-6.1 Sol Ultra produced 92% of the findings of Astra Ultra but at only 14% of the API cost ($15.12 vs $109.06). However, Sol missed some constants used as dependencies for other feature flags, indicating specific error patterns across models. By adjusting prompts to correct these omissions, the team demonstrated that lower-cost models could achieve comparable effectiveness, though they noted that future model releases might require revisiting these prompt optimizations.

Beyond code cleanup, the practice extended to CI optimization and flaky test fixes. A simple prompt helped the team submit about 30 CI optimization PRs, reducing the median CI completion time from 10 minutes to 4.5 minutes. Another example showed that adding a missing mock reduced the runtime of a resolver suite containing 63 tests from 592.6 seconds to 29.8 seconds, a 95% reduction.

The author emphasizes that while GPT-6 series significantly improved maintenance efficiency, not all proposals were reasonable, with some even increasing complexity. Therefore, prioritization and measurement remain core parts of the workflow. For engineers looking to try this approach, the recommendation is to start with small maintenance tasks, compare different models and effort levels, and incorporate human review feedback into subsequent prompts at each step before scaling to the entire codebase.

Sources

  • 原始出处developers.openai.com↗
Continue researching →