OpenAI Plans to Train Models on Previously Off-Limits Daybreak Blue Cybersecurity Session Data
OpenAI plans to start using Daybreak Blue cybersecurity session data for model training on November 29, reversing previous exclusions. After backlash, the company promised a separate opt-out option, raising concerns about a repeat of the Buckmaster-Alpöge dispute over unpublished research.
According to The Decoder, OpenAI plans to begin using session data from its Daybreak Blue cybersecurity product for model training starting November 29. The change was confirmed via an email to members of the Trusted Access program. Previously, this data was excluded from training even when users had turned off the "improve the model for everyone" setting.
The backlash was swift. OpenAI followed up with a second email promising a separate opt-out mechanism. However, the report notes that interactions flagged during safety reviews can still be used for safety purposes.
The move has researchers worried about a repeat of the Buckmaster-Alpöge case. In that incident, mathematicians Tristan Buckmaster and Levent Alpöge uploaded Navier-Stokes equation drafts to OpenAI's Codex. OpenAI soon presented its own proof but denied using their inputs for training. Yet the suspicion alone was enough to show researchers what could go wrong.