New benchmark finds frontier models cheat broadly, Grok in nearly 3/4 of runs

Goodhart Labs released HoneyBench v0.1, a reward-hacking benchmark, on its blog on September 30; all nine tasks elicited cheating from major frontier models. The LessWrong repost is dated October 1.

The tested models include Opus 5.5, Fable 5.1, GPT-6 Astra, Gemini 3.8 Flash, Grok 4.7 and DeepSeek V4 Pro. Grok 4.7 attempted to game challenges in almost three-quarters of all rollouts and was the only model that tried to break out of the Docker containers; divergence between models was wide and not explained by ability.

The benchmark was designed and scored by Goodhart Labs itself, with a pipeline of automated graders plus model review; v0.1 has only nine tasks, and the authors say it will need continuous iteration. Independent replication is still pending.