Ai2 Releases XYBench to Test If Large Models Can Identify User Misconceptions and Redirect to Real Goals

Non-profit research institute Allen Institute for AI (Ai2), in collaboration with researchers from Carnegie Mellon University, NVIDIA, and Johns Hopkins University, has released XYBench, a new benchmark comprising over 8,000 questions designed to test how AI assistants respond to requests based on misconceptions. The benchmark targets the 'XY problem': a user wants to achieve goal X but assumes approach Y will work, thus asking for help with Y while ignoring the actual issue. For example, a user might ask how to keep frosting from sliding off cupcakes, when the real problem is that the cupcakes are too warm to frost.

The research team evaluated models including Claude Sonnet 4.6, GPT-5, Gemini 3.1 Pro, Qwen3.5-9B, and DeepSeek-V4 Pro. The results showed that while these models often suggested solutions, they tended to keep their answers focused on the user's original, mistaken request rather than steering them toward their underlying goal. Compared to human reference answers, all five models were less likely to steer users away from the mistaken approach and toward a solution suited to their goal.

XYBench measures model responses across three dimensions: pragmatic solution (whether it addresses both the stated request and the intended goal), pragmatic emphasis (whether the answer focuses on the stated request or the intended goal), and misconception identification (whether it discourages the mistaken approach and guides toward the correct one). Experiments indicated that even when the user's potential goal was explicitly provided, models still underperformed humans in misconception identification, and sometimes performed worse, suggesting that detecting intent is only part of the challenge; models must also act on it when it diverges from the literal request.