Nanjing University and SenseTime Release OpenMobile-2: App Diversity Boosts Smartphone Agent Generalization
OpenMobile-2 demonstrates that training on a wider variety of simulated apps improves performance on unseen tasks more effectively than simply increasing data volume.
Researchers from Nanjing University, AI company SenseTime, and several other universities have released the OpenMobile-2 project, aiming to close the performance gap between open-source smartphone agents and proprietary models. The project introduces two agent models built on Qwen3.5 (9 billion parameters) and Qwen3.6 (27 billion parameters), along with the full training environment, data, and instructions for reproduction.
The core finding is that for smartphone agents, 'app diversity' in training data matters more than sheer 'data volume.' Holding the amount of training data constant, simply increasing the number of different apps the agent practiced on raised the success rate on unseen applications from 60.7% to 67.5%. This result challenges the conventional intuition that more data always leads to better performance, highlighting that covering a broader range of task scenarios is critical for generalization.
To address the high cost and difficulty of resetting states when training on real apps, the team rebuilt simulated versions of 35 popular applications, including Taobao, JD, DiDi, Meituan, Slack, and Google Drive. Combined with existing simulations, this created a training environment covering over 110 applications. These simulated apps retain core workflows and interface logic but allow automatic state resets, enabling the agent to interact solely via screen screenshots, mimicking human usage.
In benchmark tests, the OpenMobile-2-27B model jumped from 67.1 to 79.9 points on AndroidWorld and improved from 35.9 to 50.4 on MobileWorld, a harder benchmark involving cross-app tasks. Notably, on SPA-Bench, which tests performance on real Android devices, the model scored 59.6 points, which researchers claim is the highest score among current open models. While reinforcement learning provided limited gains, it even caused slight score drops in some environments, suggesting that supervised fine-tuning within diverse simulated environments is already highly competitive.