ReadingResearchRadarInvestment framework
Sign in / Sign up
← Back to reading
NewsTopic · OS-Helix智能体1 min read·2026-10-16

Preprint: OS-Helix Unifies GUI and Code Execution, Reaching 66.44% Success in Desktop Automation

New research shows adding code tools naively degrades GUI agents, requiring hybrid training for native coordination. OS-Helix achieves significant breakthroughs on the OSWorld benchmark.

ImportanceMaterialEvidenceE2 unreplicatedWrite-upQuick

2026-10-16 12:00 GMT+8

OS-Helix achieved a 66.44% success rate on the OSWorld-Verified benchmark, representing a 42.9% relative improvement over matched Qwen3.5 hybrid baselines. These results come from a preprint submitted on October 15, 2026, titled "OS-Helix: Unifying GUI Interaction and Code Execution in Native Computer-Using Agents" by Bowen Yang et al.

Traditional approaches attempt to combine GUI operations and code execution tools through prompt engineering, but the study reveals that such "prompt-level tool composition" cannot achieve effective native coordination. The authors note that simply adding a code tool to a GUI agent actually degrades its performance, indicating that GUI/code routing decisions must be internalized through end-to-end training on high-quality hybrid data.

To address supervision challenges, the team introduced "application-artifact composition," pairing desktop applications with authentic file artifacts to instantiate executable Ubuntu states. They also employed a decoupled instruction/evaluator generation pipeline to ensure task final states could be verified regardless of the solution modality. The model was cold-started with SFT and further improved through online reinforcement learning in live desktop environments.

These results are from a first-party benchmark (E2) and have not yet been independently reproduced. The core contribution lies in demonstrating that simple tool chaining is insufficient for complex desktop automation tasks, necessitating specialized training on a unified action space.

Sign in / Sign up
ReadingResearchRadarInvestment framework

Sources

  • 原始出处arxiv.org↗
Topic

OS-Helix智能体

上海人工智能实验室等机构发布OS-Helix,一种统一图形界面(GUI)交互与代码执行的原生计算机使用智能体。该研究指出,简单地将代码工具附加到GUI智能体上会导致性能下降,必须通过混合数据训练实现原生协调。

2 steps1 piecesUpdated 2026-10-16 22:39

How it got here

  1. 2026-10-15论文《OS-Helix: Unifying GUI Interaction and Code Execution in Native Computer-Using Agents》提交至arXiv,作者来自上海AI Lab、中科大等机构。
  2. 2026-10-16预印本公开,显示OS-Helix在OSWorld-Verified上达到66.44%成功率,并揭示了朴素添加代码工具会降低GUI智能体性能的现象。

What is still unsettled

  • OS-Helix在真实生产环境中的端到端延迟表现如何?(对比OSWorld-Human指出的高延迟问题)
  • Step-balanced GSPO算法在其他非桌面自动化任务(如Web导航)中的通用性如何?
  • 混合数据训练的“应用-工件组合”方法能否低成本扩展到更多操作系统和应用程序?
Read the full topic →
Continue researching →