読書リサーチレーダー投資フレームワーク
ログイン / 新規登録
← 読解に戻る
ニューストピック · 微软Scope评测平台重大2 分で読めるDisconfirmAI 編集部

Microsoft Open-Sources Scope to Evaluate and Trace AI Coding Agents

Microsoft has open-sourced Scope, an evaluation framework for AI coding agents like GitHub Copilot and Claude Code. It helps development teams verify correct SDK and API usage by logging execution steps and assessing results, addressing reliability issues where code works but doesn't follow intended workflows.

2026-10-20 00:00 GMT+8

On October 19, 2026, Microsoft open-sourced Scope, an evaluation platform designed to help development teams test and track the performance of AI coding agents such as GitHub Copilot and Claude Code when executing specified tasks. The platform allows software teams to verify whether agents correctly use Software Development Kits (SDKs), Application Programming Interfaces (APIs), and related tools according to documentation, while recording the entire operation process and test results.

In practice, AI agents assisting developers can make mistakes such as selecting the wrong API, ignoring important instructions in documentation, or failing to complete work after encountering errors. Even if the final generated code runs successfully, it does not guarantee that the agent used the expected toolchain. Scope addresses this pain point by enabling teams to test error-prone aspects of agent interactions with proprietary SDKs, command-line tools, and Model Context Protocol (MCP) servers used to connect external services.

Development teams can define completion criteria for tasks and provide starting code, allowing different AI agents to execute the same task. The system records the AI model version, agent version, and tool configuration for each run. After modifying documentation or tools, teams can re-test with the same task to observe if agents complete work more easily and distinguish between sporadic and recurring errors. Additionally, Scope supports setting multiple check conditions with dependencies (e.g., successful build before checking SDK initialization) and uses large language models to assess condition fulfillment, preserving operation logs and directory snapshots for troubleshooting.

ログイン / 新規登録
読書リサーチレーダー投資フレームワーク

Despite automated assessment, Microsoft notes that AI agents may produce different results even with identical settings, and automatic judgments can be erroneous. Therefore, the company recommends repeated testing and human review of records. Currently, Scope requires user deployment, offering a web management interface and API integration options. The project is MIT-licensed, source code is available on GitHub, and it remains in a research preview stage.

ソース

  • opensource.microsoft.comMicrosoft Open Source Blog↗
トピック

微软Scope评测平台

微软于2026年10月19日开源了Scope,这是一个基于Kubernetes的AI编程代理评测平台,旨在解决开发者体验(DevEx)向智能体体验(DevAx)转型中的可靠性痛点。与传统的排行榜不同,Scope通过记录GitHub Copi…

4 ステップ1 件更新日 2026-10-22 21:01

経緯

  1. 2025-08-01微软开发者关系团队开始构建Scope原型,以自动化评估AI代理行为。opensource.microsoft.com ↗
  2. 2026-10-19微软正式开源Scope平台,发布官方博客介绍其架构、MIT许可证及Azure Cosmos DB的应用案例。opensource.microsoft.com ↗
  3. 2026-10-20第三方媒体AGI Pulse报道Scope发布,指出其支持本地Docker Compose部署及不可变版本追踪功能。agipulse.org ↗

未解決の問題

  • 大模型裁判(Judge)在评估复杂逻辑或创造性代码时的偏差如何量化和控制?
  • 除了Azure Cosmos DB,其他SDK或CLI工具采用Scope后是否会出现类似的显著行为改变?
  • 社区贡献者如何利用Scope的开源特性扩展新的评估标准或集成更多类型的AI代理?
トピック全文を読む →
リサーチを続ける →