LesenForschungRadarAnlageframework
Anmelden / Registrieren
← Zurück zur Lektüre
NewsThema · HackerOneWesentlich2 Min. LesezeitDisconfirmAI Redaktion

HackerOne Benchmarks: GPT-6 Astra Finds 85.5% of Vulnerabilities, Ensemble Hits 98.4%, But AI Struggles with Business Logic Flaws

HackerOne's latest benchmark shows single frontier AI models discover most technical vulnerabilities, but multi-model ensembles are needed for near-full coverage; AI remains limited in verifying exploitability and identifying business logic flaws.

2026-11-10 17:37 GMT+8

Global bug bounty platform HackerOne has released research titled "Benchmarking the Frontier," evaluating 18 AI model variants on vulnerability discovery and verification tasks. The results show that OpenAI's GPT-6 Astra discovered 53 out of 62 known vulnerabilities in HackerOne's internal test environment "Celestia," achieving an 85.5% recall rate. When combining GPT-6 Astra, GPT-5.6 Sol, and Anthropic's Claude Mythos 5, the total discovery rate rose to 98.4% (finding 61 of 62 vulnerabilities).

However, finding a vulnerability is not the same as confirming its exploitability. In a separate experiment comparing models on public CVEs and web application reports, GPT-5.5 demonstrated advantages in verification speed, completing checks approximately three times faster than Claude Sonnet 4.6, with fewer false positives. Claude Opus 4.7 excelled at analyzing complex attack paths, such as correctly identifying that a Server-Side Request Forgery (SSRF) existed but did not lead to Remote Code Execution (RCE), thereby rejecting invalid reports. In contrast, GPT-5.5 showed inconsistency on identical inputs, requiring increased reasoning resources to match accuracy, which significantly extended analysis time.

HackerOne emphasized that optimizing agent toolchains yields greater performance gains than simply switching models. By adding code search tools and strengthening patch comparison workflows, verification accuracy across all models improved from over 80% to 98%. However, AI remains weak at identifying "business logic vulnerabilities," which do not involve code errors but abuse normal operational flows (such as duplicate coupon stacking). These require understanding service operations, where current AI reliability is low.

Anmelden / Registrieren
LesenForschungRadarAnlageframework

As AI generates a surge of low-quality vulnerability reports, submissions to the HackerOne platform have increased by 2.8 times since January 2025, while valid reports grew only 1.9 times. This forces security researchers to shift focus toward complex vulnerability analysis and manual verification. The backlog of unpatched vulnerabilities in enterprises increased by 131% over two years, despite the average remediation time shortening from 135 days to 62 days.

Quellen

  • byline.network바이라인네트워크↗
Thema

HackerOne

全球最大漏洞赏金平台 HackerOne(美国)发布的最新基准测试显示,尽管单一前沿 AI 模型如 GPT-6 Astra 能发现 85.5% 的技术漏洞,但多模型组合才能接近全覆盖(98.4%)。然而,AI 在判断实际攻击可行性及识别业务…

4 Schritte1 BeiträgeAktualisiert 2026-11-13 03:25

Wie es dazu kam

  1. 2026-11-10HackerOne 发布研究资料《Benchmarking the Frontier》,评估 18 个 AI 模型变体在 Celestia 环境中的漏洞探索能力,结果显示 GPT-6 Astra 单独运行发现率为 85.5%,三模型组合达 98.4%。byline.network ↗
  2. 2026-11-10HackerOne 公布 2026 年安全研究报告,指出自 2024 年以来,有效未解决漏洞积压量增长 131%,而关键漏洞平均修复时间从 135 天降至 62 天,形成“暴露债务”。hackerone.com ↗
  3. 2026-11-05OpenAI 推出 GPT-6 Cyber,宣称在 ExploitGym 基准测试中得分 48.5%,并在 Linux 代码库中发现 948 个权限提升利用程序,强调其在保持授权范围内的对齐能力。openai.com ↗

Was noch ungeklärt ist

  • AI 模型如何更有效地结合人类专家知识以识别复杂的业务逻辑漏洞?
  • 随着漏洞发现速度的加快,企业应如何重构其持续威胁暴露管理(CTEM)流程以消化激增的“暴露债务”?
  • 不同 AI 模型在特定任务(如广度扫描 vs 深度判断)上的最佳路由策略是什么?
Vollständiges Thema lesen →
Recherche fortsetzen →