ReadingResearchRadarInvestment framework
Sign in / Sign up
← Back to reading
NewsMaterial3 min readDisconfirmAI Editorial

When AI Pushes Code Output Up 11x, Shopify Uses Attack Surface Maps to Break Security Reviews into Small Questions

Michael Hughes of the Shopify engineering team describes their approach for mobile apps: maintaining an attack surface map grouped by surface, writing OWASP standards and lessons from past incidents into updatable skill files, reviewing changed entries in the map daily, and reading through change files for omissions. This keeps security reviews manageable despite a throughput of approximately 434,000 lines of code changes per week.

2026-11-19 04:34 GMT+8

During a 12-week rewrite of the Shopify mobile app, the team averaged approximately 434,000 lines of code changed per week, compared to about 39,000 lines per week at the same time last year. While code throughput increased 11-fold, security reviews cannot also increase 11-fold—at least not in the same way as before.

In a Shopify engineering blog post, Michael Hughes described their response strategy. The core insight is that asking a model to find vulnerabilities across an entire application yields a long, vague list; what truly works is narrowing the problem space to a range where the model can answer precisely.

The specific approach has three layers. The first layer is the attack surface map: for each app, they maintain a list of security-sensitive items grouped by surface—deep links, WebViews, stored data, permissions, dependencies, etc. For each surface, they write one or two sentences explaining why it matters. Each item includes a verdict (Fix, Tradeoff, Secure, Fixed, Debug only) and a link to the relevant code.

The second layer involves writing knowledge down as skill files. The starting point is OWASP's Mobile Application Security Verification Standard (MASVS) and Mobile Application Security Testing Guide (MASTG), supplemented with problem categories learned from past incidents and HackerOne reports. Each skill describes what to list, what good looks like, and what to check. These skills are not tied to any single tool: they are used for scheduled reviews, one-off security assessments, and can be loaded by any agent working in the code. When new knowledge is acquired, the skill is updated, and every subsequent run automatically inherits these improvements.

Sign in / Sign up
ReadingResearchRadarInvestment framework

The third layer breaks reviews into small units. Each skill first performs a complete inventory—every deep link route and its parameters, every WebView and its hosted domains, every WebView JavaScript interface method, every Keychain and Keystore entry, and every dependency with its version—and then reviews them one by one, looking only at the code involved in that specific item. For a deep link, this means examining one route, every parameter it reads, and the code that reads each parameter, then asking: What happens if the parameter is malicious? The answer must be specific enough to be verifiable.

The review rhythm runs once every workday: the latest commit's inventory results are compared with the last accepted map; an item counts as 'changed' only if the cited code line changed or the inventory reported a change. Then, the model judges each changed or new item using read-only tools and code only. A second pass reads through change files to find any security-relevant content not covered by the map. Finally, a new map and a change page are built, and a summary is posted to Slack.

Beyond the map, there is external monitoring: agents regularly read security news, checking each announcement against the map and code, notifying the team only when something affects their own app. The same skills are also used for pre-adoption reviews of third-party libraries, generating maps and verdicts for dependencies identical to those applied to their own code.

This method runs parallel to Dispatch, Shopify’s agentic vulnerability scanner. Dispatch finds and proves vulnerabilities in changed code, while the attack surface map ensures the overall application remains small and adheres to best practices. Every vulnerability found and fixed strengthens the checks—their way of building secure mobile apps in the agentic era.

Boundaries to note: all throughput figures and process effectiveness metrics are self-reported by Shopify, with no independent measurement or controlled experiments. The method applies to mobile apps where surfaces can be enumerated; in areas where the attack surface is difficult to fully inventory, the maintenance cost of the first-layer map may alter the feasibility judgment of the entire approach.

Sources

  • shopify.engineeringShopify↗
Continue researching →