When AI Pushes Code Output Up 11x, Shopify Uses Attack Surface Maps to Break Security Reviews into Small Questions
Michael Hughes of the Shopify engineering team describes their approach for mobile apps: maintaining an attack surface map grouped by surface, writing OWASP standards and lessons from past incidents into updatable skill files, reviewing changed entries in the map daily, and reading through change files for omissions. This keeps security reviews manageable despite a throughput of approximately 434,000 lines of code changes per week.
During a 12-week rewrite of the Shopify mobile app, the team averaged approximately 434,000 lines of code changed per week, compared to about 39,000 lines per week at the same time last year. While code throughput increased 11-fold, security reviews cannot also increase 11-fold—at least not in the same way as before.
In a Shopify engineering blog post, Michael Hughes described their response strategy. The core insight is that asking a model to find vulnerabilities across an entire application yields a long, vague list; what truly works is narrowing the problem space to a range where the model can answer precisely.
The specific approach has three layers. The first layer is the attack surface map: for each app, they maintain a list of security-sensitive items grouped by surface—deep links, WebViews, stored data, permissions, dependencies, etc. For each surface, they write one or two sentences explaining why it matters. Each item includes a verdict (Fix, Tradeoff, Secure, Fixed, Debug only) and a link to the relevant code.
The second layer involves writing knowledge down as skill files. The starting point is OWASP's Mobile Application Security Verification Standard (MASVS) and Mobile Application Security Testing Guide (MASTG), supplemented with problem categories learned from past incidents and HackerOne reports. Each skill describes what to list, what good looks like, and what to check. These skills are not tied to any single tool: they are used for scheduled reviews, one-off security assessments, and can be loaded by any agent working in the code. When new knowledge is acquired, the skill is updated, and every subsequent run automatically inherits these improvements.