Structural Gaps in the AI Safety Tool Ecosystem: Only ~80 of 500+ Organizations Build Tools, None of 100 Entry Paths Train Builders
LessWrong author ankurpandey42 conducted a systematic mapping of the AI safety ecosystem, finding that only about 80 of 500+ safety organizations build tools and these are concentrated in security-adjacent areas. Of the 100 fellowship and course entry paths mapped, none are dedicated to training tool builders. The author argues the problem is not that tools aren't being built, but that they lack generalizability, adoption and deployment scale, and advocates treating AI safety tools as public goods.
ImportanceMaterialEvidenceE3 inspectableWrite-upStandard
The AI safety field currently has more than 500 organizations employing roughly 2,000 full-time equivalents, averaging about four per organization. But in a systematic mapping, LessWrong author ankurpandey42 found that only about 80 of these build any kind of tool, and most of those are concentrated in security or security-adjacent areas—LLM red-teaming, model security, agent infrastructure security, and similar domains. More critically, among the 100 mapped fellowship and course entry paths, not one is dedicated to AI safety tool builders.
The author argues the core problem is not that tools aren't being built. Many safety organizations do produce tools, and the outputs of research fellowships are often tools. However, these are mostly designed for internal, hyper-specific use cases without generalizability in mind. The author's thesis can be summarized as: if we take AI progress and its implications seriously, we need generalizable, adopted and deployed AI safety tools as public goods. A push toward tool development offers at least three benefits—accelerating safety research itself, increasing affordances (enabling people to apply safety use cases in novel, creative ways), and allowing non-researchers such as engineers and builders to contribute to the field.
In distinguishing tools from products, the author explicitly prefers the term 'tools' over 'products.' Products do have advantages: they attract venture capital, are better suited for large-scale operations and deployment, force teams to stay close to real needs, and sometimes offer more independence than relying on governments or grantmakers. But the author notes that for-profit AI safety startups are for-profit first and AI safety second—extremely easy to get pulled sideways—and the market doesn't necessarily price in good outcomes until it's too late. The author therefore chooses the broader term 'tools' to avoid attaching specific incentive assumptions.