LessWrong Author Matthew Khoriaty: How AI Marketplaces Could Profit From Pausing Frontier Model Development

In an essay titled "AI Marketplaces Can Contribute to, and Profit From, the AI 2040 Plan," LessWrong author Matthew Khoriaty presents a counterintuitive thesis: the long-term commercial interests of AI marketplaces like OpenRouter may align with policies that restrict the development of frontier artificial intelligence. Khoriaty argues that if AI rapidly advances to superintelligence, these intermediary platforms could become irrelevant, as superintelligences would manage their own compute resources without needing third-party routing services.

Based on this premise, Khoriaty suggests that AI marketplaces have a strong incentive to support and prepare for scenarios involving a "pause" or slowdown in frontier model development. He references the "AI 2040" plan by the AI Futures Project, which advocates for international treaties to halt uncontrolled frontier development while encouraging widespread economic integration of AI. In such a scenario, marketplaces could become crucial intermediaries connecting users to diverse, low-cost, and personalized AI models, thereby gaining significant bargaining power and market share.

The article details specific actions marketplaces could take to solidify their position in a controlled AI ecosystem. First, platforms could incentivize model providers to open-source their inference setups, enhancing transparency and scientific reproducibility while lowering switching costs for users. Second, platforms could leverage their data to track compute usage, potentially installing verification devices to ensure chips are used only for inference rather than training—a critical component for potential regulatory treaties. Additionally, Khoriaty recommends implementing stricter Know-Your-Customer (KYC) protocols and risk-tiered access to prevent misuse, alongside monitoring Chain-of-Thought legibility to enhance safety.