Together AI Unveils 'Polymorphic Inference' Framework, Claims Compiler Already Optimizing Production Models
Together AI introduces 'Polymorphic Inference' and announces its AI-agent-driven compiler, Together Compiler, is running in production to adapt to rapidly changing models and hardware.
AI infrastructure company Together AI published a blog post on November 4, 2026, formally introducing a new paradigm called "Polymorphic Inference" and announcing the deployment of its tool, Together Compiler, in parts of its production environment. The tool uses AI agents to automatically optimize inference performance, aiming to address the challenges posed by the rapid iteration of open-source model architectures, traffic patterns, and hardware platforms.
The research team at Together AI argues that the traditional "Static Inference" model relies on manually tuned kernels for specific configurations, which struggles to keep up with the current pace of open-source model releases, increasing architectural diversity, and frequent hardware updates. According to the company, 80% of traffic on OpenRouter goes to models released in the last three months, and the top ten most popular open-source models all use different architectures. The core of Polymorphic Inference is enabling systems to dynamically adapt to these changes rather than writing new static kernels for every scenario.
As a first step toward this goal, Together AI launched Together Compiler. Described as a "recursive self-improvement" (RSI) system, it is an AI-agent-driven performance optimization loop. The company claims that many models currently running through its endpoints are fully optimized by AI agents, calling it "the world's first production-scale recursive self-improvement (RSI) inference system." Together Compiler reframes the traditional compiler translation problem as an agentic search problem, using a swarm of agents paired with a high-fidelity persistent engine to automatically identify key optimization patterns and continuously adjust code to fit new traffic shapes and workloads.
It is important to note that the claims regarding being the "world's first" and the status of production deployment are self-reported by the vendor, with no independent benchmark data or specific performance improvement percentages provided. Together Compiler is currently available only to limited beta users. For readers tracking AI infrastructure efficiency, this move indicates that leading service providers are attempting to reduce reliance on manual kernel optimization through automation, though actual effectiveness remains subject to future independent verification.