Google Releases Open-Source TorchTPU, Enabling No-Rewrite Migration of AI Apps to TPU Chips

On Tuesday, Google announced that TorchTPU, its long-awaited version of PyTorch, is now available as an open-source project. This move aims to help artificial intelligence application developers easily migrate their existing code from Nvidia GPUs to Google's self-developed Tensor Processing Units (TPUs) without rewriting applications. With a simple configuration change, developers can utilize Google Cloud's computing resources.

PyTorch is currently the most popular open-source AI development toolkit, estimated to be used by 85% of AI applications. The framework was originally developed by Meta in 2016, when Nvidia's GPUs were already the standard hardware for AI research, meaning many of PyTorch's features were designed with deep dependencies on Nvidia chips. In 2022, Meta transferred PyTorch to the Linux Foundation for management.

Google is not the only cloud provider attempting to break Nvidia's monopoly. Just on Monday, Amazon released a similar tool called TorchNeuron, which allows AI applications to seamlessly port to its self-developed Trainium chips and AWS platform. Additionally, Amazon has submitted code to the main PyTorch project to decouple specific features from Nvidia hardware bindings, making the framework default-compatible with more hardware.

Google stated that it worked with Meta, Hugging Face, and makers of popular AI development tools like vLLM and SGLang to ensure TorchTPU's features and code libraries are compatible with these ecosystems. Google also launched a dedicated TPU agent suite to assist developers in fixing errors during migration and updated its LiteRT tool to better support running PyTorch applications on mobile devices.