Tan Pei Xiang on Twinkle: Atomic Architecture for LLM Training
In a video on the official PyTorch channel, Tan Pei Xiang introduces Twinkle, an LLM training framework co-developed by ModelScope and China Merchants Bank. The episode highlights its atomic design and serverless TaaS architecture aimed at resolving flexibility-scale trade-offs in RL loops.
ImportanceLocalEvidenceE2 unreplicatedWrite-upQuick
Tan Pei Xiang introduces Twinkle, an LLM training framework co-developed by ModelScope and China Merchants Bank, in a video published on the official PyTorch channel. The presentation focuses on Twinkle's "Atomic Design," which allows users to orchestrate reinforcement learning (RL) loops in approximately 150 lines of code. This modular approach enables swapping components like Reward or Loss functions without altering the core system.
The framework utilizes a C-S Architecture that decouples algorithm semantics from GPU orchestration methods such as torchrun, Ray, and HTTP. It also features a Serverless Training-as-a-Service (TaaS) model, where multi-tenant LoRA pools support over eight concurrent tasks on a single base model to maximize resource utility. As a superset of the Tinker API, Twinkle supports FSDP2 and Megatron backends. The episode explores its architecture, rapid GRPO/DPO integration, and case studies in the financial sector, concluding with a discussion on the shift towards "micro-service training" for Agent RL.