Metastone Releases TokenFleet Scheduling Framework: Claims 32.4% Reduction in P95 First-Token Latency for GLM Tests

On October 20, 2026, Chinese compute operator Metastone released a distributed scheduling framework named TokenFleet at the CCF National High Performance Computing Academic Conference. The framework aims to address the simultaneous occurrence of request queuing and resource idling in GPU clusters by optimizing request routing, cross-datacenter capacity coordination, and mixed-load governance to improve effective output from existing computing power.

TokenFleet comprises three core technologies: TokenPilot handles intelligent inference routing by estimating both cache reuse benefits and queuing costs during instance selection; FleetMind focuses on cross-datacenter capacity coordination by aggregating model availability and network status across sites to select execution locations; and SLOMax manages production according to service requirements through classified queues and capacity feedback for mixed loads.

According to tuning data provided by Metastone for the GLM5.3 model, under tested GPU cluster and load conditions, adopting predictive latency routing reduced P95 first-token latency by 32.4% and P50 first-token latency by 71.3%. Additionally, with flow-control coordinated scheduling, total output throughput increased by approximately 1.5%, while Goodput (qualified output throughput) rose by about 29.4%, with the qualification rate increasing from roughly 67% to 86%.

Metastone emphasized that these figures are vendor self-tested benefit estimation examples, and actual gains require verification against specific workloads and service thresholds. As an independent third-party full-stack compute operator, Metastone currently manages over 20,000P+ of computing resources, and its team has won three Gordon Bell Prizes.