EdgeWise: Step-Level RL Enables Local Small Models to Autonomously Delegate Critical Steps to Cloud for Cost-Efficient Long-Horizon Agents
The EdgeWise framework uses step-level reinforcement learning to train local small models to autonomously decide when to delegate critical steps to cloud LLMs, significantly reducing inference costs for long-horizon agents while maintaining performance.
ImportanceMaterialEvidenceE2 unreplicatedWrite-upQuick
Researchers proposed EdgeWise, an edge-cloud collaboration framework. The core idea is to deploy a small language model (SLM) on the local device as the primary solver, selectively delegating critical steps to a cloud LLM only when stronger model capacity is needed.
To enable autonomous delegation without introducing additional models at inference time, EdgeWise introduces a step-level reinforcement learning (RL) method. This approach jointly optimizes task solving and delegation decisions, with fine-grained credit assignment for individual delegation choices. This means the local SLM learns during training to identify which steps are "critical," allowing it to make optimal cost-performance trade-offs at runtime.
Experiments show that EdgeWise matches or outperforms pure-LLM execution while reducing inference cost by up to 62% on SWE-Rebench and 34% on Terminal-Bench 2.1. The study further deployed EdgeWise on a local workstation, demonstrating comparable cost savings in a practical edge-cloud setting.