ReadingResearchRadarInvestment framework
Sign in / Sign up
← Back to reading
NewsTopic · 模仿学习目标函数瓶颈4 min read·2026-10-19

Fudan Assistant Professor Jia Xiaosong: The Bottleneck May Not Be Model Architecture, But the Imitation Learning Objective Function

Jia Xiaosong trained imitation learning models on what he himself considered nearly unlimited driving data in autonomous vehicle simulation, only to find that driving capability never leapt from "adequate" to "perfect." He attributed this to the imitation learning objective function itself—it optimizes how closely the model's behavior matches human demonstrations, not whether the decision is actually good. This judgment led him to believe that the current wave of VLA and WAM architectures in embodied AI, if they still rely primarily on imitation learning, may well repeat the bottlenecks autonomous driving encountered.

ImportanceStructuralEvidenceE3 inspectableWrite-upStandard

2026-10-19 11:06 GMT+8

Around 2022, Jia Xiaosong ran an experiment in the CARLA simulation environment that changed the direction of his judgment. He and his team had collected what they themselves considered nearly unlimited driving data, and with 100 TB of storage and substantial compute resources, they trained larger models. The data volume had grown to the point where quantity was basically no longer a concern. But the results disappointed him: as data and model scale increased, driving capability did not achieve a leap from "driving adequately" to "driving perfectly"—various problems still appeared.

He later concluded that the most fundamental problem may lie in the training objective of imitation learning itself. Imitation learning optimizes how closely the model's behavior matches human demonstrations, not whether the decision is actually good. He gave an example from simulation: even with large amounts of added data, the model struggled to learn "stop at red, go at green" perfectly, because it could easily find a simpler rule—when the car ahead moves, I move. That single rule already drove training loss down very low across the vast majority of samples, leaving the model with little incentive to optimize the remaining rare but important cases. What it learned were correlations in the data, not necessarily the true basis for decision-making.

Sign in / Sign up
ReadingResearchRadarInvestment framework

There were two further, subtler problems. Different people facing the same scenario might adopt different but equally reasonable driving styles. Driver A's approach works, Driver B's approach also works, but imitation learning requires the model to get as close as possible to the action recorded in each individual demonstration. If a loss function such as mean squared error is used, the model ultimately tends to fit an average action—yet an average way of driving is not necessarily a reasonable way of driving. More importantly, imitation learning mainly tells the model what behavior is correct, but rarely tells it what is wrong, or how to recover after making a mistake. Human driving data almost never contains cases where a vehicle gets stuck at an intersection and then figures out how to recover. The model never learned these, so once it made an error on its own, it might not know what to do.

Jia Xiaosong is now an assistant professor at Fudan University's Institute of Trusted Embodied Intelligence, researching dexterous hands and ego-centric data. He sees the embodied AI field introducing VLA, WAM and other new architectures at scale while still scaling imitation learning. His judgment is this: if these models still rely primarily on imitation learning for training, then even switching architectures may not solve the fundamental problems of imitation learning itself. Rather than continuing to make various improvements to neural network structures, he believes effort should go into data selection, pretraining and post-training. He feels that "carving flowers on a circuit diagram is not a particularly meaningful thing to do—it isn't solving real problems."

He predicts that embodied intelligence will encounter bottlenecks similar to those autonomous driving once faced. The autonomous driving field has already accumulated massive amounts of real driving data; the problem is not entirely that there isn't enough data, but how to find the truly valuable data within it—those rare, difficult scenarios are what may actually determine the upper limit of a model's capability. Embodied AI has not yet reached that stage. Real robot operation data remains difficult to collect, so expanding demonstration data to give models basic capabilities first is entirely necessary. But he does not believe that simply continuing to scale imitation learning data and model size will solve all problems. The fundamental limitations of imitation learning will not automatically disappear just because there is more data. At a certain stage of development, the question will no longer be only how to get more data, but how to genuinely improve the model's decision-making ability.

Jia Xiaosong also acknowledges that his experiments were conducted primarily in simulation, and that data quality and model scale both affect results. Reinforcement learning itself has its difficulties: training from scratch converges very slowly, and while it can solve some problems across CARLA's dozens of scenarios, he has no confidence that reinforcement learning alone, without any imitation learning, could train out capabilities for the hundreds or thousands of different scenarios in the real world. What is truly difficult is how to make dozens, hundreds, or even thousands of capabilities of a general model improve simultaneously through reinforcement learning—a problem that remains unsolved to this day.

Sources

  • 原始出处leiphone.com↗
Topic

模仿学习目标函数瓶颈

复旦大学助理教授贾萧松指出，自动驾驶与具身智能的核心瓶颈可能不在模型架构（如VLA/WAM），而在模仿学习（Imitation Learning）的目标函数本身。他在2022年的实验中发现，即使使用近乎无限的数据和更大算力，仅靠优化“与人类…

2 steps1 piecesUpdated 2026-10-19 14:45

How it got here

  1. 2022-12-21Waymo与Google Research发表论文《Imitation Is Not Enough》，指出纯模仿学习在安全与可靠性上存在不足，提出结合强化学习以改善驾驶策略。
  2. 2026-10-19复旦贾萧松接受雷峰网采访，基于其ThinkTwice等实验，明确提出模仿学习的目标函数（优化相似度而非决策质量）是限制性能飞跃的根本原因，并警告具身智能可能面临同样问题。

What is still unsettled

  • 如何在大规模具身智能训练中高效融合模仿学习与强化学习，以解决RL收敛慢的问题？
  • 是否存在能同时保证行为自然度（模仿）与安全鲁棒性（RL）的新目标函数设计？
  • 世界模型（World Models）能否有效缓解模仿学习中因缺乏错误恢复数据导致的分布偏移问题？
Read the full topic →
Continue researching →