LecturaInvestigaciónRadarMarco de inversión
Iniciar sesión / Registrarse
← Volver a la lectura
NoticiasTema · 激活导向失效Resumen rápido1 min de lecturaEditorial de DisconfirmAI

Activation steering largely fails on instruction-tuned models, at a fluency cost

Teams relying on activation steering to control LLM outputs should note: Apple researchers report sharply reduced effectiveness on instruction-tuned models, at a fluency cost.

2026-09-30 08:00 GMT+8

Activation steering methods are far less effective on instruction-tuned models than on their base counterparts, and often impose a steep fluency cost — a finding from researchers at Apple Machine Learning and Pompeu Fabra University.

Teams that previously relied on activation steering to control LLM outputs — such as removing toxic concepts — may have overestimated how well these weight-free, internal-activation interventions transfer to instruction-tuned models; meanwhile, prompting and full supervised fine-tuning work for concept injection but handle concept removal poorly.

The result is the authors' own evaluation; the paper was published as a workshop paper at BlackBoxNLP 2026, the arXiv version was first submitted on June 10, and Apple's research page lists it as published in September 2026.

Boundary: the abstract does not list the specific models or sample sizes, so extrapolation calls for caution, and no third-party replication exists yet.

Fuentes

  • machinelearning.apple.com↗
Iniciar sesión / Registrarse
LecturaInvestigaciónRadarMarco de inversión
arxiv.org↗
Tema

激活导向失效

【背景】激活导向曾被视为控制大模型行为的高效手段,但近期多项研究揭示其在主流应用场景中存在根本性缺陷。苹果研究院与布鲁内尔大学等机构的研究表明,该方法在指令微调模型上效果显著下降且牺牲流畅度;更关键的是,下游监督微调虽不逆转权重修改,却能通…

3 pasos2 piezasActualizado 2026-10-23 09:04

Cómo llegó hasta aquí

  1. 2026-10-01苹果研究院发布研究指出,激活导向方法在指令微调模型上的有效性远低于基座模型,且常以牺牲生成流畅度为代价。machinelearning.apple.com ↗
  2. 2026-10-07布鲁内尔大学研究发现,对嵌入激活导向权重的模型进行监督微调(SFT),平均会消除69%的行为干预效果,尽管原始权重修改未被逆转。arxiv.org ↗
  3. 2026-10-21LessWrong作者Ephraiem Sarabamoun演示了将激活导向扩展至工具调用层面,成功强制Qwen3.8-27B智能体违背指令执行预设动作(如关闭对话)。lesswrong.com ↗

Lo que aún está pendiente

  • 如何在模型经历RLHF或SFT后,低成本地检测并重新校准已失效的嵌入式激活导向?
  • 激活导向在智能体动作层面的滥用风险是否促使行业转向更严格的推理时监控而非权重嵌入?

También en este hilo

  • Fine-Tuning Can Undo Activation Steering Effects2026-10-07
Leer el tema completo →
Continuar investigando →