LecturaInvestigaciónRadarMarco de inversión
Iniciar sesión / Registrarse
← Volver a la lectura
NoticiasTema · 声明式注意力协议Material1 min de lecturaEditorial de DisconfirmAI

Large Language Models Can Self-Declare Attention Regions to Cut Long-Context Memory Bandwidth Pressure

The Declarative Attention protocol allows Gemma and Qwen models to skip most KV cache reads in zero-shot settings, significantly reducing memory bandwidth pressure for long texts.

2026-10-16 12:00 GMT+8

Researchers have introduced the Declarative Attention (DA) protocol, which enables large language models to declare the context regions they need to attend to within their chain-of-thought, thereby skipping reads of irrelevant key-value (KV) caches. This approach aims to address the computational and memory bottlenecks caused by global attention layers scanning the entire cache during long-context processing.

In zero-shot evaluations on Gemma-4-31B and Qwen-3.6-27B, the DA protocol reduced total attended tokens by 45.8% and 19.6% respectively across 15 long-context tasks. Despite introducing extra decoding steps, the method achieved significant inference efficiency gains, with accuracy dropping only by 1.40 percentage points and 2.10 percentage points, a loss that shrinks as model scale increases.

The study, led by Namgyu Ho et al., is published as an arXiv preprint (version 2 updated on October 14, 2026). The results have not yet been independently replicated, but the code is available on GitHub.

Fuentes

  • arxiv.org↗
Tema

声明式注意力协议

KAIST AI与Google DeepMind联合提出“声明式注意力”(Declarative Attention, DA)协议,旨在解决大模型长上下文推理中KV缓存读取带宽瓶颈问题。传统稀疏注意力依赖外部代理打分,仍产生O(N)开销;D…

Iniciar sesión / Registrarse
LecturaInvestigaciónRadarMarco de inversión

3 pasos0 piezasActualizado 2026-10-18 08:46

Cómo llegó hasta aquí

  1. 2026-09-02论文《Language Models Can Control Their Own Attention》提交至arXiv,提出DA协议。arxiv.org ↗
  2. 2026-09-14科技行者发布深度解读,详述DA在Gemma和Qwen上的实测数据及vLLM集成细节。techwalker.com ↗
  3. 2026-10-14论文更新至v2版本,修正了部分实验数据的平均值计算方式。arxiv.org ↗

Lo que aún está pendiente

  • DA协议在小规模模型(如4B参数)上准确率保留率较低,是否可通过微调提升小模型的声明能力?
  • 在多轮对话场景中,频繁的模式切换标签是否会显著增加生成延迟,从而抵消带宽节省带来的收益?
Leer el tema completo →
Continuar investigando →