Large Language Models Can Self-Declare Attention Regions to Cut Long-Context Memory Bandwidth Pressure
The Declarative Attention protocol allows Gemma and Qwen models to skip most KV cache reads in zero-shot settings, significantly reducing memory bandwidth pressure for long texts.
ImportanceMaterialEvidenceE2 unreplicatedWrite-upQuick
Researchers have introduced the Declarative Attention (DA) protocol, which enables large language models to declare the context regions they need to attend to within their chain-of-thought, thereby skipping reads of irrelevant key-value (KV) caches. This approach aims to address the computational and memory bottlenecks caused by global attention layers scanning the entire cache during long-context processing.
In zero-shot evaluations on Gemma-4-31B and Qwen-3.6-27B, the DA protocol reduced total attended tokens by 45.8% and 19.6% respectively across 15 long-context tasks. Despite introducing extra decoding steps, the method achieved significant inference efficiency gains, with accuracy dropping only by 1.40 percentage points and 2.10 percentage points, a loss that shrinks as model scale increases.
The study, led by Namgyu Ho et al., is published as an arXiv preprint (version 2 updated on October 14, 2026). The results have not yet been independently replicated, but the code is available on GitHub.