Large Language Models Can Self-Declare Attention Regions to Cut Long-Context Memory Bandwidth Pressure
The Declarative Attention protocol allows Gemma and Qwen models to skip most KV cache reads in zero-shot settings, significantly reducing memory bandwidth pressure for long texts.
Researchers have introduced the Declarative Attention (DA) protocol, which enables large language models to declare the context regions they need to attend to within their chain-of-thought, thereby skipping reads of irrelevant key-value (KV) caches. This approach aims to address the computational and memory bottlenecks caused by global attention layers scanning the entire cache during long-context processing.
In zero-shot evaluations on Gemma-4-31B and Qwen-3.6-27B, the DA protocol reduced total attended tokens by 45.8% and 19.6% respectively across 15 long-context tasks. Despite introducing extra decoding steps, the method achieved significant inference efficiency gains, with accuracy dropping only by 1.40 percentage points and 2.10 percentage points, a loss that shrinks as model scale increases.
The study, led by Namgyu Ho et al., is published as an arXiv preprint (version 2 updated on October 14, 2026). The results have not yet been independently replicated, but the code is available on GitHub.