Preprint: LLM Working-Memory Bottleneck Lies in Readout Under Interference, Not Attention Precision

Large language models reproduce a human working-memory signature on the N-back task: performance declines as load increases, and errors concentrate on lures—items repeating the current stimulus at a neighboring but incorrect serial position. Two competing accounts have persisted: the precision hypothesis holds that softmax attention cannot concentrate weight on the item N positions back, retrieving adjacent items instead; the interference hypothesis holds that attention selects the correct item, but competing content corrupts the decision.

Dongyu Gong's arXiv preprint dissociates these accounts across 15 pretrained LLMs from three families (70 million to 14 billion parameters) and minimal transformers trained from scratch. In all three families, task-carrying attention heads attend to the target on most missed matches. Linear probes decode near-perfectly whether a matching letter occurred at the target or lure position, even when the model false-alarms to the lure.

Key intervention evidence: sharpening attention yields negligible gains, whereas suppressing the letter identity of non-target items improves sensitivity (d') in nearly all model-by-load conditions. In larger models, the task spreads over more heads and removing interference helps more. In minimal transformers, single-layer attention is precision-limited, but a second layer removes this constraint.

Conclusion: the working-memory limit of LLMs lies in reading items out under interference rather than in locating them, consistent with interference accounts of human working memory. The result is based on synthetic letter-sequence N-back tasks and has not been independently reproduced in natural-language or production inference settings.