LecturaInvestigaciónRadarMarco de inversión
Iniciar sesión / Registrarse
← Volver a la lectura
NoticiasTema · 分块KV压缩技术Material1 min de lecturaEditorial de DisconfirmAI

ByteDance Research: Chunked KV Compression Means Token Position Determines Whether Information Is Readable

A paper from the ByteDance team reveals that models using chunked KV Cache compression, such as DeepSeek V4, suffer from phase sensitivity issues: a token's position within a compressed chunk determines whether its information is retained.

2026-10-10 19:40 GMT+8

Asking DeepSeek the same question twice, with only a few extra spaces added, can yield one correct answer and one completely wrong one. A paper released by the ByteDance team in late September, titled "Periodic Weak Spots: Phase Sensitivity from Chunked KV-Cache Compression," reveals the underlying cause of this phenomenon: chunked KV Cache compression technology, while alleviating VRAM pressure, introduces phase sensitivity—a token's position within a compressed chunk determines whether it can be read by the model.

Fuentes

  • leiphone.com↗
Tema

分块KV压缩技术

字节跳动Seed团队最新研究揭示,采用分块KV Cache压缩技术的模型(如DeepSeek V4系列)存在“相位敏感性”缺陷:Token在压缩段内的绝对位置决定其信息能否被保留。实验显示,同一问题因输入前缀长度微调(如增加空格),模型准确…

3 pasos1 piezasActualizado 2026-10-25 18:42

Cómo llegó hasta aquí

Iniciar sesión / Registrarse
LecturaInvestigaciónRadarMarco de inversión
  1. 2026-09-28字节跳动Seed团队提交论文《Periodic Weak Spots: Phase Sensitivity from Chunked KV-Cache Compression》,指出DeepSeek V4等模型因分块压缩导致检索准确率随Token位置周期性波动。alphaxiv.org ↗
  2. 2026-10-10雷峰网报道该发现,称其为DeepSeek的“隐藏Bug”,举例说明添加装饰性空格会导致代码补全结果从正确变为错误,且错误率与压缩步长相关。leiphone.com ↗
  3. 2026-10-25字节研究进一步确认,即使将DeepSeek-V4.1-Flash的压缩步长减半,相位敏感性差距虽缩小至6.1%,但仍未完全消除;从头训练的Qwen3基线模型复现了该现象。leiphone.com ↗

Lo que aún está pendiente

  • 动态分块技术(如ChunkKV)能否在不显著增加硬件开销的前提下,彻底解决固定步长带来的相位敏感性问题?
  • 主流大模型厂商是否会调整KV Cache压缩策略,或转向更复杂的稀疏注意力机制以规避此类结构性缺陷?
  • 现有的平均基准测试分数是否普遍掩盖了其他采用类似压缩技术模型的潜在位置依赖性风险?
Leer el tema completo →
Continuar investigando →