読書リサーチレーダー投資フレームワーク
ログイン / 新規登録
← 読解に戻る
ニュースクイックテイク1 分で読めるDisconfirmAI 編集部

Fathom adapts K-cache bits per query, 1.67x faster decode step

Fathom lets each query adaptively choose how many bits of the 4-bit K cache to read, making Qwen3-8B million-token decode-step GPU time 1.67x faster than 136-bit sparse scanning, self-reported by the author

2026-09-21 12:00 GMT+8

In the million-token setting, each query can now adaptively decide how many bits of the 4-bit K cache to read: the author self-reports that Qwen3-8B decode-step GPU time is 1.67x faster than 136-bit sparse scanning, with the KV cache and index residing in host memory.

Previously, sparse scanning read the K cache at a fixed bit width and could not adjust the read amount per query.

The comparison is the author's own measurement: timing used synthetic KV, and in a real coding-agent session 92 bits already matched the step consistency of the most accurate scan; there is no speedup when the index is in GPU memory.

Single-author arXiv preprint (2609.17652), submitted September 15, updated to v2 on the 17th, not independently verified.

Sources: arXiv preprint ↗, author's code repository ↗

ソース

  • arxiv.org↗
  • github.com↗
ログイン / 新規登録
読書リサーチレーダー投資フレームワーク
リサーチを続ける →