c't Reporter Tests Local LLMs: Memory Bandwidth, Not Compute Power, Determines Generation Speed
c't reporter Jan-Keno Janssen's hands-on tests of local large language models on the RTX 4090, RX 9070 XT, Intel Arc Pro B70, Mac Studio M5 Ultra, and AMD Strix Halo show that token generation speed is primarily determined by memory bandwidth. DDR5-5200 is more than 12 times slower than GDDR6X, explaining the massive gap between CPU and GPU inference. The Mac matches the 4090 during the generation phase but processes long prompts at less than one-third its speed—both phases require attention for coding agent workflows.
In a video published in November 2026, c't magazine reporter Jan-Keno Janssen released a series of hardware benchmarks for local large language models, with the core finding being that the data transfer rate of memory—not the floating-point compute power of the GPU—determines token generation speed. Using the standard llama.cpp tool to test Qwen 3.8-27B (Q4_K_M quantization), the RTX 4090 achieved 51.1 tokens per second, whereas running CPU inference on the same machine using only DDR5-5200 memory resulted in an output so slow it was visibly character-by-character—a difference of about 15 times. He explicitly pointed out that DDR5-5200 has a bandwidth more than 12 times slower than the 4090's GDDR6X, a ratio that largely corresponds to the difference in inference speeds.
This mechanism explains the practical trade-offs different hardware platforms face in coding agent workflows. The Mac Studio M5 Ultra (with 256GB of unified memory) is nearly on par with the RTX 4090 during the token generation phase, but achieves less than one-third of its speed during prompt processing (reading long code or documents): while the 4090 reads about 4,000 tokens of video text in just over a second, the Mac takes about 4 seconds. This makes little difference for pure chat, but coding agents need to repeatedly read codebases and documentation, meaning bottlenecks fall on bandwidth and compute power respectively. Although the AMD RX 9070 XT (€920, 16GB VRAM) ran at 34.4 tokens per second, the weights for Qwen 3.8-27B plus context almost completely filled the VRAM, leaving only about 8K tokens of usable context. It cannot simultaneously support image recognition and the 60K+ context required for coding agents—meaning it can run benchmarks but struggles with actual workloads.