Homelabber Runs 2.8-Trillion-Parameter Kimi K3 on CPU Using Intel Optane, Warns of Heat Risks
A Reddit user successfully ran Moonshot AI's 2.8T-parameter Kimi K3 model on a CPU-only setup using discontinued Intel Optane PMem 200, but required aggressive cooling to prevent damaging adjacent DRAM.
According to Tom's Hardware, a Reddit user in the r/homelab community demonstrated an experimental build running Moonshot AI's 2.8-trillion-parameter Kimi K3 model entirely on a CPU. The system utilizes a 40-core 3rd-generation Xeon Scalable processor paired with discontinued second-generation Intel Optane Persistent Memory 200 (PMem 200) modules. This configuration aims to bypass skyrocketing DRAM prices by leveraging cheaper secondhand Optane memory to reach the minimum 610GB capacity required for such large models.
However, the build faces significant thermal challenges. Optane DIMMs generate extreme heat during sustained inference, with individual modules consuming 12W to 15W. Without adequate cooling, this heat can endanger neighboring DDR4 server memory, potentially causing it to overheat or "burn out." To mitigate this, the builder installed dual 12W blower fans directed at the memory modules, advising others to treat active DIMM cooling as mandatory.
Performance remains sluggish, achieving only 0.85 tokens per second even with 1-bit quantization. This is significantly slower than previous builds using first-generation Optane combined with GPUs, which reached approximately 4 tokens per second. While too slow for real-time interaction, the setup may still serve overnight batch processing tasks. The current results are pre-tuning, and the developer plans to add GPU acceleration and optimize memory tiering in future iterations.