iPhone 18 Pro runs a 27B model on-device in demo, but memory is still the bottleneck
On-device model size moves up a tier, but 12GB of memory can't fit 2-bit quantized Bonsai 2 — watch whether Apple increases memory next.
ImportanceLocalEvidenceE3 inspectableWrite-upQuick
An iPhone 18 Pro has been shown running the 27B-parameter Bonsai-27B model fully offline, at roughly twice the speed of an iPhone 17 Pro.
The result comes from a demo by user Adrien Grondin on X, as reported by IT之家 ↗; it is not a standard benchmark, and the 2x speed claim has not been independently reproduced. The hardware basis is the A20 Pro's dual 16-core neural engine, 12GB of LPDDR5X memory, and 115.2GB/s of memory bandwidth.
The limit is just as clear: the 2-bit quantized Bonsai 2 is too large to fit entirely in 12GB of memory and loses performance as a result. The next on-device AI bottleneck has shifted from compute to memory capacity.